Swaroop Mishra

https://lnkd.in/eXbCsAnD “He who controls the GPUs, controls the universe.” Access to GPUs, and in particular those made by Nvidia, the leading supplier, is vital for any company that wants to be taken seriously in #ai. Nvidia’s dominance has driven its market value to more than $2trn. 'GPUs do the computational heavy-lifting needed to train & operate large #ai models. Yet, oddly, this is not what they were designed for. The initials stand for “graphics processing unit”, because such chips were originally designed to process video-game graphics. It turned out that, fortunately for Nvidia, they could be repurposed for #ai workloads. 'Might it be better to design specialist ai chips from scratch? That is what many companies, small & large, are now doing in a bid to topple Nvidia. Dedicated ai chips promise to make building and running ai models faster, cheaper or both. Any firm that can mount a credible threat will have no shortage of customers, who dislike its lofty prices and limited supplies. 'GPUs contain several thousand processing engines, or “cores”, which let them run thousands of versions of the same simple task at the same time. Running #ai models similarly involves running lots of copies of the same task in parallel. Figuring out how to rewrite ai code to run on GPUs was one of the factors that triggered the current ai boom. 'Yet GPUs have their limitations, particularly when it comes to the speed with which data can be shuffled on and off them... When training very large #ai models, some GPu cores may be idle as much as half of the time. Andrew Feldman, the boss of Cerebras, a startup based in Sunnyvale, CA, likens it to the gridlock in a grocery store on the day before Thanksgiving. “Everybody’s in a queue, so there are blockages in the parking lot, there are blockages in the aisles, blockages at the checkout. That’s exactly what’s happening with a gpu.” 'Cerebras’s response is to put 900,000 cores, plus lots of memory, onto a single, enormous chip, to reduce the complexity of connecting up multiple chips & piping data between them. 'Groq, a startup from Mountain View, CA, is taking a different approach. Its ai chips, called language processing units (lpus), are optimised to run #llms particularly quickly. 'Yet another approach is that taken by MatX, also based in CA. GPUs contain features & circuitry that provide flexibility for graphics, but are not needed for llms. The firm is working on getting rid of such unnecessary cruft.

13

3 Comments

Nvidia AI Releases Llama-3.1-Nemotron-51B: A New LLM that Enables Running 4x Larger Workloads on a Single GPU During Inference Nvidia unveiled its latest large language model (LLM) offering, the Llama-3.1-Nemotron-51B. Based on Meta’s Llama-3.1-70B, this model has been fine-tuned using advanced Neural Architecture Search (NAS) techniques, resulting in a breakthrough in both performance and efficiency. Designed to fit on a single Nvidia H100 GPU, the model significantly reduces memory consumption, computational complexity, and costs associated with running such large models. It marks an important milestone in Nvidia’s ongoing efforts to optimize large-scale AI models for real-world applications. A standout feature of the Llama-3.1-Nemotron-51B is its ability to manage larger workloads on a single GPU. This model allows developers to deploy high-performance LLMs in more cost-effective environments, running tasks that would have previously required multiple GPUs on just one H100 unit. ... Read our full article: https://lnkd.in/gYSYECxj Model: https://lnkd.in/guaA6CYC NVIDIA NVIDIA AI #llm #llms #ai

42

𝐔𝐧𝐢-𝐌𝐨𝐄-𝐯2: 𝐌𝐮𝐥𝐭𝐢𝐦𝐨𝐝𝐚𝐥 𝐋𝐚𝐫𝐠𝐞 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥 (𝐌𝐋𝐋𝐌) 🚀 Uni-MoE-v2, the latest iteration, handles audio, speech, images, text, and video. This framework enhances multi-GPU training and inference, speeding up optimization and expanding scale. 🌟 𝐊𝐞𝐲 𝐅𝐞𝐚𝐭𝐮𝐫𝐞𝐬: - Multimodal Integration: Modality-specific encoders and connectors create a unified representation. - Three-Phase Training:  1. Cross-Modality Alignment: Connectors map different modalities and languages into a unified space.  2. Expert Activation: Modality-specific experts trained with cross-modal data ensure deep understanding.  3. LoRA Fine-Tuning: Integrates and fine-tunes experts into LLMs on mixed multimodal data using LoRA. - Enhanced Architecture: DeepSpeed MoE distributes expert weights across multiple GPUs, improving load balance and parallel processing. - Optimized Distribution: LoRA-integrated MLP reduces computational complexity while maintaining effective weight distribution. 𝐏𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞 𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬: - Top-Tier Performance: Matches or surpasses other multimodal LLMs on 10 vision and audio tasks. - Benchmark Leader: Outperforms existing unified multimodal models on comprehensive benchmarks. 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞𝐬: - Paper: https://lnkd.in/gDfDFPPN - GitHub: https://lnkd.in/gFUBKjb6 - Project Page: https://uni-moe.github.io Uni-MoE-v2 sets new standards for unified multimodal AI. Check out the paper and project for detailed insights! 🌈 #UniMoE #MultimodalAI #MoEArchitecture #AIResearch #MachineLearning #NLP #ComputerVision #DeepLearning #TechInnovation #SRBlog

41

2 Comments

🌟 Exploring the Intersection of Business and Technology re AI! 🌟 Before my decade-long focus on AI, connecting technology and business in various domains has always been my passion; This is why I enjoyed this conversation with David Luan - No BS, No Twitter exaggerations, No clickbate - Just a super well informed, super smart David sharing his insights from years of hands-on experience: #Startegy, #Software, #Hardware, #Platforms, #ModelSpace and everything in between - LMK what you think of his projections. In This Episode: 1. The Biggest Lessons from OpenAI and Google Brain: What did OpenAI realize that no one else did that allowed them to steal the show with ChatGPT? Why did it take 6 years after the introduction of transformers for ChatGPT to be released? What are 1-2 of David’s biggest lessons from his time leading teams at OpenAI and Google Brain? 2. Foundation Models: The Hard Truths: Why does David strongly disagree that the performance of foundation models is at a stage of diminishing returns? Why does David believe there will only be 5-7 foundation model providers? What will separate those who win vs those who do not? Does David believe we are seeing the commoditization of foundation models? How and when will we solve core problems of both reasoning and memory for foundation models? 3. Bunding vs Unbundling: Why Chips Are Coming for Models: Why does David believe that Jensen and Nvidia have to move into the model layer to sustain their competitive advantage? Why does David believe that the largest model providers have to make their own chips to make their business model sustainable? What does David believe is the future of the chip and infrastructure layer? 4. The Application Layer: Why Everyone Will Have an Agent: What is the difference between traditional RPA vs agents? Why are agents a 1,000x larger business than RPA? In a world where everyone has an agent, what does the future of work look like? Why does David disagree with the notion of “selling the work” and not the tool? What is the business model for the next generation of application-layer AI companies? https://lnkd.in/dUpWWzvN

4

Learn about the latest NeMo Retriever NIMs and how they enable developers to deploy models with high accuracy.

4

We are all trying to keep up with the pace of #AI #innovation - well, here's a free course from Andrew Ng and Vasudev Lal to help you build LLM and Vision #GenAI pipelines. #IamIntel Cool stuff!

12

Sohu by Etched: A Revolution in AI Hardware 🚀 Sohu chip is breaking boundaries with up to 𝟱𝟬𝟬,𝟬𝟬𝟬 𝘁𝗼𝗸𝗲𝗻𝘀 𝗽𝗲𝗿 𝘀𝗲𝗰𝗼𝗻𝗱 throughput on models like LLaMA 70B! Key Highlights: 💥 𝗕𝗹𝗮𝘇𝗶𝗻𝗴 𝗦𝗽𝗲𝗲𝗱 & 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆:   Sohu delivers an order of magnitude more speed than NVIDIA’s top Blackwell B200 GPUs, with up to 10 petaflops per second (10,000,000,000,000,000 FLOPS) while consuming only 𝟭𝟬 𝘄𝗮𝘁𝘁𝘀 of power. 💥 𝗛𝗮𝗿𝗱𝘄𝗮𝗿𝗲 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗲𝗱 𝗳𝗼𝗿 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀:   Sohu is the world’s first transformer-specific ASIC (application-specific integrated circuit). By focusing entirely on transformers, Sohu’s streamlined design cuts out non-essential functions, offering far more FLOPS—achieving over 500,000 tokens per second on LLaMA 70B models. 💥 𝗠𝗮𝘅𝗶𝗺𝗶𝘇𝗲𝗱 𝗖𝗼𝗺𝗽𝘂𝘁𝗲 𝗣𝗼𝘁𝗲𝗻𝘁𝗶𝗮𝗹: Typical GPUs, like the H100, dedicate less than 𝟰% of transistors to matrix multiplication, while Sohu devotes everything to transformers. With 𝟭𝟰𝟰𝗚𝗕 𝗺𝗲𝗺𝗼𝗿𝘆 per chip, it can scale models with up to 𝟭𝟬𝟬 𝘁𝗿𝗶𝗹𝗹𝗶𝗼𝗻 parameters. 💥 𝗩𝗲𝗿𝘀𝗮𝘁𝗶𝗹𝗲 𝗔𝗜 𝗖𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝗶𝗲𝘀: Equipped with tools like speculative decoding, beam search, Monte Carlo Tree Search, and Mixture of Experts, Sohu is built for cutting-edge AI applications. Its open-source software stack invites developers to unleash new innovations. 💥 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆 𝗔𝘁 𝗦𝗰𝗮𝗹𝗲: An 8 x Sohu server replaces 𝟭𝟲𝟬 𝗛𝟭𝟬𝟬 GPUs, enabling efficiency and scalability for industries worldwide, from NLP to video processing and more. More on Sohu chip - https://lnkd.in/de2SCTVm

122

2 Comments

While a small number of mathematicians have since accepted that Mochizuki's papers prove the conjecture, other researchers say there are holes in his argument and it needs further work, dividing the mathematical community in two and prompting a prize of up to $1 million for a resolution to the quandary. https://lnkd.in/e9xfpHJz #InnovateForImpact #technology #tech #innovation #engineering #science #software #huawei #design #electronics #fourthindustrialrevolution #programming #coding #education #computer #mathematics #artificialintelligence #math #uni #student #college #phd #research #quantumcomputing #quantummechanics #physics #technology #quantumcomputer #industry40 #artificialintelligence #ai #machinelearning #emergingtechnologies #computerscience #maths #innovation #settheory #it #quantumtechnology #datascience #mit #sustainability #it #hydrogen #computerscience #logic #theoreticalphysics #data #finance #asic #infrastructure #manufacturing #digitaltransformation #futureofwork #datascience #teamwork

1

Modeling in the sciences is in fact almost always generative modeling. What a BOLD statement to start a paper with here --> 1.1 Motivation One major division in machine learning is generative versus discrimi- native modeling. While in discriminative modeling one aims to learn a predictor given the observations, in generative modeling one aims to solve the more general problem of learning a joint distribution over all the variables. A generative model simulates how the data is generated in the real world. “Modeling” is understood in almost every science as unveiling this generating process by hypothesizing theories and testing these theories through observations. For instance, when meteorologists model the weather they use highly complex partial differential equations to express the underlying physics of the weather. Or when an astronomer models the formation of galaxies s/he encodes in his/her equations of motion the physical laws under which stellar bodies interact. The same is true for biologists, chemists, economists and so on. Modeling in the sciences is in fact almost always generative modeling

1

1 Comment

TopoX is a great first step. I found the lack of tutorials on Topological Deep Learning and Topological Data Analysis is an hindrance to its adoption. Topics such as persistent homology and filtration methods can be challenging for beginners.

3

2 Comments

Biden to further limit Nvidia AI chip exports in final push Summary: In a last-minute geopolitical move, the Biden administration is expanding restrictions on AI chip exports, targeting China, Russia, and other adversaries while maintaining preferential access for US allies. These new rules—set to be announced on January 10, 2025—aim to control global AI development by limiting the flow of Nvidia's and AMD’s advanced AI chips to foreign markets. 🔹 New Export Control Framework Three-tiered system: 1️⃣ Unrestricted Access – The US and select allies (Germany, Netherlands, Japan, South Korea, Taiwan) retain full access. 2️⃣ Conditional Access – Most countries will face chip import limits but can qualify for exemptions by aligning with US security and human rights standards (VEU status). 3️⃣ Total Ban – China, Macau, and arms-embargoed nations will be completely cut off from high-performance AI chips. Stricter controls on closed AI model weights to prevent adversaries from fine-tuning AI models for sensitive applications. US to maintain a computing power advantage over the rest of the world. 🔹 Industry & Market Reactions Nvidia and AMD stocks dipped following the news, signaling concerns over revenue impact. Nvidia strongly opposes the move, arguing it will hurt US economic growth and global competitiveness. Chipmakers and data center operators will face new compliance hurdles and potential market realignments. 🔹 Strategic & Economic Implications AI chip supply chains will be reshaped, with China likely to accelerate domestic semiconductor development. The US is leveraging its dominance in AI hardware to secure its geopolitical influence. Global AI leadership battle intensifies as countries navigate the evolving landscape of tech restrictions. 💡 Will these curbs protect US national security, or will they backfire by driving China and others to innovate faster? Source: https://lnkd.in/gYAQPWgX #AI #Semiconductors #Nvidia #TechPolicy #Geopolitics #Biden #ChipWar #USChina #DataCenters #ArtificialIntelligence #ExportControls

25

The 7th workshop on Neural Scaling Laws is about to start: https://lnkd.in/gQSB5k8v Scaling laws, emergency, alignment, transfer learning, with a focused track on multilingual models. Looking forward to kick it off with Irina Rish and discuss all these topics with Yuhai Tu, Paul Bogdan, Hailey Schoelkopf, Darshil Doshi, Rio Yokota, Neha Sengupta, Preslav Nakov, Tatiana Shavrina, PhD, Yishi Xu. Join in person or virtually! #icml2024 #ai

37

𝗠𝗼𝗼𝗿𝗲’𝘀 𝗟𝗮𝘄 𝗶𝘀 𝗗𝗲𝗮𝗱. 𝗪𝗲𝗹𝗰𝗼𝗺𝗲 𝘁𝗼 𝘁𝗵𝗲 𝗔𝗴𝗲 𝗼𝗳 𝗛𝘆𝗽𝗲𝗿 𝗠𝗼𝗼𝗿𝗲’𝘀 𝗟𝗮𝘄! In just eight years, NVIDIA has achieved a 1000x increase in computational power—leapfrogging the performance gains once defined by Moore’s Law. In a recent interview on 'No Priors', Jensen Huang shared NVIDIA's key strategic priorities, revealing an exciting path for transformative AI. Here are the key takeaways: 1. 𝐌𝐨𝐯𝐢𝐧𝐠 𝐁𝐞𝐲𝐨𝐧𝐝 𝐌𝐨𝐨𝐫𝐞’𝐬 𝐋𝐚𝐰 𝐛𝐲 𝐂𝐨-𝐃𝐞𝐬𝐢𝐠𝐧𝐢𝐧𝐠 𝐇𝐚𝐫𝐝𝐰𝐚𝐫𝐞 𝐚𝐧𝐝 𝐒𝐨𝐟𝐭𝐰𝐚𝐫𝐞 Nvidia is targeting 2-3x performance gains each year—not just on individual chips but at scale—while reducing costs and energy by similar factors. This is made possible by co-designing both hardware and software, allowing Nvidia to break through traditional limits on speed and efficiency. 2. 𝐅𝐥𝐞𝐱𝐢𝐛𝐥𝐞 𝐈𝐧𝐟𝐫𝐚𝐬𝐭𝐫𝐮𝐜𝐭𝐮𝐫𝐞 𝐟𝐨𝐫 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐚𝐧𝐝 𝐈𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞 Nvidia’s adaptable infrastructure seamlessly switches between training large models and running them in real-time (inference). This flexibility maximizes efficiency and makes AI more accessible across industries. 3. 𝐀𝐈 𝐅𝐚𝐜𝐭𝐨𝐫𝐢𝐞𝐬 𝐭𝐡𝐚𝐭 𝐠𝐞𝐧𝐞𝐫𝐚𝐭𝐞 𝐭𝐨𝐤𝐞𝐧𝐬 𝐨𝐟 𝐯𝐚𝐥𝐮𝐞 Jensen described treating traditional data centers as “AI factories.” These aren’t meant for just storing data; they’re producing intelligence. AI factories generate "tokens" — the building blocks for language models, robotics instructions, synthetic molecules, and more. Every industry will need its own intelligence generator. 4. "𝐃𝐚𝐭𝐚 𝐂𝐞𝐧𝐭𝐞𝐫 𝐚𝐬 𝐚 𝐏𝐫𝐨𝐝𝐮𝐜𝐭" 𝐀𝐩𝐩𝐫𝐨𝐚𝐜𝐡 Nvidia now builds entire data centers with compute, networking, cooling, and optimized software, delivering real-world performance that matches its promises. This “data center as a product” approach makes Nvidia’s tech scalable and customizable, sold as modular components that integrate seamlessly into diverse environments. 5. 𝐀𝐈-𝐃𝐫𝐢𝐯𝐞𝐧 𝐂𝐡𝐢𝐩 𝐃𝐞𝐬𝐢𝐠𝐧 𝐭𝐨 𝐝𝐫𝐢𝐯𝐞 𝐢𝐦𝐩𝐚𝐜𝐭 𝐢𝐧𝐭𝐞𝐫𝐧𝐚𝐥𝐥𝐲 𝐚𝐭 𝐍𝐯𝐢𝐝𝐢𝐚 AI isn’t just a tool at Nvidia—it’s a collaborator. AI-driven chip design allows Nvidia to explore more complex possibilities and optimize chips in ways that human engineers alone couldn’t. This virtuous cycle of AI designing for AI amplifies innovation. 6. 𝐓𝐡𝐞 𝐅𝐮𝐭𝐮𝐫𝐞: 𝐀𝐈 𝐄𝐦𝐩𝐥𝐨𝐲𝐞𝐞𝐬 & 𝐑𝐨𝐛𝐨𝐭𝐢𝐜𝐬 Jensen envisions a future workforce with “AI employees”—intelligent agents specialized in tasks like marketing, supply chain, and design. These aren’t mere assistants; they’re autonomous and capable, expanding the possibilities of AI to physical robots as well. The image was created with midjourney, showing a futuristic AI factory/ data center. A robot designing a chip. #generativeai #nvidiastrategy

12

4 Comments

Working on deploying ONNX models for performance-critical applications? Try our NVIDIA Nsight Deep Learning Designer ⚡ A user-friendly GUI and tight integration with NVIDIA TensorRT that offers: ✅ Intuitive visualization of ONNX model graphs ✅ Quick tweaking of model architecture and parameters ✅ Detailed performance profiling with either ORT or TensorRT ✅ Easy building of TensorRT engines Learn more and download ➡️

22

Apple stopped using Nvidia 💥 Tim Cook's masterstroke or blunder? 🤔 It's time to explore 3 key reasons behind this decision: 1. Strategic Shift to In-House Silicon • Tim Cook led Apple to develop its own silicon. • The M1 processor integrating CPU, GPU, and more. • This shift reduced reliance on 3rd Party suppliers like Nvidia. 2. Operational Efficiency and Supply Chain Management • Cook's expertise in supply chain management . • Developing its own silicon streamlined Apple's supply chain. • It reduced dependencies and mitigated risks from third-party suppliers. 3. Tactical vs. Strategic Leadership • Jensen Huang focuses on long-term AI and graphics innovation. • Cook's leadership is more tactical, optimising existing technologies. • This approach ensures operational excellence and incremental innovation. Apple's decision reflects Cook's leadership style: • Strategic control over technology and supply chain. • Operational efficiency. • Tactical innovation. Is this the right move? Time will tell. Share your thoughts on this Leadership decision by Tim in the comments. #Leadership #AI #Innovation #ManagementTips #CareerGrowth #LeadershipDevelopment #bitesize

9

4 Comments