Burak Yenier

New video on building a Q&A chatbot 💬 https://nvda.ws/3XaWchY ✅ 𝐄𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠 𝐂𝐫𝐞𝐚𝐭𝐢𝐨𝐧: Use NVIDIA NIM microservices to convert text into top-quality embeddings. ✅ 𝐕𝐞𝐜𝐭𝐨𝐫 𝐃𝐚𝐭𝐚𝐛𝐚𝐬𝐞: Store and retrieve embeddings efficiently with GPU-accelerated Milvus. ✅ 𝐈𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞 𝐰𝐢𝐭𝐡 𝐀𝐈 𝐚𝐭 𝐌𝐞𝐭𝐚 𝐋𝐥𝐚𝐦𝐚 3: Utilize the NIM API Llama 3 model for accurate responses to user queries. ✅ 𝐎𝐫𝐜𝐡𝐞𝐬𝐭𝐫𝐚𝐭𝐢𝐨𝐧 𝐰𝐢𝐭𝐡 𝐋𝐥𝐚𝐦𝐚𝐈𝐧𝐝𝐞𝐱: Seamlessly integrate and manage all components for a smooth Q&A experience.

3

Recommended viewing: Today's TEDTalk by Mustafa Suleyman on Future and Danger of AI. Having dedicated the last 14 years of my life and career to developing AI desktop software and smart sensor systems at TinMan Systems, I found today's TED Talk by Mustafa Suleyman particularly inspiring. He is an industry pioneer and his experienced viewpoint resonates with me deeply. He proposes that we consider AI as a new digital species, a concept that is both arresting and enlightening. This metaphor encourages us to ponder the true nature of what we’re creating—entities that are dynamic, emergent, and integrated into our lives in ways that traditional tools are not. Suleyman’s presentation is a persuasive reminder to ensure that human judgment remains the driving force in the creation of AI, while also being cautious of any unforeseen issues that might emerge. Of course his metaphor isn’t for literal interpretation, yet it sharpens our understanding of the essential matters concerning AI. It encourages us to reflect on the nature of AI we aim to develop and the guidelines we ought to establish. The statistics he presents are mind-blowing: advanced AI systems can ingest over 8 trillion words in a single month of training, a feat that far surpasses the human capacity for reading in a lifetime. This, along with the prediction that AI could soon be at the heart of every business, device, and building on the planet, paints a picture of a future where AI’s role is not just functional but also deeply personal. Yet, this doesn’t fully capture the essence of AI. These entities have memory, personality, creativity, and the ability to act autonomously. They are far more sophisticated than any tool we’ve ever known. This realization is not meant to alarm but to inspire a thoughtful approach to AI development. The potential benefits of AI are immense, particularly in solving critical issues in healthcare, education, and the climate crisis. As we stand on the brink of what could be the most productive decade in human history, we must choose wisely the AI we bring into the world. This is a story still unfolding, and nothing should be accepted as a given. We all have a role to play in shaping the future of AI—a future that promises to be radically better if we navigate it with care and intention. For those interested in exploring these ideas further, I have attached the link to the 22-minute YouTube video of Suleyman’s talk. It’s a must-watch for anyone invested in the future of AI and technology. #AIDevelopment #DigitalSpecies #HumanAgency #FutureOfAI #EthicalAI #AIStatistics #Innovation #TechnologyEvolution #MustafaSuleyman

1

The most important new product from NVIDIA that is not getting enough attention is NIM. And the most important element of NIM that is not getting enough attention is Kubernetes. NIM is the tip of the spear of Nvidia's software strategy (so it does not become Cisco) Unpacking this and more in my new (premium) post. https://lnkd.in/eFQcV9sK

14

2 Comments

NVIDIA Acquires Run AI: Impact on #AI Industry: NVIDIA's Bold Move: What the Acquisition of Run AI Means for the #AILandscape NVIDIA has officially closed the deal on its acquisition of Israeli software #startup Run AI, and let me tell you, this is a game-changer in the world of artificial intelligence. As the chief editor at Mindburst.ai, I’m thrilled to dive into the implications of this acquisition, especi... https://lnkd.in/gbXFsJB4

4

https://lnkd.in/eXbCsAnD “He who controls the GPUs, controls the universe.” Access to GPUs, and in particular those made by Nvidia, the leading supplier, is vital for any company that wants to be taken seriously in #ai. Nvidia’s dominance has driven its market value to more than $2trn. 'GPUs do the computational heavy-lifting needed to train & operate large #ai models. Yet, oddly, this is not what they were designed for. The initials stand for “graphics processing unit”, because such chips were originally designed to process video-game graphics. It turned out that, fortunately for Nvidia, they could be repurposed for #ai workloads. 'Might it be better to design specialist ai chips from scratch? That is what many companies, small & large, are now doing in a bid to topple Nvidia. Dedicated ai chips promise to make building and running ai models faster, cheaper or both. Any firm that can mount a credible threat will have no shortage of customers, who dislike its lofty prices and limited supplies. 'GPUs contain several thousand processing engines, or “cores”, which let them run thousands of versions of the same simple task at the same time. Running #ai models similarly involves running lots of copies of the same task in parallel. Figuring out how to rewrite ai code to run on GPUs was one of the factors that triggered the current ai boom. 'Yet GPUs have their limitations, particularly when it comes to the speed with which data can be shuffled on and off them... When training very large #ai models, some GPu cores may be idle as much as half of the time. Andrew Feldman, the boss of Cerebras, a startup based in Sunnyvale, CA, likens it to the gridlock in a grocery store on the day before Thanksgiving. “Everybody’s in a queue, so there are blockages in the parking lot, there are blockages in the aisles, blockages at the checkout. That’s exactly what’s happening with a gpu.” 'Cerebras’s response is to put 900,000 cores, plus lots of memory, onto a single, enormous chip, to reduce the complexity of connecting up multiple chips & piping data between them. 'Groq, a startup from Mountain View, CA, is taking a different approach. Its ai chips, called language processing units (lpus), are optimised to run #llms particularly quickly. 'Yet another approach is that taken by MatX, also based in CA. GPUs contain features & circuitry that provide flexibility for graphics, but are not needed for llms. The firm is working on getting rid of such unnecessary cruft.

14

3 Comments

These features help HPC users save 50-80% on EC2 compute costs...here's how they work -- Optimize is a feature we developed for stateful jobs running on EC2 On-Demand and migrates them to run on EC2 Spot instances: 1. This compute optimization feature uses our memory checkpointing technology to ensure the job doesn't lose state This is key because for long-running scientific computing jobs it is difficult to restart what could be hours, days or even weeks of runtime just to save $ 2. During extremely busy periods, EC2 Spot interruption rates can be as high as 50-80% and responding to each rebalance notification is very disruptive If you're running on a Spot instance that actually get reclaimed and our software can't easily find another Spot instance, we perform a stateful migration to on-demand instances and Optimize helps look for a Spot instance 10, 20, 30 mins later to move back to -- How many times have you run out of memory? Out of memory protection is a feature we developed for HPC jobs that are hard to deterministically predict memory requirements pre-run: 1. Being able to run on smaller EC2 instances and migrating to larger instances only when needed without losing state results in material cost savings 2. Our software monitors resource utilization on all the worker nodes and when we detect memory utilization approaching 100% and/or swap is hit, we perform a full application checkpoint, migrate you to the larger instance, and resume running -- Some of these capabilities that we have developed to optimize Cloud costs are also quite useful for on-prem HPC application. Drop a comment below if you have questions or shoot me a DM if you want to chat on how implement memory checkpointing in your HPC architecture. #AWS #HPC #SuperComputing #Slurm #PEARC24

11

2 Comments

It's great to see a specialization of Llama to semiconductor manufacturing technologies. The use of a neurosymbolic framework to boost problem-solving abilities is particularly interesting.

34

And with lowest power - uses only 16 RDU chips to achieve this performance!

58

1 Comment

A really goood DEEP review of the H100 and Nvidia A100 chip lineup and ecosystem. Secret: Most majors have not used H100’s for some time as they have major problems with crazy instant range. They fly to 100 percent down to zero in pulses almost. It kills electrical infrastructure as the grid and equipment is looking for consistency not pulses. Microsoft and others like Google, Meta and others have moved to the newer chips for better performance and stability. This is how x.ai was able to get so many H100’s. They are betting they can fix the issues and make it work. And are betting that having them in 1 facility vs many will give some advantage. X will be at 100k now and then 200k. They also built out in 120 days so it’s a Rush job. High risk with possibly pay out? Sound like anybody? Microsoft will be on newer and more like 1 million + better chips:) So x is basically working on left overs because it shows up to dinner table late and is eating left overs. But it sounds like they are not giving up. Expect black outs in and around facility in the short term! Can some better brains be built!! Obviously these are my opinions from experts and experience. Happy for corrections. Zachary Huhn Brian Burke Summer Crenshaw Joe Oquist💡 Jarrod Anderson Jerry Felix #genai #aichips #microsoft https://lnkd.in/ge9qx_Xz

14

8 Comments

What an event (based architecture) 🔥 We’re thrilled to share a new article by Yossi Rizgan, Engineering Group Lead at Gong, where he explores the architectural patterns and core building blocks behind our scalable, low-latency, and reliable communication capture systems. Dive into the details and discover how we’ve built these powerful systems!  Read the full article here 👉 https://lnkd.in/e2yXA2iD

10

Congrats Neural Magic -- especially Prof. Nir Shavit and Dr. Alex Matveev -- on the acquisition announcement by Red Hat (and parent IBM). Love it! It's great to see Nir, Alex, and company do amazing things. I'm super grateful to have worked with some of you. :) From all of us at Merly, thanks for building amazing technology that's advancing our scientific understanding and making AI more efficient. And, best yet, helping us reducing the computational footprint and friction of DNNs/LLMs. Your success is evidence that we *can* push the boundaries of science and achieve technological breakthroughs, while simultaneously reducing compute costs: financial, temporal, and environmental. The big cloud providers could learn a lot from you. You're role models to all of us, especially me! https://lnkd.in/grVUvD4Y

12

🔍 Key Papers to Demystify How GPT-o1 Works 1) Training Language Models to Self-Correct via Reinforcement Learning: https://lnkd.in/gwgdE5fb 2) STaR: Bootstrapping Reasoning With Reasoning: https://lnkd.in/gRdir7PP 3) Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking: https://lnkd.in/g47fqB7p 4) V-STaR: Training Verifiers for Self-Taught Reasoners https://lnkd.in/gGRRyBrF 5) Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters: https://lnkd.in/g9atmpvC 6) Chain of Thought Empowers Transformers to Solve Inherently Serial Problems: https://lnkd.in/gCyMncsf 7) Large Language Monkeys: Scaling Inference Compute with Repeated Sampling https://lnkd.in/ghy2r5nH 8) An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models: https://lnkd.in/gRHU2xyU 💡 Stay Tuned! I will continually update this post with more insights and a special focus on the "how" aspects of these papers.

20

2 Comments

Generative AI is unlocking new intelligent capabilities in applications, but its probabilistic nature and reliance on natural language bring unique risks. Using a layered mitigation mechanism helps in addressing errors, defending against adversarial attacks like jailbreaks and prompt injections, and reducing harmful content. In my latest blog, I explore key strategies for securing generative AI systems, guided by core Responsible AI principles. https://lnkd.in/gwUenSXz Lee Stott | Amy Kate Boyd | April Speight | Carlotta Castelluccio | Vivek Sridhar #Microsoft #AI #ResponsibleAI

30

1 Comment

The future of vector embedding storage is integrated, not isolated 🙌 Learn how new solutions are addressing traditional vector database challenges by treating embeddings as indexes. [Timescale Weaviate Pinecone LanceDB] 📈 https://lnkd.in/gRQg7Z_8

19

1 Comment

See our GPUaaS software demo

19

NVIDIA Acquires Run AI: Impact on #AI Industry: NVIDIA's Bold Move: What the Acquisition of Run AI Means for the AI Landscape NVIDIA has officially closed the deal on its acquisition of Israeli software startup #RunAI, and let me tell you, this is a game-changer in the world of artificial intelligence. As the chief editor at Mindburst.ai, I’m thrilled to dive into the implications of this acquisition, especi... https://lnkd.in/gbXFsJB4

1

1 Comment

Interesting news - bringing some container-based sanity to the heterogeneous edge, especially the tiny edge, has always been challenging. In an ideal world you’re managing these tiny assets like you would cloud assets - but we don’t live in an ideal world! If this can provide the right balance of performance, portability and openness amongst back end platforms - it could be quite impactful. #ochestration #edgecomputing #edgeai

7

1 Comment

Nvidia CEO Jensen Huang at the AGM yesterday: "By customer demand, ecosystem adoption and bookings, the Blackwell architecture platform will likely be the most successful product in our history and even for the entire computer and even of the entire computer history" #nvidia #wallstreet #earningscall #marketcap #artificialintelligence #stocks #stockmarket #investors #shareholders #nasdaq #nyse #technology #ai #innovation #business credit: the transcript

7

1 Comment