Sailesh Kumar

We are excited to back Morphing Machines. As Dr. Ranjani, CTO and co-founder says: “For the first time in India and globally, we have a processor that is truly runtime reconfigurable, catering to emerging workloads.” While Nvidia’s growth and high valuation are driven by robust demand for “infra-compute” in high-performance GPUs, spurred by Gen AI, there is a distinct need for innovative solutions tailored to a new wave of reactive applications. Enter Morphing Machines. It addresses this demand with its unique product designed for next-gen telecom, autonomous vehicles, collaborative robots, avionics, and online data analytics. This product is specifically engineered to handle heterogeneous workloads and diverse tasks with varying performance demands simultaneously, ensuring a high level of scalability. As we move from an institutionally-backed research team at IISC to productization, Golden Sparrow is collaborating with Speciale Invest and others to support Deepak Shapeti, who joined as CEO three years ago after earning his Masters at Stanford. He brings not only technical expertise but also the business acumen necessary to guide the product to commercial success. Venkat Raju Afreed Faizan Anand Rao Ritik Singh Michael Marmor

68

5 Comments

The 7th workshop on Neural Scaling Laws is about to start: https://lnkd.in/gQSB5k8v Scaling laws, emergency, alignment, transfer learning, with a focused track on multilingual models. Looking forward to kick it off with Irina Rish and discuss all these topics with Yuhai Tu, Paul Bogdan, Hailey Schoelkopf, Darshil Doshi, Rio Yokota, Neha Sengupta, Preslav Nakov, Tatiana Shavrina, PhD, Yishi Xu. Join in person or virtually! #icml2024 #ai

37

A good read how to do performance tuning for serving AI models. “To summarize, the performance bottleneck of vLLM is mainly caused by the CPU overhead that blocks the GPU execution.” If we can model everything beforehand, we should not wait for one year to locate the bottleneck. There are still a lot of cross-layer optimization opportunity for AI model serving. With hardware software co-design, there are more rooms for improving efficiency.

11

An IEEE SA whitepaper authored by ADWG members have been published. -Simulation, Testing, Verification, and Validation (STV2) of Autonomous Driving. It can be downloaded at https://lnkd.in/gvVsF5P3.

10

Check out this technical conversation between Dr. Waku and Groq Compiler Engineer Gautam Rayaprolu. They cover how inference works on GPUs v LPUs, issues of reliability in large clusters, the Groq approach to building data centers, & more. https://hubs.la/Q02RqZR10

39

The session at re:Invent “how Netflix benchmarks Foundation Models (FMs) and LLMs across hardware chipsets” by Amit Arora and Amer Ather highlighted some critical considerations for model benchmarking, including inference server configurations, resource optimization, and performance metrics across different hardware chipsets. Benchmarking and evaluating models becomes essential when users take their applications external facing. When this happens, three aspects become the top of mind for them: Inference at scale, cost to serve and model accuracy. To choose the right model for your generative AI workload, it is essential to target all three vectors: Performance efficiency (latency optimizations, token throughput), Cost optimization (dollar cost per transaction for a generative AI workload that serves a given number of users) and model accuracy (either on domain specific or open source data). In the session, FMBench is highlighted as a tool that targets all three vectors above and provides the most optimal recommendations that users can streamline their genAI decision making with. FMBench offers a comprehensive feature set, offering customization through YAML files, support for multiple inference engines (Triton, vLLM, TensorRT, DJL, HF TGI), deployment support for AWS silicon/GPU/CPU hardware and built-in analytics capabilities. What stands out is the flexibility it offers in benchmarking any model (this includes custom, fine tuned models), on open source/custom datasets, and evaluate those in a scalable way using a panel of LLM judges. You can use this tool today for your use case and benchmark for your generative AI requirements. Here are some helpful resources to get started: 1. Introductory Videos (https://lnkd.in/eXSTHeKd, https://lnkd.in/egzqbSa8): These are introductory/set up videos that contain information on FMBench, how to set it up and a walkthrough of a sample report. 2. FMBench Website (https://lnkd.in/eaEEXAPr): This website is fully searchable to ease common tasks such as installation, finding the right config file, benchmarking on various hosting platforms (EC2, EKS, Bedrock, Neuron, Docker), model evaluation, etc. 3. FMBench GitHub Repository (https://lnkd.in/eM8mmHrJ): The FMBench GitHub repository is open source. Get started with benchmarking and evaluating for your use cases today and reach out if you have any questions or ideas! Manju Prasad #AWS #MachineLearning #Benchmarking #GenerativeAI

141

8 Comments

Ai : Hunanity's greatest creation for Good and Bad. To secure ourself #zerotrustai In the journey, MS brings up VASA-1 will AI model which essentially spins a selfie image and turn it into a talking clip of you. All you have to do is upload a photo along with a voice note and let the AI model do the talking for you.

2

Cerebras Systems, under the leadership of CEO Andrew Feldman, made a significant leap in AI performance with its breakthrough in fast inference. By achieving lightning-speed processing of the world’s largest AI models, the company unlocked new possibilities for real-time, complex applications like multi-agent collaboration and advanced reasoning. This rapid inference technology is set to transform industries, enabling real-time responses and dramatically enhancing the capabilities of emerging AI systems. Feldman emphasized that this progress opens doors to use cases, driving the future of AI beyond simple query-based tasks. Read more here: https://lnkd.in/g__sm6Be

1

SEMICON WEST is on fire 24x7 Tuesday afternoon continued the focus on Front-End Manufacturing. Great session on wafer map visual analytics with Janhavi Giri, Intel, using self-supervised learning for image analysis, detecting failures and optimizing manufacturing processes. Naor Alfassi, Applied Materials, spoke about ML-guided Electron Beam review for wafermap image classification and defect detection. Janhavi Giri, PhD Naor Alfassi Sessions on virtual metrology from Srividya Jayaram, Siemens, on high product mix; and Mike Yim, Gauss Labs, on high-volume, from Lab to Fab. In our experience, virtual metrology has led to massive time savings in product testing.  Srividya Jayaram Mike Young-Han Kim One very fun and topical presentation was from Christoher Nguyen, Aitomatic, and Daisuke Oku, Tokyo Electron - announcing SemiKong, a new Open-Source Semiconductor LLM, jointly developed by Tokyo Electron, Aitomatic, FPT AI Center and the AI Alliance. This seems to be based on Meta's Llama3 with additional HTM context. Christopher Cuong T. Nguyen Daisuke Oku I just returned from visiting our HTM customers in Japan and Korea, where we showed our Spotfire Copilot running in Japanese and Korean .. and this presentation on SemiKong was a joy to see!! Feel free to check out my latest blog on the Japan visit including a Spotfire Seminar with our partner NTT Com and customers including Texas Instruments, Tokyo Electron, Toshiba, Sony and many others. MOC Blog on Spotfire Seminar in Tokyo, Japan https://lnkd.in/gAeSZSSC   Wednesday sessions in Smart Manufacturing focused on how AI/ML can optimize back-end semiconductor manufacturing process eg OEE, predictive and condition-based maintenance.. We are currently revamping our Spotfire OEE solution with Spofire Copilot, so this is quite timely. Shawn Sarbacker, Kulicke and Soffa,  gave a nice presentation on Semiconductor Assembly.  Shawn Sarbacker I really enjoyed the session on Zero Defect Manufacturing from Prasad Bachiraju, Onto Innovation. Totally aligned with our Spotfire work on visual data science for defect analysis .. interesting backend data integration .. we’ve been working on data federation and libraries of virtualization views for that part of the data and analytics flow.  Prasad Bachiraju I also liked the IBM WatsonX GenAI session from Megan Arp and Francois Bernard - many similarities to our Spotfire Copilot e.g. they refer to the “Watson Orchestrator” in an analogous architecture to our Spofire Copilot Orchestrator. Their LLM is able to predict user intent when a technician reviews a ticket (i.e., query vs. task). Tasks are handled through custom and prebuilt skills training, while queries are handled through RAG. Megan Arp Francois Bernard Spotfire #VisualDataScience https://lnkd.in/eSeVcE5h https://lnkd.in/ecWq3PSH

62

4 Comments

Researchers from NVIDIA, CMU and the University of Washington Released ‘FlashInfer’: A Kernel Library that Provides State-of-the-Art Kernel Implementations for LLM Inference and Serving FlashInfer incorporates a block-sparse format to handle heterogeneous KV-cache storage efficiently and employs dynamic, load-balanced scheduling to optimize GPU usage. With integration into popular LLM serving frameworks like SGLang, vLLM, and MLC-Engine, FlashInfer offers a practical and adaptable approach to improving inference performance. FlashInfer's unique features include: ✅ Comprehensive Attention Kernels: covering prefill/decode/append attention for various KV-Cache formats (Page Table, Ragged Tensor, etc.) for both single-request and batch-serving scenarios. ✅ Optimized Shared-Prefix Batch Decoding: 31x faster than vLLM's Page Attention implementation for long prompt large batch decoding. ✅ Efficient Attention for Compressed KV-Cache: optimized grouped-query attention with Tensor Cores (3x faster than vLLM's GQA), fused-RoPE attention, and high-performance quantized attention...... Read the full article here: https://lnkd.in/g69DqsgM Paper: https://lnkd.in/g4KK_9x4 GitHub: https://lnkd.in/gY5MeJYC NVIDIA NVIDIA AI Machine Learning Department at CMU Zihao Ye Lequn Chen Ruihang Lai Wuwei Lin Yineng Zhang Stephanie Wang Tianqi Chen Baris Kasikci Vinod Grover Arvind Krishnamurthy Luis Ceze

134

5 Comments

A talk, “Compiler-assisted Kernel-based P4 Pipeline Offloading Using Intel IPU” was presented at the 2024 P4 Workshop by Debashis Chatterjee and Neha Singh from Intel Corporation. The video and slides from their talk can now be viewed on-demand. Overview: Introduction to P4-TC, a bridge for growing the network programmability ecosystem. The P4-TC datapath is defined using P4 and it utilizes a Linux kernel native P4 implementation. Benefits to this approach - some mundane developer knowledge is automated into the compiler and it utilizes common interfaces for either software or hardware datapaths. Learn more! View video on-demand | https://lnkd.in/g-hkXY28  View slides | https://lnkd.in/gVn-NnDN #P4 #P4Workshop #P4lang #OpenSource

20

1 Comment

☯️ Tensor networks and quantum computing just go together -- the duo takes on simulations of Hamiltonian dynamics. Plus, PsiQuantum is officially headed to Chicago's quantum kingdom. ⚆ PsiQuantum collaborates with Illinois to build the first US-based utility-scale 1 million qubit fault-tolerant quantum computer at the forthcoming Chicago quantum campus, supported by a $500M state budget ⚆ Atom Computing introduces a method to maintain 99% occupancy in a 1225-site neutral-atom array; advances large-scale neutral-atom quantum computers one more bit ⚆ Nullspace, Inc. releases Nullspace ES -- a specialized software for designing and analyzing large-scale ion traps; accelerates simulations and reduces prototyping costs ⚆ QuTech's development of somersaulting spin qubits simplifies control in large semiconductor qubit arrays; uses baseband signals and small magnetic fields for efficient qubit management ⚆ IBM announces an exclusive Quantum Developer Conference in November; will include hands-on experiences with the latest quantum tools and opportunities for networking with industry experts ⚆ Despite growing quantum courses, disparities in resources and access persist, necessitating targeted investments in smaller, rural, and minority-serving institutions to encourage equitable quantum education ⚆ IBM Quantum researchers propose a hybrid method combining quantum computing and tensor networks to accurately simulate larger systems and longer times in out-of-equilibrium phenomena with manageable computational costs ⚆ Don't miss a single qubit -- follow me & subscribe to the free newsletter for your daily quantum computing news & research insights. #quantumcomputing #quantumtechnology #innovation #quantumscience #quantumphysics

15

2 Comments

Meet AGUVIS: A pure vision-based framework for autonomous Computer/Mobile-use GUI agents, operating seamlessly across web, desktop, and mobile platforms without UI code. Key Features & Contributions - Pure Vision Framework: First fully autonomous pure vision GUI agent capable of performing tasks independently without relying on closed-source models - Cross-Platform Unification: Unified action space and plugin system that works consistently across different GUI environments - Comprehensive Dataset: Large-scale dataset of GUI agent trajectories with multimodal grounding and reasoning - Two-Stage Training: Novel training pipeline focusing on GUI grounding followed by planning and reasoning - Inner Monologue: Explicit planning and reasoning capabilities integrated into the model training - Generalization to OOD (Out-of-Distribution): AGUVIS demonstrates impressive generalization on OSWorld benchmark even without desktop trajectory data during training. Project Page: https://lnkd.in/guzkwpbB Paper: https://lnkd.in/gs7wXyJS GitHub: https://lnkd.in/gWeSkFmK Collaboratiion among Yiheng Xu Zekun Wang Junli Wang Dunjie Lu Tianbao Xie Amrita Saha Doyen Sahoo Tao Yu Caiming Xiong

155

6 Comments

The age of optimization. We are in an interesting position in the AI space. The success of Nvidia has come at the expense of practical implementation. Nvidia has done an excellent job of tightly coupling compute and memory into a tiered pricing package. Want more compute? More memory comes along with it. For various technical and business reasons memory on Nvidia GPUs is not user upgradable. The side effect of this is companies are quietly or not so quietly building their own AI chips. With the hope of one day not relying on Nvidia. We are going to enter an AI winter if the cost of AI implementation does not make the use cases scalable. One of the contributing factors is the shortage of interdisciplinary data science and software development teams, these different skill sets can work in silos and have different often competing priorities. I can foresee some enterprising startups like Groc and others that don't necessarily need to be hardware focused, but rather focused on driving down costs and improving efficiency start to make an impact and serve as a stop gap measure until better hardware is available. I think hardware and bloated software in the machine learning/deep learning space are two of the biggest problems facing the industry.

2

1 Comment

Interesting to see Silicon Labs claiming their position in the #ambientIoT ecosystem. Watch out for my forthcoming interview with Dhiraj Sogani their senior Director of Product Marketing for more on what they are doing and why. https://lnkd.in/gpcRxCdE

10

Flow Computing, 100X CPU performance company, a fairly new addition to our portfolio, just published their 4M€ pre-seed round. I ended up making pretty hefty predictions in the press release in relation to their impact on advancing the baseline performance of computing. Anyhow, I think it's true - at least what comes to next 10 years or so, possibly much longer. Lets hope so. :-D It's not just I don't want to turn out to be proven dumb optimist (which I perhaps am regardless), but this level of performance gains will have quite substantial effect on many industries... #flowcomputing #supercpu #parallelprocessing #computing #seriousperformance #cputech #deeptech I'm sure the entire Butterfly Ventures team is as thrilled as I am. In deed, super cool tech from VTT with solid scientific backing, peer reviewed articles, IP portfolio and excellent founder team. Also super happy to collaborate with Sarsia, Superhero Capital, Stephen Industries and FOV Ventures.

83

4 Comments

🌟 Exploring the Intersection of Business and Technology re AI! 🌟 Before my decade-long focus on AI, connecting technology and business in various domains has always been my passion; This is why I enjoyed this conversation with David Luan - No BS, No Twitter exaggerations, No clickbate - Just a super well informed, super smart David sharing his insights from years of hands-on experience: #Startegy, #Software, #Hardware, #Platforms, #ModelSpace and everything in between - LMK what you think of his projections. In This Episode: 1. The Biggest Lessons from OpenAI and Google Brain: What did OpenAI realize that no one else did that allowed them to steal the show with ChatGPT? Why did it take 6 years after the introduction of transformers for ChatGPT to be released? What are 1-2 of David’s biggest lessons from his time leading teams at OpenAI and Google Brain? 2. Foundation Models: The Hard Truths: Why does David strongly disagree that the performance of foundation models is at a stage of diminishing returns? Why does David believe there will only be 5-7 foundation model providers? What will separate those who win vs those who do not? Does David believe we are seeing the commoditization of foundation models? How and when will we solve core problems of both reasoning and memory for foundation models? 3. Bunding vs Unbundling: Why Chips Are Coming for Models: Why does David believe that Jensen and Nvidia have to move into the model layer to sustain their competitive advantage? Why does David believe that the largest model providers have to make their own chips to make their business model sustainable? What does David believe is the future of the chip and infrastructure layer? 4. The Application Layer: Why Everyone Will Have an Agent: What is the difference between traditional RPA vs agents? Why are agents a 1,000x larger business than RPA? In a world where everyone has an agent, what does the future of work look like? Why does David disagree with the notion of “selling the work” and not the tool? What is the business model for the next generation of application-layer AI companies? https://lnkd.in/dUpWWzvN

4

Thanks People+ai! Check out the video below to know what I am up to these days! I am building hardware to make AI accessible to all!

59

5 Comments