Srini Srinivasan

Really interesting to see this: how Europe is moving up. It is interest to contrast this qualitative survey with the #siliconcatalyst analysys that UK is sixth in Europe in funding value fir silicon investment. And I will put in a plug for our #UKTIN Forward Capabilities paper on recommendations for UK Semiconductor sector: https://lnkd.in/eNNAsCAz

8

At Corgnit Research Lab, India, we are excited to share our latest findings on the convergence of Reinforcement Learning and Federated Learning and their implications for advancing AGI. Key Insights: - 𝐒𝐜𝐚𝐥𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐚𝐧𝐝 𝐄𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐜𝐲: The distributed nature of FL enables scalable and efficient training of RL models, significantly reducing computational overhead and accelerating convergence. - 𝐏𝐫𝐢𝐯𝐚𝐜𝐲-𝐏𝐫𝐞𝐬𝐞𝐫𝐯𝐢𝐧𝐠 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠: By ensuring that sensitive data remains local, FL facilitates the development of AGI in privacy-sensitve domains, such as healthcare, without compromising data security. - 𝐂𝐨𝐥𝐥𝐚𝐛𝐨𝐫𝐚𝐭𝐢𝐯𝐞 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠: Multiple RL agents can share knowledge through FL, enhancing overall learning efficiency and robustness without the need to share raw data. - 𝐏𝐞𝐫𝐬𝐨𝐧𝐚𝐥𝐢𝐳𝐞𝐝 𝐀𝐝𝐚𝐩𝐭𝐚𝐭𝐢𝐨𝐧: The integration of FL allows AGI models to be tailored to individual users or environments, combining global intelligence with local customization. - 𝐑𝐨𝐛𝐮𝐬𝐭𝐧𝐞𝐬𝐬 𝐚𝐧𝐝 𝐑𝐞𝐬𝐢𝐥𝐢𝐞𝐧𝐜𝐞: Aggregating diverse experiences through FL results in more generalized and resilient AGI models, mitigating the risk of overfitting. Benefits for Researchers: - 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞 𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐚𝐭𝐢𝐨𝐧: Leveraging distributed computational resources lowers costs and enhances the feasibility of large-scale AGI experiments. - 𝐄𝐧𝐡𝐚𝐧𝐜𝐞𝐝 𝐂𝐨𝐥𝐥𝐚𝐛𝐨𝐫𝐚𝐭𝐢𝐨𝐧: Model updates can be shared instead of raw data, promoting cross-institutional and interdisciplinary research. - 𝐀𝐜𝐜𝐞𝐥𝐞𝐫𝐚𝐭𝐞𝐝 𝐃𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭: Faster training times and rapid iteration lead to quicker advancements in AGI. - 𝐄𝐭𝐡𝐢𝐜𝐚𝐥 𝐀𝐈 𝐃𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭: Prioritizing user privacy and data security aligns with ethical standards and regulations, fostering responsible AI research. - 𝐑𝐞𝐚𝐥-𝐖𝐨𝐫𝐥𝐝 𝐀𝐩𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬: Immediate feedback and improvements from real-world data enable practical deployment in sectors like smart cities, healthcare, and IoT. Our research at Corgnit underscores the transformative potential of merging RL and FL in the pursuit of AGI. We believe these insights will catalyze further innovations and collaborations in the AI community. #ArtificialGeneralIntelligence #ReinforcementLearning #FederatedLearning #AIResearch #CorgnitResearchLab #PrivacyPreservingAI #ScalableAI #EthicalAI Corgnit, Corgnit Research India, Reinforcement Learning, Federated Learning Consortium

20

Very timely, and should be insightfully interesting debate. I think, its a must for any energy industry professional and stakeholder. Simple questions lead to complex answers: How come predictions and projections made by energy experts, just 20 years back (recall the early 2,000s) have all turned out wrong today!! What happened, here?? And, what makes us think that we may not be wrong again in 20 yrs from now. What would ENERGY 2040 look like? How do we successfully transition into a new era of safe, abundant, flexible, equitable and reliable energy?? Look forward to it.

1

1 Comment

MSR has showcased BitNet, a novel 1-bit Transformer architecture, designed to address the challenges associated with scaling LLMs. A key innovation in BitNet is the introduction of BitLinear, which replaces traditional linear layers and facilitates the training of 1-bit weights from the ground up. This architecture not only achieves competitive performance in language modeling but also significantly reduces memory usage and energy consumption when compared to advanced 8-bit quantization methods and FP16 Transformer baselines.   The architecture employs quantization-aware training, optimizing the model for low-precision representation from the outset, which enhances both accuracy and stability during the training process. BitNet features stacked blocks of self-attention and feed-forward networks with binarized weights, while retaining higher precision for other components, thereby minimizing computational costs. This design allows for efficient matrix multiplication with 1-bit weights, resulting in notable energy savings, especially as model sizes increase.   The training methodology incorporates techniques such as the straight-through estimator for gradient approximation and mixed precision training. These approaches enable BitNet to converge effectively with larger learning rates compared to traditional FP16 Transformers, leading to improved training efficiency and stability, which is particularly beneficial in resource-constrained environments.   Comparative evaluations indicate that BitNet consistently outperforms existing quantization methods across a range of benchmarks, including zero-shot and few-shot tasks, while also maintaining lower inference costs. These findings highlight the advantages of quantization-aware training over post-training quantization, positioning BitNet as a prominent approach for developing efficient LLMs.    Arxiv: https://lnkd.in/e-D959-U

3

Great comments from Vinod Khosla on the importance of autonomous systems. "Robotics is coming up with a Chat GPT movement." National sovereignty and global productivity will go to the nations that deploy the Intelligent Infrastructure enabling Industry 4.0. INTELLIGENT INFRASTRUCTURE will revitalize the nation: https://lnkd.in/gWTQ4a-b We are leveraging the Defense Industrial Complex to establish the INNOVATION Industrial Complex - Building a Stronger and More Connected Nation. Hundreds of leading scientists, engineers, developers, investors, and innovators are engaged. Are you? https://lnkd.in/gwv2pnXe INTELLIGENT INFRASTRUCTURE and autonomous robotics are the equivalent of the atomic energy race of the 1940s. Autonomy is not a feature, it is the North Star.  Autonomy will drive the largest productivity gain in history for the nation that leads. Autonomous robotics brings the power of automation from the factory directly to our communities and cities. It will bring back manufacturing to the United States and will empower individuals: independent manufacturers, farmers, contractors, drivers, engineers, and many more. Autonomy Institute https://lnkd.in/gr_av7CQ INTELLIGENT INFRASTRUCTURE, State of the Future, CONTINUUM of Innovation Capital, AUTONOMY PRIME, ATLAS PROGRAM, SKYHATTAN, Autonomous Nation #infrastructure #future #digital #innovation #data #people #visionzero #dataexchange #partnerships #supplychains #digitaltwin #privacy #investment #p3programs #northstar #leadership #innovation #investments #ai #robotics #productivity #computing #5g #construction #resilience

29

2 Comments

The 7th workshop on Neural Scaling Laws is about to start: https://lnkd.in/gQSB5k8v Scaling laws, emergency, alignment, transfer learning, with a focused track on multilingual models. Looking forward to kick it off with Irina Rish and discuss all these topics with Yuhai Tu, Paul Bogdan, Hailey Schoelkopf, Darshil Doshi, Rio Yokota, Neha Sengupta, Preslav Nakov, Tatiana Shavrina, PhD, Yishi Xu. Join in person or virtually! #icml2024 #ai

37

https://lnkd.in/esZgE24d Seven years ago I gave this talk at IIT Kanpur. Back then it seemed inevitable that we would have self-driving cars by now, and some predictions I made were wrong. Overall, though, I think this talk has aged reasonably well. Little did I know back then that the reference to the 2016 MaxSAT I gave would spark the creation of a new company, InsideOpt. The Seeker Solver that we sell is based on the technology outlined, and it is changing the optimization landscape. On the social implications, we since edited a special issue of the AI Magazine on the social disruption that AI brings. More work is needed. We need to have a wide societal discourse on how we are going to share the increasing wealth that AI brings to the world. #insideopt.com

8

Unravel the JOYA Team Spec Formation Process, the backbone of optical system development. Learn the crucial steps, risk management, and optimization techniques. When considering an optical system development process, it’s imperative to grasp the importance of the System Engineering backbone that follows this process and leads it through the different stages. One of the System Engineering tools that serves this purpose is the Spec (Specification Document). In this post we unfold this process and take you through it considering the following points: 🔹Optical Spec formation starting point 🔹What development question this process covers 🔹The output of the Spec formation process 🔹Detailed step-by-step explanation of the process 🔹Risk management: what may happen if you miss a step 🔹Applicable Tips: what can help you with Spec formation https://lnkd.in/d5VWDqkv #JOYATeam #systemengineering #developmentprocess #spec

24

3 Comments

My Video Podcast episode on Software Defined Vehicles from the Ojo-Yoshida report is now available without a registration wall on YouTube. I talk about what I think SDV should mean, why SDVs are cool, software issues, architecture, and more. youtu.be/PmTFgZyPIEc

24

1 Comment

A study in perspectives on AV safety: a Waymo robotaxi approaches an intersection in San Francisco. Some other driver is doing "really fast donuts" in the intersection, and has been doing so long enough that the robotaxi rider looks up because of the smell of burning rubber entering the cabin. Robotaxi enters the intersection, then stops. The other driver halts and narrowly misses hitting the robotaxi (by perhaps a foot). The rider: "I can trust the Waymo, but I can't trust this driver". States that the other driver was upset but the robotaxi rider "shrugged, and pointed to the invisible driver." What ya' gonna do? It's a Waymo. "Still going to use Waymo." Presumed Waymo response to regulators and media: The human driver doing donuts was irresponsible. If there had been a crash it would have been the other driver's fault. (If I got this wrong would be happy to be corrected by Waymo. They have not commented as far as I know.) A broader safety engineering response: This is not the first time a robotaxi has encountered another car doing donuts in an intersection. In fact, a Cruise robotaxi was hit in a similar situation in March 2023 (SGO 30412-4960). From the description it seems there was plenty of warning this was happening. Why did the robotaxi not stop well clear of the intersection as a prudent human driver would have? Is Waymo monitoring SGO data and learning from what has been reported by other companies? Has Waymo included "other car doing donuts" in its hazard log? Did Waymo decide to ignore mitigation of this hazard for some reason? Will that change now that this has happened to Waymo instead of just Cruise? I'm glad in this case there was no injury, but there certainly could have been but for chance from a rare circumstance that is already known to have caused a crash. Being saved by the “grace of the Gods” is not a safety argument. I hope these questions are being asked and resolved by Waymo. But we won't know until the next donut incident. Primary Source: https://lnkd.in/eFHEMVFX Article: https://lnkd.in/eFVpfjec

65

14 Comments

I am advising a pre-A deep tech company that is revolutionizing the audio industry by developing an SDK for GPU (NPU/TPU) chips found in phones, laptops, cars, and the cloud. They foresee a trend called Accelerated Computing, similar to the transformation seen in graphics and machine learning, but for audio. This shift will revolutionize how we record, edit, perform, perceive, and experience audio. Currently, audio is static, blended, and plain. However, with real-time processing, it can become dynamic, immersive, spatially directed, and emotionally impactful. The company has: ✔️ A legal entity, cap table, and IP in Switzerland, with plans to restructure and move to Delaware in preparation for Series A. ✔️ A team of 16 members, including 10 PhDs in computer science or applied mathematics. ✔️ A very sexy business model based on licensing fees, allowing them to sell to enterprises without dealing with customer-facing processes such as sales and support. ✔️ A main product, the SDK, enables advanced computing on devices like GPUs, offering capabilities, features, and functionalities that current hardware and systems cannot implement or run at the user end. ✔️ Proven success in the Pro Audio industry, with entry into the Automotive and Game Development sectors. They also have a promising future in the Entertainment industry. Previously raised $8M+ led by the top-notch VC, and are successfully closing pre-A funding of $2.5M. We are onboarding 50% of the round and have oversubscribed the rest. However, until the funds are in the bank account, we can negotiate with the founder. I have a small allocation for my community of $300k. DM me if you want to see my investment memo.

19

2 Comments

We are excited to back Morphing Machines. As Dr. Ranjani, CTO and co-founder says: “For the first time in India and globally, we have a processor that is truly runtime reconfigurable, catering to emerging workloads.” While Nvidia’s growth and high valuation are driven by robust demand for “infra-compute” in high-performance GPUs, spurred by Gen AI, there is a distinct need for innovative solutions tailored to a new wave of reactive applications. Enter Morphing Machines. It addresses this demand with its unique product designed for next-gen telecom, autonomous vehicles, collaborative robots, avionics, and online data analytics. This product is specifically engineered to handle heterogeneous workloads and diverse tasks with varying performance demands simultaneously, ensuring a high level of scalability. As we move from an institutionally-backed research team at IISC to productization, Golden Sparrow is collaborating with Speciale Invest and others to support Deepak Shapeti, who joined as CEO three years ago after earning his Masters at Stanford. He brings not only technical expertise but also the business acumen necessary to guide the product to commercial success. Venkat Raju Afreed Faizan Anand Rao Ritik Singh Michael Marmor

68

5 Comments

This is a HUGE update for us! I’ve spent a ton of time with Keerti Melkote over the past few months and the energy is through the roof. Very very few founders have done what he’s done (starting a business in his garage and scaling it to over $5B in revenue). AI is in its infancy, and the next decade will be an unprecedented AI infrastructure buildout. Ion Stoica, Philipp Moritz, and I are thrilled to tackle that together with Keerti. Over the past year, we’ve enabled so many companies to push the frontiers of AI, #Ray is growing explosively, and we’re barely scratching the surface. ChatGPT and generative AI were an inflection point for our business. Since then, Ray has taken off. 💙 Pinterest runs 5000 training jobs and 300 batch inference jobs every month on Ray. 💙 Runway used Ray to help build Gen-3 Alpha. 💙 OpenAI used Ray to train GPT-4. 💙 Uber standardized model training on Ray. 💙 Amazon processes exabytes of data with Ray. 💙 Ant Group serves over one million requests per second with Ray. 💙 Similar stories abound at Canva, ByteDance, Instacart, Spotify, Niantic, Inc., Samsara, Netflix, Airbnb, Adobe, Shopify, Zoox, Snorkel AI, LinkedIn, Handshake, and many many more. 💙 Much more to share at #RaySummit in October. If you’re interested in helping build the foundational infrastructure powering AI, please reach out. We’re hiring across the board and would love to work together.

264

18 Comments

Venkat Tummalapalli and I were lucky enough to visit with one of our favorite customers this week. Team Nesh has been working with IEEE over the 2 years to enhance their #AskIEEE solution. #AskIEEE leverages Nesh’s GenAI capabilities to provide accurate and contextually rich answers to queries of IEEE’s vast library of technical literature in engineering and science. With Nesh fueling #AskIEEE, the organization’s 421,000 members gain streamlined access to the information they need from IEEE’s wealth of scientific content. This week, we met with the IEEE team in Toronto at their Board of Directors Meeting to showcase the benefits of a solution like #AskIEEE. Here are some of my takeaways from our conversations: 1. AskIEEE and Nesh have melded the concern of content distribution behind a paywall with the need for GenAI-powered productivity. 2. AskIEE helps IEEE’s global membership work seamlessly across timelines — enabling them to access the niche-knowledge they need even when other collaborators aren’t online. 3. And as always — GenAI enthusiasm continues. Companies and organizations should focus on meeting this enthusiasm from their members and employees. Start with defining a narrow use cases that can scale once proven. Great things ahead for #AskIEEE, the IEEE team Priscilla Amalraj, CAE, Rakesh Kumar, Steve Woods, Sampath Heragu, and Nesh! #GenAI #GenerativeAI #AI #ArtificialIntelligence #IEEE

46

1 Comment

I find this paper interesting from Prateek Verma at Stanford: https://lnkd.in/gCaXcU8w This brings signal processing to LLM. If signal representations can be drawn out of LLM weights, it can ride on lot of learnings from age old analog/digital processing field like how to cancel quantization noise in QLoRA using techniques like delta sigma noise cancellations done in ADCs

10

#ObsessivelyEnthusiasticInvestor #UnreasonablyEnthusiasticInvestor #IrrationallyEnthusiasticInvestor at REFASHIOND Ventures: The Industrial Transformation Fund. We look for the people we believe are doing the best work in Industrial Transformation & Supply Chain Technology; Attempting the impossible. Then we go all-in on them. With obsessive and unreasonable enthusiasm. Join us. #IndustrialTransformation #SupplyChain #Innovation #Technology #Startups #VentureCapital #DemandTheImpossible #UnreasonableEnthusiasm #ObsessiveEnthusiasm #IrrationalEnthusiasm

14

1 Comment

Lately, I have been inspired by the neural operator work and research coming out of Caltech led by Anima Anandkumar. I was reflecting today that perhaps we aren’t approaching the energy storage and density dilemma in the right way. We tend to only think in terms of battery chemistry or fossil fuels, but what if there’s a third category that we haven’t yet discovered? And what if we could unlock that discovery using AI-driven methods? I don’t think most people fully grasp the staggering difference in energy densities between batteries and fossil fuels. If we want to fully transition to renewable energy, we need to make significant investments in advancing battery chemistry or discovering an entirely new approach. Batteries - Alkaline (zinc-manganese dioxide): 0.4-0.59 MJ/kg - Zinc-carbon: 0.13 MJ/kg - Nickel Cadmium (NiCd): >0.14 MJ/kg - Lead Acid: >0.14 MJ/kg - Nickel Metal Hydride (NiMH): >0.36 MJ/kg - Lithium-ion batteries 0.36-0.875 MJ/kg (theoretical limit 43.2 MJ/kg) Fossil Fuels - Coal 24 MJ/kg - Gasoline 46 MJ/kg - Uranium-235 3,900,000 MJ/kg (this is crazy!) What excites me about Anima and her team's work is how using neural operators they've been able to make monumental increases in modeling speeds. For instance, AI-based modeling using their neural operators is 45,000x faster than current weather forecasting models. There is so much more we could learn about how different elements and natural factors play roles in areas such as electromagnetism, electrostatic forces, subatomic electronic charge, and much more. #AI #energy #modeling #batteries #research https://lnkd.in/enE-U3uN

4