Vinayak Hegde

This is a story about why Claude 3.5 Sonnet > ChatGPT 4-turbo > Gemini Pro 1.5 (all free consumer versions) In the IIT Joint Entrance Exam which everyone takes to enter the IIT system in India, there was this beautiful last problem in the Math section when I took it. "Prove the larger of the two numbers e^Pi or Pi^e." Many friends I spoke to after the test had been stumped by it. My solution was to power both sides by 1/(Pi*e) and reduce the problem to finding the extrema of the function x^(1/x). From there it was straightforward first and second differentiation find that the maximum was at e and get to the result that e^Pi is greater than Pi^e. So I asked the question to Claude 3.5 Sonnet, ChatGPT 4-turbo, Gemini Pro 1.5 with the exact same prompts > you are an expert at calculus. you know how to do mathematical proofs. here is a problem: prove which one is the larger of the two numbers. e to the power Pi or Pi to the power e Claude understood right away to use calculus and to construct a function that would reflect the two values. Here is the snippet of the response Claude [ This is an interesting problem comparing e^π and π^e. Let's approach this step-by-step using calculus techniques. To prove which is larger, we'll use the natural logarithm function, which is strictly increasing. This means that for any a and b, if ln(a) > ln(b), then a > b. Let's define f(x) = x^(1/x) for x > 0. ] And then it did the first and second derivatives to find the maxima is at e and completed the proof. You can try this yourself. Very impressive. ChatGPT tried to do it numerically first. I had to redirect it to ask to solve with a proof. It then decided to create a function and proceeded to solve it using calculus to find first and second derivatives and find that the maximum was at e. Good work. Gemini on the other hand confidently provided a proof with total errors. It was well written, but completely wrong in a particular step. It tried simple log functions of both sides. I pointed out the error and it proceeded to create another proof that was equally flawed. Then it did it a third time and finally gave up. I then had to prompt it, like I might a high school calculus student, to guide it to the approach of creating a function and finding its extrema. > try this approach: power both sides by 1/(Pi * e). then we are asking the question whether e^(1/e) is greater or lesser than Pi^(1/Pi). this then reduces to the problem of finding the extrema of the function x^(1/x). can you prove it from here? It then understood the how to solve it and then actually did the proper steps of finding the first and second derivatives like a good guided high schooler might. There is another complex test I have done with creating CPUs and assembly code where Claude also performed better. Net result: Claude 3.5 Sonnet is very impressive and better than the other two in my smoke tests. Well done!

70

13 Comments

India is not just catching up in the AI race; we're innovating and leading in ways that are uniquely ours. The Economic Times recently highlighted how our approach to Generative AI goes beyond just Large Language Models (LLMs), addressing the specific challenges and opportunities within our diverse market. Rahul Agarwalla, Managing Partner at SenseAI Ventures, shared his invaluable insights into this AI revolution. His leadership and vision are truly driving innovation in the field, and it's an honour to be part of this journey with him. For anyone interested in the developments in AI and how India is making its mark on the global stage, this is a must-read.

46

3 Comments

A few weeks ago Meenal Nalwaya who heads up multimodal LLMs at Meta and I were chatting about what is going on in the world of agentic infra globally. There have been a bunch of opinions on what agents are, what they can do, but there was no clear canonical architecture that was emerging. Even doing something as basic as a personal agent to book "a flight ticket to SFO tomorrow" needs the agent to know what "today" is, where "here" is to book something from here to SFO, my flight + seat + meal preferences, my FF number, my ID & credit card info, then once booked, schedule an uber ride, get my boarding passes & place it on my calendar, and ensure a pickup & a hotel once I land, and so much more. A simple task like this has so, so many complexities. We felt we needed to dig into what is the architectural framework needed to solve some of this. We also felt that this problem needs a 2-sided approach - you can build a bicycle but then you also need the roads to build bicycle lanes or it doesn't work. While we believe this utopian world where agents can book us a flight, a car, a hotel, is still not here yet, we wanted to write a series of articles to help us think about the opportunities and challenges ahead. This is the first part of a series of articles, feedback & comments welcome - link in the first comment - and hope you enjoy the Homer Simpson references :) part 1: "Endless Homers, Infinite Agents":

474

32 Comments

Observability at the infrastructure and application levels must be harnessed to deal with today’s demanding workloads in #GenAI. Come talk to us to see how you can use the new AI with observability data to optimize your services.

17

Nice to have 'Top New Release' status on Amazon for the latest book (my 25th) that explores an alternative paradigm for quantum computation. Illustrations (111) by Dr. Narendra Madhav Joshi. My take is that we owe it to ourselves at this early stage of the industry's lifecycle to explore alternative approaches. Here is a brief overview: "This book attempts to reveal something of the vastly different and fundamentally creative quality of computing that must accompany any computation involving the quantum-levels. To linearly project digital computing laurels manifest as increasing speeds and the ability to process vaster amounts of information, as the inevitable trajectory of quantum computing is perhaps, in the aphorism attributed to the Buddha, to look only at the finger and to miss the moon and the sky that it is pointing to. The cover figure highlights all that is being missed, summarized as an egg-like structure synonymous with the term ‘Hiranyagarbha’ in Sanskrit. This depicts the womb of creation abundant with many layers and patterns reflecting luminescence, due to the constant superposition and entanglement inherent to the quantum computation that derives from a light-centered interpretation of quantum dynamics." The book is available here: https://lnkd.in/gXu4H9uW #quantumcomputation #quantumtech #technologyleadership #cosmology #systemsthinking

16

5 Comments

Watching Vinod Khosla talk with Rocky Yu about exponentials at the AGI House. Vinod said you could see the future of the Internet in the 1970s if you watched carefully. So, here are my guesses on what new fields AGI will revolutionize in what order: 1) Creative industries: image generation, music generation and movies 2) Robotics: foundational models will enable much better robotics to be deployed much more widely 3) Biotech: the creation of new drugs, better therapies etc…

34

1 Comment

https://lnkd.in/esZgE24d Seven years ago I gave this talk at IIT Kanpur. Back then it seemed inevitable that we would have self-driving cars by now, and some predictions I made were wrong. Overall, though, I think this talk has aged reasonably well. Little did I know back then that the reference to the 2016 MaxSAT I gave would spark the creation of a new company, InsideOpt. The Seeker Solver that we sell is based on the technology outlined, and it is changing the optimization landscape. On the social implications, we since edited a special issue of the AI Magazine on the social disruption that AI brings. More work is needed. We need to have a wide societal discourse on how we are going to share the increasing wealth that AI brings to the world. #insideopt.com

8

Great course by Andrew Ng and Shreya Rajpal on guardrails. And the best model on Guardrails hub for minimizing hallucinations is none other than our Bespoke-MiniCheck model. It is better and cheaper than frontier models. Get started with it today: https://lnkd.in/gKzciipz PS: If you are wondering how we beat GPT-4 or other frontier models for this task, it's basically cool synthetic data creation. We are cooking a library so you can also fine-tune models that beat frontier models on your narrow task. DM if you are interested in testing an early beta.

38

Grammarly just acquired startup Coda (last valued at 1.2bn) with a goal of a possible IPO. This could be a big move in productivity, especially for product teams. Coda CEO Shishir Mehrotra was recently on Lenny's podcast (first comment below). https://lnkd.in/gmgfcm2P

1

1 Comment

😲 OMG - "Are you human after all???" Carole Robin, Ph.D. models vulnerability in sharing one of many personal anecdotes in an hour chock full of powerful learning whether you're a leader or not. Lots of useful insights on relationships and communications for all. Some of my takeaways which is just a sampling of what you can learn in Carole's 1 hour segment with Lenny - one of the best hours you'll ever spend: 💡 Appropriate use of feelings in the workplace can build stronger connections 😧 Anger is a distancing emotion though. Use connecting emotions like happiness, joy, fear (sharing your fear not inflicting fear!), loneliness 📝 Give feedback in a way that builds relationships. All feedback is data. All feedback is positive - constructive or complimentary. ⚡ Listen in to learn what AFOG and "Stay on your side of the net!" mean. 🙂 Seriously it's worth your while! YouTube: https://lnkd.in/dTwniSUG Spotify: https://lnkd.in/dq26Ciww Apple: https://lnkd.in/dZsTvV34 Website: https://lnkd.in/dn8-hNfa

23

11 Comments

NVIDIA loses to Google, Microsoft, and Apple but still wins on What? Patents but Sometimes Quality > Quantity of patents They Say Nvidia is at the Right place at the Right Moment And Patents have a big role to play As a result, 30X growth in stocks in 5 years and the rally is still continuous Andrew Klein ran an analysis of Nvidia Patents about 2589 publications from “2023” and it displayed the keyword cluster map. The result This phrase was heavily used in ✅Neural Networks ✅Machine Learning ✅Data structure ✅Parallel Processing Damn, a perfect Recipe for 𝗔𝗿𝘁𝗶𝗳𝗶𝗰𝗶𝗮𝗹 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲

11

9 Comments

There's a common belief that scaling 3 dimensions - model size, training compute and size of the dataset will lead us to AGI. But, this is an oversimplified view of reality. Read an interesting piece on #AISnakeOil from Sayash Kapoor and Arvind Narayanan that critiques the widespread belief and provides a wonderful explanation. While scaling has improved model predictability, the real value lies in unexpected capabilities that appear as models grow. They term it as - the "Emergent Abilities" of the model. Undoubtedly, until now scaling up has led to remarkable improvements in the predictability of responses. While this is interesting from a technical standpoint, it might not necessarily be what end users care about. Maybe what really matters is the unexpected capabilities that models acquire as they grow in size. Another misconception is the role of synthetic data. While it’s often considered as a solution to the data scarcity problem, it’s primarily used to fill specific gaps or enhance domain-specific tasks, not to increase the overall volume of training data. In summary, the future of AI and LLM development isn’t about endless scaling. It’s about smarter, more efficient use of resources to unlock the best outcomes. Apologies for the long post but I found it super interesting. What are your thoughts? You can read the complete article here 👇

18

2 Comments

What if your queries required access to multiple sub-related data from different documents, for an LLM to parse and make sense of before returning a coherent and accurate output? We often ask complicated questions, right? 🔹Well to solve this problem, these researchers released Multi-Head RAG (MRAG) which leverages activations of Transformer’s multi-head attention layer, instead of the decoder layer, as keys for fetching multi-aspect documents. They are taking advantage of the fact that multi-head attention can learn to capture different data aspects. 🔹The interesting part is that MRAG is both cost-effective and energy-efficient. In other words, it does not require additional LLM queries, multiple model instances, increased storage, or multiple inference passes over the embedding model. 🔹Moreover, MRAG can be seamlessly integrated with existing RAG frameworks and benchmarking tools like RAGAS as well as different classes of data stores. 🔹The researchers provide a comprehensive evaluation methodology, including specific metrics, synthetic datasets, and real world use cases to demonstrate MRAG’s effectiveness. 🔸Do check it out! #genai #technology #artificialintelligence Click #linkedangle and follow for more content Set the 🔔 notification on my page, don’t miss a post!

181

10 Comments

It has been a while since I posted about GLIA Ecosystems I spent 30 years as a lead producer at a bunch of investment banks and boutiques. Successful at times (a couple of exits). Failure at others (a couple of closures). I was a first generation gamer and loved technology. I was an early adopter of Bloomberg back in the days when knowing more fast codes and what the latest product roll outs where made you smarter than the mass. This was back in the days when information was slow, prices were OTC for size, when risk capital was dominated by a handful of banks and when research had price impact. That has all gone away. The game has changed but still retail carries on thinking Trading Views and their ilk (25 mln users looking at charts and back tested technical strategies) can help. Charts, prices, news aggregation and price back testers are nowhere near to a competitive tool kit but it is all most platform users, and market participants, have access to! This at a time when Citadel, Renaissance and their cohort are using weapons of mass destruction to scalp the markets. Guess where all the money is being made? As Robinhood get paid for their “odd lot” order flow and Citadel ride roughshod over it. 90% of trading accounts lose money. 25% of US market flows are retail. 70% of the Western world is financially illiterate. Einstein, or someone close, said “Insanity is repeating the same thing over and over and expecting a different outcome”. That is every data platform user in the markets today, it is the vast bulk of institutional brokers (minus their flow); it is the market at large. We are making leap into the future - a place of differentiation - occupying a space intended to close the gap on the winners; light years (well a good 30 years) ahead of the mass utility offerings. A space where we hope financial success can live. A fund management game with a twist and $1 a day financial intelligence for the mass market. We look forward to sharing much more soon it has been a journey. At long last there is a realisation that mediocre systems breed mediocre outcomes. The future is intelligent; knowledge is curated, content is relevant, and users are at the centre of their digital universe. #learntoearn #joiningthedots #knowingtheknowable #knowledgetoact Peter Bentley Peter O' Toole Adam Toms Matt Barrett Mike Hammond Jon Chase Matthew Fletcher Aba Schubert Ricardo Santos Silva Alf Long Matthew S. Mark Gamble Noah Liot Marco Merighi Ameet Patel

71

8 Comments

AI Appreciation Day celebrates the advancements and impact of artificial intelligence on our lives, industries, and future. This special day acknowledges the remarkable progress in AI technologies, from machine learning and natural language processing to generative models and autonomous systems. On this day, we honor the researchers, engineers, and visionaries who contribute to the AI field, driving innovation and addressing ethical considerations to ensure AI benefits all of humanity. It is also a moment to reflect on the responsible use of AI, emphasizing the importance of ethical frameworks and inclusive development. Let's celebrate the milestones achieved and look forward to the endless possibilities AI holds for a brighter tomorrow. #AIDay #IndustryInnovation #IntelligentAutomation #AIAppreciationDay #ArtificialIntelligence

19

An alternative view on 'AI for good' by Payal Arora describing what I refer to as the tyranny of good intentions, in this case regarding AI, though as Payal discloses, intentions are not always as good as they claim to be. I've experienced this several times when we either hired outsourcing firms claiming to provide 'AI for good', or we were asked by others to assist for altruistic endeavors, including by some of the largest and wealthiest organizations in the world. What I've found is other agendas are usually lurking beneath the surface, including age-old greed and lust for power in NGOs, governments and companies. I've concluded that we can contribute the most to society by providing more efficient and secure systems that empower customers (KISS). "Meanwhile, we’ve seen philanthropies such as the Bill & Melinda Gates Foundation launch grand challenges for AI to help alleviate burdens on African healthcare systems. This has resulted in winners such as IntelSurv in Malawi, an intelligent disease surveillance data feedback system that computes data from smart wearables. Yet, even with hundreds of patents for such devices being registered every year, they’re not yet capable of consistently capturing high-quality data. In places like Malawi, these devices may become the single source of training data for healthcare AI, amplifying errors in their healthcare system." "Tech altruism has increasingly become suspect as AI companies are now facing an acute data shortage. They’re scrambling for data in the Global South, where majority of tech users live. Take, for instance, the case of Worldcoin, co-founded by OpenAI CEO Sam Altman. It plans to become “the world’s largest privacy-preserving human identity and financial network, giving ownership to everyone.” Worldcoin started as a nonprofit in 2019 by collecting biometric data, mostly in Global South countries, through its “orb” device and in exchange for cryptocurrency. Today, it’s a for-profit entity and is under investigation by many countries for its dubious data-collection methods."

4

1 Comment

✨ Great conversations spark great ideas, and last Friday’s meeting with Vijay Shenoy from Langoor was one such instance. 🤝 It’s always inspiring to connect with people building in the AI space. We discussed: 📊 AI’s role in marketing research and data analytics 🎨 Generative AI for creative campaigns 🔮 AI for decision-making and forecasting At LLMate, we’re focused on bridging the gap between data and decisions—helping businesses: ✅ Unify complex datasets 📈 Visualize them effectively 💡 Derive actionable insights Adding conversational AI into the mix? That makes it even easier for non-technical users to navigate these complexities. 💡 One key takeaway: It’s not just about having access to data but also about asking the right questions to unlock its true potential (a great reminder from Ginni Rometty's words). 🚀 Excited to see how the AI space in India continues to grow with innovative minds driving it forward. If you’re building or exploring something in this space, let’s connect—I’d love to hear your story and exchange ideas! #marketing #genAI #DataAnalytics

11