Sharon Zhang

Gemini 2.0 handles complex math and planning tasks previously out of reach of LLMs. You can turn on "code generation" in AI Studio and in the APIs. For context, let's talk about function calling and voice AI agents. (I spend a lot of time working on voice AI.) Voice AI agents use function calls for: - Interacting with external systems specific to particular use cases and deployment contexts. - A wide range of RAG-like information lookup tasks. - Conversation persistence, including complex actions like saving a privacy-redacted version of a conversation. - Script following. Some of these uses were obvious before we had built and deployed production voice AI systems. But some of them weren't. (Script following, for example.) We had to build experiments, and then production systems, to figure out what features and tooling these new systems really need. I'm convinced that code generation will be as important as function calling, once we've accumulated some experience building with Gemini 2.0 and similarly capable new LLM versions. Credit for this insight goes entirely to Shrestha Basu Mallick, who shepherds work at Google on the Gemini APIs and AI Studio. When Shrestha first told me that really good code generation was going to make a lot of new things possible, I didn't really get it. But after playing with code generation for a few weeks ... I still don't fully get it, but in a good way, now. It's clear that there are some obvious things to explore: - Context-aware math on demand driven by natural language. - Generative UI. - Better "binding" behavior with function calling. But I'm also convinced now that there is a lot of unexplored, non-obvious (at least to me, yet) territory. Let's take that last one — improved function calling performance. Function calling and code generation go together like peanut butter and jelly. Functions often take arguments that are fairly complicated. With code generation enabled, Gemini has access to this new pattern: natural language ➡️ code that creates argument structs for functions ➡️ multiple function calls Think about how powerful this is. The ability to use even very basic code structures — loops, for example — massively expands what LLMs can do. In fact, my impression is that function calling in Gemini 2.0 is now trained as a subset of the model's code generation capabilities. This feels quite different than how function calling works in other models that I have access to. In other models, function calling feels like a close cousin to structured output. Which is great. But this new "modality" is even better. (More on the prompt in this post, and the meta-prompt for this post, here: https://lnkd.in/gycKtnNX)

49

10 Comments

Exclusive: Health-care AI startup Suki on Wednesday announced it's building patient summary and Q&A features with Google Cloud as it looks to push beyond clinical documentation. Suki’s flagship product, called Suki Assistant, allows doctors to record their visits with patients and automatically turn them into clinical notes, helping physicians avoid the headache of manually writing out all of that information. The new features with Google Cloud will allow Suki to provide clinicians with more assistive tech as they care for patients, the startup said. It is the next frontier for the seven-year-old company. “We were never really building a clinical documentation tool only, it was supposed to be an assistant,” Punit Singh Soni, the founder and CEO of Suki, told CNBC. “An assistant can help you with documentation, but it can also start doing other things.” More details here!

55

5 Comments

The cost of training top models is increasing 2-3X per year, and at this rate will reach $1 billion per model by 2027. The findings are from Epoch AI and follow their recent analysis of increases in compute for foundation models running at 4-5X per year. They've also provided some interesting data on the relative breakdown of cost components like staff, hardware, and energy. A few thoughts after reading this: - I would assume staff costs won't increase as much as hardware and energy costs, so the relative contribution of human labor to future AI models will decrease, in yet another sign of capital's increasing dominance over labor in the industry (just look at NVIDIA's ever-rising stock price). - If compute is increasing 4-5X per year but cost is increasing 2-3X, is it safe to assume that training runs are getting more efficient? - Some people, like Anthropic CEO Dario Amodei, have said we'll see $1 billion training runs in the next year. Does this mean the 2-3X pace is about to increase? That Epoch is wrong in its estimate? Or that Amodei is overstating the cost, possibly to put fear into competitors who can't keep pace? - Only a few companies are going to be able to train future models. We're likely going to see further consolidation. Perhaps we get to a future with only two large general purpose models, like the Windows/MacOS and Android/iOS duopolies we've seen in the past. Article here: https://lnkd.in/g5SejXen

7

This is a good article on using LLMs in the real world. It covers many aspects that we learned at Nurish over the last few months and more. Using LLMs is an inexact science, but has its own needs in processes and tools. While most of the attention has been on the LLMS, their capabilities, and RAG, there are mundane issues that need to be taken care of, issues where the solutions for pre-LLM development are not that well-suited. These include version management, integration, testing, data management, quality assessment, and more. This article covers many of those practical aspects. Eugene Yan, Bryan Bischof, Charles Frye & more https://applied-llms.org/

9

Created an AI assistant app to connect patients and doctors using Vercel AI SDK, Assistant-UI.com components, function calling, thread suggestions, check it out (Guillermo Rauch, Simon Farshid, Kajari Ghosh)

10

#RAG DOES NOT REDUCE HALLUCINATIONS IN #LLMS: Think I've called this out a few times, but the validation is nice - https://lnkd.in/eAi6zxW2. The article makes two well established points: 1) "Hallucination is a problem of reasoning and not relevance! Any amount of relevant text fed through RAG to a machine will retain the original perplexity and entropy in the system to hallucinate independent of the text"; and 2) "The initial challenge arises from the retrieval phase’s reliance on semantic similarity, which is quantified by inner product spaces or cosine similarities. This metric, while capturing semantic closeness, does not differentiate between factually accurate and inaccurate information." This should be clear to everyone, but the industry still shills flimsy RAG scaffolding anyway (shovels to gold miners).

104

32 Comments

Artificial intelligence (AI) holds immense promise for transforming numerous fields, from healthcare and finance to education and transportation. The potential for AI to contribute to the medical field is particularly compelling. Given its ability to process vast amounts of data and identify complex patterns, AI could play a significant role in accelerating the discovery of treatments for diseases that have previously defied conventional medical approaches, such as cancer and AIDS. By leveraging AI's capabilities, researchers may be able to uncover potential solutions that could lead to groundbreaking advancements in the treatment of these challenging and persistent health issues.ArtificialIntelligence #MedicalResearch #HealthcareInnovation #DataProcessing #DiseaseTreatment #ResearchAdvancements

3

We are not slowing down on Gen AI, come see why...and how it changes things, the nature of work. https://lnkd.in/gW3ettg3

2

Thanks to #genai360 for the shoutout here for Consilience AI and our AlphaIQ product. Davit Buniatyan Joseph Byrum, Ph.D., MBA

10

I’ve been fortunate enough to do AI shows with Sam Altman, Yann LeCun & Arthur Mensch. But this show today with David Cahn at Sequoia Capital is the best of all of them: 🧠 AI’s $600BN question 🏃 Why no one will never train a frontier model on the same data center twice 🏗️ Servers, steel & power: The 3 pillars of the future of AI Not one to be missed, my 8 key takeaways 👇 1. The Oligopoly That is Blowing Billions on AI - Microsoft, Azure, & Google now represent $7T market cap. - Of course they will spend billions to protect their oligopoly - What is most interesting is Meta is the only incumbent that is financing this not with cloud being their cash cow. 2. How Startups Could Win with the Cost of Compute Coming Down - Big tech companies are producers of compute & startups are consumers of compute. - Overproduction of compute = lower prices.  - This gives startups higher margins & makes them more valuable. 3. Is Compute the Currency of the Future or Do the Costs Come Down and It Is Commoditised? - We are forgetting “compute” is a euphemism for a $2BN data center in the middle of nowhere. - They have GPUs & liquid cooling systems that need to be constantly optimized & upgraded. - We cannot simply build 15 years worth of compute all at once.  No one's ever going to train a frontier model on the same data center twice. 4. Data Centre Teams and Model Teams Need to Be Coupled - You cannot have separate teams running data centers & building models. - Elon & Zuck control all their data centres. - It will not work as these models get better & better. 5. Why the Data Center Is the Most Important Asset - There is not much difference between these models today. - AI researchers are jumping from one lab to another. - Scaling laws will become more dominant as these models get bigger. 6. Servers Steel and Power: The Three Bottlenecks of AI Three things: 1. Servers: NVIDIA, AMD, Broadcom… 2. Steel: Construction & real estate developers for data centers. 3. Power: All the energy required for AI. - More factories will be built in 12 months for AI and we will see another industrial revolution. 7. You Have to Have a Cash Machine to Play the Big Model Game - Amazon has AWS - Meta has Instagram - Microsoft has Azure - You need a cash machine to compete and it CANNOT be the AI business. 8. There is One Definition of Success in Venture: Are you a Slugger? - If you are generating billion dollar gains, you are a slugger. - Doug Leone, Pat Grady, Andrew Reed are all sluggers. - You cannot be good at this business without being a slugger. (links in comments) #founder #funding #business #investing #vc #venturecapital #entrepreneur #startup #seed #funding #sequoia

190

26 Comments

Something that stuck with me during my interview with Naveen Rao, VP of Generative AI at Databricks: "I take the hard path." He built an AI-focused chip company Nervana in 2014 when machine learning was still academic theory and the chip industry was deeply out of favor. Naveen saw computing architecture would need to fundamentally change for ML workloads. And then in 2019, observing the power of LLMs like BERT, Naveen foresaw the need for enterprise-grade services to manage their training, deployment, and customization. That vision led to a $1.3B acquisition by Databricks in 2023. What I love about Naveen's approach is his willingness to build for markets that don't exist yet. "There isn't a market, employees don't even know about it. So how the hell do you build a company?" You educate. You convince. You get people to see what you see. It's brutal work, but it's important work. You're reshaping how people think about problems.

176

8 Comments

I just read an insightful piece by Eric Broda (published in Towards Data Science) that outlines the Agentic Mesh—a new ecosystem designed for autonomous agents to interact, collaborate, and transact. His work introduces a structured approach for managing the coming surge in autonomous agents, showing how they can function independently yet collectively. Agentic Mesh serves as a networked environment where autonomous agents operate in a safe, transparent, and efficient manner. It allows agents to find each other and securely interact under a framework of policies. Each agent has a specific purpose and autonomy, making it a valuable contributor to the ecosystem without sacrificing security or control. Core Components: 1. Marketplace: A centralized space where users discover agents by tasks and capabilities, offering a trusted environment for engagement. 2. Registry: A metadata repository organizing each agent’s purpose, owner, and policies, allowing agents and users to securely find and connect with each other. 3. DNS Integration: Each agent has a recognizable digital identity, enabling seamless discovery and interaction, just like an internet service. 4. Agent Stack: This layered structure of communication protocols, control services, and interfaces helps agents operate effectively and responsibly. It ensures that agents act independently within controlled parameters. Agentic Mesh’s effectiveness depends on agents sharing consistent attributes, including: • Purpose: A clear objective drives each agent’s actions. • Ownership: Transparency around ownership fosters responsibility and trust. • Autonomy: Agents act independently within set parameters to complete tasks. • Discoverability: Easily locatable, ensuring efficient interaction within the network. • Intelligence: Agents utilize language models to plan and execute tasks, adapting to complexity as needed. Operational Flow: Registration, Discovery, and Execution Agent interactions in the Mesh follow three core flows: 1. Registration: Agents set up purpose, policies, and registration data to gain visibility in the mesh. 2. Discovery: Through the registry, agents and users locate potential collaborators based on defined needs. 3. Execution: Engaged agents use AI models to plan and perform tasks, enlisting collaborators when necessary to complete workflows. Read the full article: https://lnkd.in/eVmq5BFC

16

1 Comment

Notion vs Glean reminds me a lot of Tesla vs Waymo... Just like I think Tesla is going to solve the 'software' problem of autonomous vehicles before Waywmo solves the 'hardware' problem... I think that Notion is going to solve 'enterprise knowledge search' before Glean solves the problem of document and knowledge creation... Why? A couple reasons: 1) Just like Tesla, Notion has a huge advantage when it comes to the amount of underlying data and documents to train on... They already have 100M+ users, and likely 10x that in terms of the number of documents, files, etc. in team's workspaces. 2) It's very hard to change people's behaviour re: where/how they create 'work'. It's why a lot of companies have come and gone in that category... As exciting as Glean's AI writing assistants might seem, most people are just fine (or even prefer) a good old fashioned editor/writer. I could be way off here, but this will be an intersting one to follow. Thoughts? Where am I getting it wrong? #ai #enterprise #knowledge #management #notion #glean

3

1 Comment

Building with Large Language Models (LLMs) can be incredibly frustrating! The hype is real, but the reality? It's like we're in the Lotus Notes era of LLMs—promising, yet limited and painful to manage. Integrating LLMs into your tech stack, alongside your compute cluster, relational database, and object storage, reveals their quirks and limitations, making them just another complex piece of the puzzle. They're not magic 🧙‍♂️! For anyone diving into the world of LLMs, here are some critical challenges to be aware of: 1. Unpredictable Behaviour: LLMs aren't deterministic, meaning getting them to perform specific tasks can feel like an endless struggle. 2. Shaky Developer Features: Many infrastructure providers boast about their developer features, but in practice, these tools often fail to deliver. The documentation may promise full support, but real-world usage tells a different story. 3. Provider Reliability: OpenAI is the only truly reliable provider for developers, yet their models are typically too large and slow for real-time applications like mine. They're great for the chat modality, if you're into building wrappers 🎁. 4. The Groq Hype: People are excited for Groq because of its speed and impressive chips, but their software stack is still in a very beta state. Anyone claiming to use Groq in production is either lying or taking a huge risk with their customer experience. That said, it's awesome having a new tool in our toolbelt for building new, creative software capabilities we've never had before. It's just really hard to merge the creativity with the need for most business use cases to be deterministic!

19

8 Comments

$10B to train an SOTA LLM model. I've been asking fellow Product, Engineering and GTM leaders if SOTA LLMs are delivering positive business outcomes. Answer - > without a value chain of other AI models like ETL, RPA, ASR, TTS, w/ lots of fine tuning and non AI pre-post processing, enterprise customers aren't see enough value to pay more. In fact, they want to pay way LESS to get the same jobs done. This reminds me that not much has changed for product and engineering leaders. We still have to listen to the voice of our customers and translate that into a product that has positive unit economics for the companies we work for. Its very expensive to build and maintain AI solutions. Very expensive to run AI solutions. The right question isn't should I leverage AI in my product, rather if I use AI will it improve the net profit margin. $$$

15

2 Comments

In case you missed it.... We published a great H:F guest episode with Joel Portice, CEO of egnite - delving into the transformative role of data and AI in revolutionizing cardiovascular care. Joel shares his journey of leading five successful healthcare data companies, including the genesis and rapid growth of egnite, which was spun out from Edwards Life Sciences. egnite's flagship platform, Cardio Care, uses advanced analytics, machine learning, and AI to identify and manage cardiovascular patients, saving lives daily. This episode is a strong addition to our Heart Health Series on Health:Further. Please check it out: https://bit.ly/4fKmzCH

59

5 Comments

Unlocking Healthcare Insights with AI: 5 Steps to the Right Approach for LLMs Over my 17+ years in IT, working with many organizations I've witnessed how transformative AI can be useful in healthcare. Large Language Models (LLMs) are now revolutionizing patient care by providing deeper insights into population health, predicting disease trends, and personalizing treatment plans. 🏥 However, implementing LLMs effectively isn't just about adopting new technology—it's about finding the right and cost-effective approach. Here are some steps I've found valuable: 1️⃣ Leverage Existing Infrastructure: Utilize your current systems and integrate LLMs without a complete overhaul, saving costs and resources. 2️⃣ Ensure Compliance from the Start: Align with HIPAA and HITECH regulations early to avoid costly adjustments later. 3️⃣ Embrace Interoperability Standards: Implement HL7 and FHIR protocols for seamless data exchange between systems, reducing integration expenses. 4️⃣ Start Small and Scale Responsibly: Begin with pilot projects to demonstrate value before investing heavily, minimizing financial risk. 5️⃣ Collaborate with Experienced Partners: Work with experts who understand both the technical and regulatory landscapes to navigate complexities efficiently. At Arihant Technologies, we combine deep industry experience with practical solutions to help healthcare organizations adopt AI in a way that's both effective and budget-friendly. #ArihantHealthTech #ArihantTechnologies #healthcare #ai #llms #hipaa #hitech #interoperability #costeffective #experiencematters #healthtech #populationhealth #patientcare #dataanalytics #healthcareinnovation #fhir #hl7 #compliance #digitalhealth #medicalai #affordableai

13

3 Comments

Today, I found a fascinating way to think about “Tokens” in LLMs. It came from a conversation between Jensen Huang and Patrick Collison at a recent Stripe event. When asked about the future of compute capacity in five years, Jensen smartly dodged that question a bit, but then gave an excellent explanation and analogy about why tokens are the new forces that will power the next decades of humanity. He explained that we're now producing something unprecedented (and at scale): floating point numbers that possess “value”, which we now call tokens. These tokens are valuable because they encapsulate “intelligence”. People are now taking these tokens and transforming them into English, French, images, videos, chemicals, proteins, robotic movements, etc. And many are working hard to expand the range of concepts and ideas we can create with these tokens. He then goes on to made a compelling comparison between tokens and electricity. In the past industrial revolution, we successfully found a way to convert “atoms” into “electrons” (by boiling water to power electricity turbines). And now, we've discovered a way to convert "electrons" into "tokens" (by using energy to power data centers that train and run LLMs). When electricity was first introduced, few people understood its value. Today, paying for kilowatts is routine. The same will happen with tokens. Right now, only early adopters and builders are paying tokens. Soon, everyone will be paying for tokens on a daily basis to supercharge productivity and power new products and services. Many new industries will be born from and built on top of tokens. When I first heard about this way of thinking, I got goosebumps. Perhaps it’s because Jensen has a good storytelling skill that helps him sell what he builds, but I have to admit this way of thinking gave me a brief surge of pride. I'm proud of humanity's collective effort and creativity, which have taken us from living under rocks to building machines that can "think." That's nothing short but a miracle.

18

5 Comments