Mohamed Elgendy

Despite the disdain for "GPT-wrappers", LLM-driven apps is the layer where value is accruing. This is cuz evals are the differentiator & not easily replicable, unlike AI models. The reason why OpenAI pursues regulatory capture is cuz the moat is shallow, not cuz of existential risk of humanity. In Clay Christensen's lesser known "Law of Conservation of Modularity", he spells this out, summarized and outlined by Stratechery. https://lnkd.in/gt-2isr9 So if you don't want your lunch eaten. Get started on your evals today, and pick up a copy. https://lnkd.in/gxwnHVND

1

AI memory is fundamental for conversational AI and AI agents. It is critical for building personalized experiences and improving reasoning. Check out Letta’s course to understand how to use AI memory to enhance the customization and quality of your AI app.

15

Vishal Sachdev highlights the strategic integration of open source and proprietary tech in architecting tech stacks, developer ecosystems and resulting business models. The world class companies effectively balance value commoditization in open source and value capture in proprietary tech. This is the strategic challenge for companies operating in #blockchain and #AI. I believe those who operate at the intersection of blockchain *and* AI stand to win this strategic battle...

3

1 Comment

Thanks for sharing these insights. What I am seeing is that orgs test an idea using a proprietary #LLM like #OpenAI, then put it into production with an open source LLM using a development platform like #Llamaindex. That way, private data remains private.

1

We're launching the inaugural edition of Beyond Benchmarks, a deep dive into the metrics and trends observed across the early-stage enterprise cloud market. This is the first look at some of the business data around GenAI in SaaS. Some initial observations: 1) 60% of SaaS companies have already integrated GenAI into their offerings (60% of these as part of their current product and 40% as a new product), and an additional 20% are planning to do so this year. Almost half aren't monetizing these features. Those that are are primarily experimenting with both usage-based and flat fee pricing models. 2) Most companies are utilizing OpenAI as their primary LLM, but many are experimenting with multiple models - and we are noticing an early trend towards intelligently routing GenAI inference requests to different models based on cost, performance, and security. 3) Though we are in the initial innings of GenAI, we are seeing some positive signals - companies that implemented GenAI had 7% higher NDR than those that did not. Still early days, but we'll be doing this benchmarking report regularly to track and share how things progress in this new era. Link to report in comments.

104

6 Comments

NVIDIA's Jensen said "ChatGPT democratized computing, Llama2 democratized generative AI" ...and now Llama3 is the next level 🔥 It's fascinating to watch Meta's strategic moves. With Llama3, they prioritized training on a massive 15T token dataset to pack all into a lean 70B param model, instead of building a massive model. This allows Llama 3 to match trillion+ parameter models like GPT-4, but at 1/10th the compute, storage and inference costs! 💰 This technique was published by Google DeepMind a few years ago https://lnkd.in/gx7VU8aA Meta is not a dark horse in AI anymore. They might be the top dog. https://lnkd.in/gRRJYuxt #llama2 #llama3 #llm #chatgpt #gpt4 #ai #ml

16

5 Comments

Generative AI and machine learning: two titans of tech, often confused but distinctly powerful. One creates, the other predicts. Both are reshaping industries at breakneck speed. Which will have the greater impact on your business? The answer might surprise you. #AI #MachineLearning #TechInnovation #BusinessStrategy #FutureOfWork What's your take on the AI revolution? Share your thoughts below.

3

Sure, the opportunity to build AI Agents is there, but existing SaaS companies have a major advantage: their distribution channel. It’s easier to experiment providing agentic solutions (under their tailored business model) if you have an existing customer base and deep domain expertise, otherwise brace for 360° disruptive competition.

10

1 Comment

TRIPLE BAM! If you missed it, here's the recording of our live Q/A with Jay Alammar and Joshua Starmer PhD! We talked about many AI topics, answered lots of great questions, and talked about two great books: Hands-On Large Language Models: Language Understanding and Generation, and The Illustrated Guide to Neural Networks and AI!!! https://lnkd.in/dcPHs7YQ

44

1 Comment

This is a good article on using LLMs in the real world. It covers many aspects that we learned at Nurish over the last few months and more. Using LLMs is an inexact science, but has its own needs in processes and tools. While most of the attention has been on the LLMS, their capabilities, and RAG, there are mundane issues that need to be taken care of, issues where the solutions for pre-LLM development are not that well-suited. These include version management, integration, testing, data management, quality assessment, and more. This article covers many of those practical aspects. Eugene Yan, Bryan Bischof, Charles Frye & more https://applied-llms.org/

9

I just read an insightful piece by Eric Broda (published in Towards Data Science) that outlines the Agentic Mesh—a new ecosystem designed for autonomous agents to interact, collaborate, and transact. His work introduces a structured approach for managing the coming surge in autonomous agents, showing how they can function independently yet collectively. Agentic Mesh serves as a networked environment where autonomous agents operate in a safe, transparent, and efficient manner. It allows agents to find each other and securely interact under a framework of policies. Each agent has a specific purpose and autonomy, making it a valuable contributor to the ecosystem without sacrificing security or control. Core Components: 1. Marketplace: A centralized space where users discover agents by tasks and capabilities, offering a trusted environment for engagement. 2. Registry: A metadata repository organizing each agent’s purpose, owner, and policies, allowing agents and users to securely find and connect with each other. 3. DNS Integration: Each agent has a recognizable digital identity, enabling seamless discovery and interaction, just like an internet service. 4. Agent Stack: This layered structure of communication protocols, control services, and interfaces helps agents operate effectively and responsibly. It ensures that agents act independently within controlled parameters. Agentic Mesh’s effectiveness depends on agents sharing consistent attributes, including: • Purpose: A clear objective drives each agent’s actions. • Ownership: Transparency around ownership fosters responsibility and trust. • Autonomy: Agents act independently within set parameters to complete tasks. • Discoverability: Easily locatable, ensuring efficient interaction within the network. • Intelligence: Agents utilize language models to plan and execute tasks, adapting to complexity as needed. Operational Flow: Registration, Discovery, and Execution Agent interactions in the Mesh follow three core flows: 1. Registration: Agents set up purpose, policies, and registration data to gain visibility in the mesh. 2. Discovery: Through the registry, agents and users locate potential collaborators based on defined needs. 3. Execution: Engaged agents use AI models to plan and perform tasks, enlisting collaborators when necessary to complete workflows. Read the full article: https://lnkd.in/eVmq5BFC

16

1 Comment

LLMs are terrible at basic math. Why? Here's why: Unlike people, LLMs don’t conceptualize mathematics. They simply recognize patterns in the training data (text) they’ve learned from. When asked what 2 + 2 is, an LLM recalls instances of seeing ‘2’, ‘+’, ‘2’ and so on, allowing it to approximate solutions but not match the precision of a calculator. That’s how we end up with LLMs asserting that 2 + 2 = 5. But for analytical teams, like finance, that makes LLMs really hard to adopt in day-to-day workflows. The productivity gains of having AI do calculations are negated by needing to fact-check and audit every calculation and number. LLMs are also not deterministic, so no amount of prompting is going to guarantee you’ll get the correct answer every time. And that’s just for basic math – the outputs I’ve seen from LLMs on more complex calculations and analyses can get very wonky. Add thousands of rows of data and you’re squarely in hallucination land. So what’s the solution? Calculate things manually? No. The good news is that this is changing quickly and there are options: 1 - AI is getting better at math. Researchers continue to train LLMs with more specialized datasets that include a much wider array of mathematical problems and solutions. The newest models, like OpenAI’s latest family of models, are incorporating things like symbolic reasoning and “chain of thought” methods to break complex problems into smaller, logical steps the LLM can more easily process. Results have been really promising and we’ve seen each new iteration of models continue to beat previous benchmarks. 2 - Leverage AI agents For intensive workflows, and to really make sure your outputs are correct, working with companies that are building high-performing AI agents is likely your best bet. Agentic workflows allow LLMs to be used for what they are great at (inferring user intent, making decisions, etc.) while outsourcing the things they struggle with (complex calculations) to other parts of a broader system. Text-to-SQL, “tool-use”, few-shot prompting, and other strategies can be combined with real-time data integrations to dramatically improve performance on analytical operations. At Concourse we’ve spent the better half of 2024 getting our AI Agents to be really good at math, and I’m so excited to continue growing our capabilities to tackle even more complex mathematical problems for our customers in 2025. Reach out if you’re interested in learning more about how we’ve built Concourse and the impact we’re already delivering for finance teams.

42

5 Comments

LLMs are not vast, infallible databases. As models grow larger so does their tendency to confidently state falsehoods (hallucinations). Matei Zaharia, co-founder & CTO of Databricks, warns against treating LLMs as internet-scale encyclopedias and proposes we focus on what they excel at:  → Reasoning → Interpreting language → Understanding context For founders building AI applications, treat LLMs as powerful reasoning engines and not fact repositories. Build your knowledge base separately, feeding the LLM with trusted up-to-date information through retrieval mechanisms and tool-calling. This approach lets you harness the LLM's strengths in language analysis and generation while sidestepping the risks of its unreliable memory. Building with AI? Share your approach to balancing LLM reasoning power with factual accuracy in the comments. Let's hear it!

137

22 Comments

This article is intended to clarify what technical drawings are for, and how they’re different from models. They’re as distinct from each other as the cosmos is from a lens for looking at it, as distinct as the universe is from a telescope, They’re not in any way the same kinds of things and one cannot replace the other. However, telescopes certainly can and do EVOLVE:

4

LLM Data Engineering Pattern 3: Aggregating Unstructured Insights at Scale Modern financial analysis often requires synthesizing information from a vast array of unstructured sources—regulatory filings (e.g., 10-K, 10-Q), research reports, market commentary, and even audio transcripts of earnings calls. In many cases, these sources must be combined to form a comprehensive view of an industry’s risk profile. Historically, analysts have relied on manual selection and review of documents, a labor-intensive process that does not scale well in today’s data-rich environment. Consider analyzing credit risk in the U.S. commercial real estate sector: 1) Regulatory Filings: An LLM can systematically parse through hundreds of 10-K and 10-Q filings to extract commentary on asset valuations, loan-to-value ratios, or refinancing challenges. 2) Research Reports: It can synthesize qualitative assessments from research houses covering property market trends, default probabilities, or vacancy rates. 3) Earnings Call Transcripts: It can highlight executives’ forward-looking statements regarding rental income stability, interest rate exposures, or occupancy forecasts. By integrating these unstructured data points, an analyst can address questions like: “Given the commentary on sector vulnerabilities in these filings, research reports, and macroeconomic indicators, what are the key risk factors influencing the commercial real estate market?” Financial people, how are you doing this today? #llm #dataengineering #llmdataengineering #financialdata #finance

20

1 Comment

🚀 Earlier this month, we announced a major update -- that Deepchecks LLM Evaluation is now natively available in the Amazon Web Services (AWS) environment (SageMaker AI). This was huge for us. I’ve had countless conversations with various teams this year about their LLM evaluation needs. The biggest headache for them is to 𝐨𝐧𝐛𝐨𝐚𝐫𝐝 𝐨𝐧 𝐭𝐡𝐞 3𝐫𝐝 𝐩𝐚𝐫𝐭𝐲 𝐚𝐧𝐝 𝐝𝐚𝐭𝐚 𝐩𝐫𝐢𝐯𝐚𝐜𝐲 𝐢𝐬𝐬𝐮𝐞𝐬. 🔐 Making Deepchecks available within AWS was a lot of work, but Kudos to Shir, Noam, and the whole Deepchecks team for making it happen. 👏 The key benefits are: > Data Locality within your AWS account. > No Infrastructure Management. SageMaker AI handles setup, upgrades, and scaling. > Zero-Time Procurement Process.  > No Security Questionnaires or Architecture Discussions. ✅ Companies building LLM apps can use their existing AWS credits to get a fully managed, on-prem version of Deepchecks LLM Evaluation. 𝐓𝐨 𝐠𝐞𝐭 𝐬𝐭𝐚𝐫𝐭𝐞𝐝 𝐨𝐫 𝐫𝐞𝐪𝐮𝐞𝐬𝐭 𝐚𝐜𝐜𝐞𝐬𝐬: https://lnkd.in/dSk2rrFb Or feel free to reach out to me on LinkedIn or via email. 😊 Swami Sivasubramanian, Ankur Mehrotra, Guy Maidan, Mair Hasco, Thomas Zerbach, Sai Kumar Devulapalli, Jia You, Sumit Thakur, Gwen Chen, Eric Raúl Peña, Vineet Sharma, Rafe Gándola #LLMs #AWS #LLMEvaluation #GenAI LLMOps Space

52

Generative AI: miracle worker or sophisticated guesser? While AI can now write, code, and create, it's essentially still a prediction machine. This revelation challenges how we view and utilize AI in business. What does this mean for your AI strategy? Let's discuss. #ArtificialIntelligence #BusinessStrategy #TechInnovation #AILimitations #FutureOfWork

2

I’ve been fortunate enough to do AI shows with Sam Altman, Yann LeCun & Arthur Mensch. But this show today with David Cahn at Sequoia Capital is the best of all of them: 🧠 AI’s $600BN question 🏃 Why no one will never train a frontier model on the same data center twice 🏗️ Servers, steel & power: The 3 pillars of the future of AI Not one to be missed, my 8 key takeaways 👇 1. The Oligopoly That is Blowing Billions on AI - Microsoft, Azure, & Google now represent $7T market cap. - Of course they will spend billions to protect their oligopoly - What is most interesting is Meta is the only incumbent that is financing this not with cloud being their cash cow. 2. How Startups Could Win with the Cost of Compute Coming Down - Big tech companies are producers of compute & startups are consumers of compute. - Overproduction of compute = lower prices.  - This gives startups higher margins & makes them more valuable. 3. Is Compute the Currency of the Future or Do the Costs Come Down and It Is Commoditised? - We are forgetting “compute” is a euphemism for a $2BN data center in the middle of nowhere. - They have GPUs & liquid cooling systems that need to be constantly optimized & upgraded. - We cannot simply build 15 years worth of compute all at once.  No one's ever going to train a frontier model on the same data center twice. 4. Data Centre Teams and Model Teams Need to Be Coupled - You cannot have separate teams running data centers & building models. - Elon & Zuck control all their data centres. - It will not work as these models get better & better. 5. Why the Data Center Is the Most Important Asset - There is not much difference between these models today. - AI researchers are jumping from one lab to another. - Scaling laws will become more dominant as these models get bigger. 6. Servers Steel and Power: The Three Bottlenecks of AI Three things: 1. Servers: NVIDIA, AMD, Broadcom… 2. Steel: Construction & real estate developers for data centers. 3. Power: All the energy required for AI. - More factories will be built in 12 months for AI and we will see another industrial revolution. 7. You Have to Have a Cash Machine to Play the Big Model Game - Amazon has AWS - Meta has Instagram - Microsoft has Azure - You need a cash machine to compete and it CANNOT be the AI business. 8. There is One Definition of Success in Venture: Are you a Slugger? - If you are generating billion dollar gains, you are a slugger. - Doug Leone, Pat Grady, Andrew Reed are all sluggers. - You cannot be good at this business without being a slugger. (links in comments) #founder #funding #business #investing #vc #venturecapital #entrepreneur #startup #seed #funding #sequoia

190

26 Comments

Great post from Sequoia Capital summarizing trends in the LLM/Agentic ecosystem. Worth the read. Some takeaways: * Custom “Cognitive Architectures” are essential for real-world AI applications. Cognitive Architectures are defined as the “the flow of code and model interactions that takes user input and performs actions or generates a response.” Also the corollary - “Application layer AI companies are not just UIs on top of a foundation model. Far from it. They have sophisticated cognitive architectures …” * Shift towards "Service as Software" – shift towards providing a complete solution for the entire job to be done. * Moving away from Product-Led Growth (PLG) to top-down, service-heavy sales models. “Cloud companies liked to go bottoms-up, with frictionless distribution. AI companies are increasingly going top-down, with high-touch, high-trust delivery models.” * The potential for AI to displace existing software. “Could the opportunity for AI be both selling work and replacing software?” #ai #agents https://lnkd.in/eRwQMNNi

33

3 Comments