Kumar Mettu

I spent five days in the SF Bay Area with B2B SaaS founders, funders, and senior software engineers. The talk is all about adding GenAI and LLM features to software apps, but they were unclear exactly how or when the AI big wave revolution will occur. AI is very cool and will be big, but it's still early to see how this will play out in the next year on the front lines of business. There's a lot of hurrying up to say you have AI features with a healthy skepticism about the user benefit of early GenAI attempts. Coders in big tech say, "Our product leaders told us we need to add GenAI features, but it still doesn't seem that interesting for users. We'll design it, code it, and ship it anyway." Founders in startup SaaS say, "We are running a lot of experiments and keeping up with the big AI platform changes. We're all more productive now but it hasn't made our app much more useful." Smaller SaaS funders say, "It's hard to tell where the winners will be to make our bets. AI tools help software companies grow more efficiently, so they don't need our funding as much. We're being very careful." Big enterprise SaaS buyers say, "We need to start investing, testing, and trying GenAI technology, which means we need to spend less on our existing SaaS apps in our tech budget." Big investors say, "It will be the next big tech wave, so we are making big early bets on chips, infrastructure, foundational models, and platforms. NVIDIA, Microsoft, Google--the big will get bigger." It's all very exciting, but it's still early and moving around quickly. That's a lot different from the last big software tech wave when SaaS apps on the web and mobile took over the last 15 years. It was easier to bet on the massive transition from on-premise Windows software to cloud-based web and mobile apps for businesses everywhere and billions of consumers. Despite that, I'm more excited about GenAI and other ML platforms than I was about VR, blockchain, and other platform opportunities that didn't materialize. It's early, but the practical AI wave will happen in ways we can't imagine or expect. What are you seeing in the current AI wave? #practicalfounders

240

108 Comments

AI memory is fundamental for conversational AI and AI agents. It is critical for building personalized experiences and improving reasoning. Check out Letta’s course to understand how to use AI memory to enhance the customization and quality of your AI app.

15

NVIDIA's Jensen said "ChatGPT democratized computing, Llama2 democratized generative AI" ...and now Llama3 is the next level 🔥 It's fascinating to watch Meta's strategic moves. With Llama3, they prioritized training on a massive 15T token dataset to pack all into a lean 70B param model, instead of building a massive model. This allows Llama 3 to match trillion+ parameter models like GPT-4, but at 1/10th the compute, storage and inference costs! 💰 This technique was published by Google DeepMind a few years ago https://lnkd.in/gx7VU8aA Meta is not a dark horse in AI anymore. They might be the top dog. https://lnkd.in/gRRJYuxt #llama2 #llama3 #llm #chatgpt #gpt4 #ai #ml

16

5 Comments

Vishal Sachdev highlights the strategic integration of open source and proprietary tech in architecting tech stacks, developer ecosystems and resulting business models. The world class companies effectively balance value commoditization in open source and value capture in proprietary tech. This is the strategic challenge for companies operating in #blockchain and #AI. I believe those who operate at the intersection of blockchain *and* AI stand to win this strategic battle...

3

1 Comment

Can we trust machines to judge machines when it comes to LLMs? There's an ongoing battle between manual and automated evaluation of LLMs. Manual evaluation is predominantly used by many companies, especially large ones with sufficient budgets. This approach involves internal QA teams or external consultants who manually create test cases and evaluate the outputs in spreadsheets. This process is time-consuming, but can provide insights into: - model's performance - specific failure modes such as hallucinations - tone discrepancies - biases - copyright violations - privacy issues Smaller companies without such budgets often resort to using expensive engineering time to manually review outputs, which can be inefficient. The primary advantage of manual evaluation is the accuracy and reliability of human judgment. But it's not scalable and introduces human biases, which can affect the consistency of the evaluations. Also manual processes cannot keep up with the rapid development cycles and the volume of outputs generated by modern LLMs. Automated evaluation aims to reduce the reliance on human labor by employing metrics and models to assess LLM performance. Techniques used in automated evaluation include: - Metrics-Based Evaluation: It utilizes established metrics such as BLEU score, ROUGE score, and perplexity. These metrics help assess aspects like fluency, accuracy, and relevance. But these metrics often do not correlate well with human judgments and can be inadequate for comprehensive evaluation. - Heuristic-Based Functions: These are specific functions to test particular capabilities or behaviors of the LLM e.g. toxicity detection or PII (Personally Identifiable Information) detection. These approaches can be tailored to specific use cases but may lack generalizability. - Model-Based Evaluation: It uses another trusted language model to evaluate the outputs of the target LLM. This can automate the evaluation process but requires careful calibration to ensure reliability. Automating this process can be a huge unlock. But developers attempting to create their own automated evaluation systems report very low confidence in the results. Lot of opportunity on this front.

3

1 Comment

We're launching the inaugural edition of Beyond Benchmarks, a deep dive into the metrics and trends observed across the early-stage enterprise cloud market. This is the first look at some of the business data around GenAI in SaaS. Some initial observations: 1) 60% of SaaS companies have already integrated GenAI into their offerings (60% of these as part of their current product and 40% as a new product), and an additional 20% are planning to do so this year. Almost half aren't monetizing these features. Those that are are primarily experimenting with both usage-based and flat fee pricing models. 2) Most companies are utilizing OpenAI as their primary LLM, but many are experimenting with multiple models - and we are noticing an early trend towards intelligently routing GenAI inference requests to different models based on cost, performance, and security. 3) Though we are in the initial innings of GenAI, we are seeing some positive signals - companies that implemented GenAI had 7% higher NDR than those that did not. Still early days, but we'll be doing this benchmarking report regularly to track and share how things progress in this new era. Link to report in comments.

104

6 Comments

Old school and probably not popular but I still believe products should be customer-satisfaction focused/led rather than technology-led. Use cases for AI should be byproducts of the product improvement process primarily. The thing though is that founders need to fundraise so they're likely to sprinkle some AI on their decks to even look serious. Not sure that's how the most enduring companies have been built. Of course, if sources of capital become AI-biased, founders have little choice. Happy to keep an open mind and dive in when a use-case that can 10x customer experience for Good Tenants customers and users shows up.

5

Why rely on one #foundationalmodel for your #GenerativeAI strategy when you can federate + #innovate across them. Tabnine adds GPTx + Mistral + Anthropic and gets named "Luminary" status by Everest Group. Trusted by millions of developers and thousands of company's - AI you control. I love #investing in #extraordinary #founders!

3

California Law SB 1120 has gone into effect, and prohibits health insurance providers from denying claims solely based on an algorithmic decision. This means that "yes" decisions can be automated, but "no" decisions must be routed to a human expert reviewer for a final decision. The law is intended to disallow mass automated rejections, as has happened before, and matches how most health insurers already operate. https://lnkd.in/gKDx_XVa #healthcare #ai #regulation

42

4 Comments

Apple’s Private Cloud Compute and the business of privacy With Private Cloud Compute, Apple has gone to extraordinary lengths to safeguard user privacy with AI-related tasks. In doing so, Apple has surely also incurred an extraordinary expense. How can Apple justify the cost of providing PCC if it doesn’t directly monetize its Apple Intelligence features? Either through the improved brand perception captured in a broader privacy narrative or by monetizing PCC with other non-AI use cases. While the narrative explanation is clear, the more interesting theory about Apple’s motivation in building PCC is that it can be expanded to other use cases that can be directly monetized. The first alternative use case that springs to mind is advertising attribution. PCC sounds conceptually similar to a data clean room, albeit far more sophisticated than most implementations, which merely use IP addresses and other identifiers to match publisher-observed ad interactions with advertiser-observed conversions. While the PCC nodes seem hardware-optimized for processing AI-related tasks, other nodes could be built that aren’t.

19

3 Comments

VC funding for compute silicon startups in 2024 remained robust, with many companies achieving billion-dollar valuations. The strong emphasis on compute is creating significant opportunities in the communication domain, particularly in Ultra Ethernet and CXL-based switches, which address the next bottlenecks in achieving optimal performance. - Cerebras - Expecting IPO in 2025 - Tenstorrent - Raised $693M in series D round in 2024 - Groq - Raised $640M in series D round in 2024 - Etched.AI - Raised $120M in series A round in 2024 - D-matrix - Raised $110M in Series B round in 2023 - Matx - Raised $80M in series B round in 2024 - Rain.AI - Looking for $150M backed by OpenAI CEO Sam Altman - FlexAI - Raised $30M in seed round in 2024 #AI #silicon #compute #CXL Andrew Feldman Jonathan Ross Jim Keller Gavin Uberti Sid Sheth William Passo Brijesh Tripathi Bipin Shah Shailesh Rao Bala Joshi Murali Chirala Sailesh Kumar

28

1 Comment

🚨🚨 🚨 Unlocking Innovation: The Power of the Adjacent Possible Innovation programs can stall when the team tries to ideate way beyond their collective perspective and frame of reference. Like a software engineer brainstorming genetic modification of mycelia, or a senior executive ideating process improvements which they haven't been participated in for decades. It's challenging, exhausting, and ultimately self-defeating. Enter the concept of the "Adjacent Possible" - a game-changer for innovation. What is it? The Adjacent Possible, coined by Stuart Kauffman, represents the realm of new ideas within reach based on our current knowledge, resources, and capabilities. Why it matters: ➡️ Realistic Innovation: it focuses on achievable advancements, not far-fetched concepts. ➡️ Incremental Progress: each innovation opens doors to new possibilities. How to leverage it: • Constantly explore the boundaries of your adjacent possible. What's in your line of sight is what you know best. • Combine existing ideas in novel ways. Even small perturbations from status quo can generate large innovation leaps. • Stay curious about developments in adjacent industries. Cross-domain patterns create fresh sources of ideas, while also expanding the adjacent possible. Most of all, it takes the 'genius' pressure off the team. No one knows everything, but each one of us knows a lot about something, and we can all drive plenty of innovation potential by focusing on the spaces we know best. Are you exploring your Adjacent Possible? Share your thoughts below! 👇 #Innovation #innovationsystems #AdjacentPossible

8

2 Comments

There are two interesting and relatively new/emerging revenue models in hardware space. (1) Software Enabled Hardware Hardware is sold at lower but slowly improving gross margins (eg 20-40%), and software recurring revenues layered on top. Software sells at higher margins, enabling blended margins to appear in 50%+ category. These are applied in situations where hardware is not expected to be replaced or greatly improved upon in deployment for a significant number of years to come. Consider your car, or military equipment, or even infrastructure equipment. As always, there are often multiple business/revenue models utilized by different players in the same industry. (2) Hardware As A Service Hardware and software are bundled and sold with annually recurring revenues. Often multi-year contracts are signed to be able to amortize hardware over time, and enable blended gross margins over the length of the contract to be high (approaching 60-75%). This is more common in situations where vendor takes responsibility to upgrade hardware as needed along the way to provide maximum and highest functionality to the customer. Hardware inventory costs are, over time, passed on to suppliers or third party financing parties in lieu of few gross margin points. Consider robots as a service in factories (ala Formic), or AI-enabled equipment (eg Evolv, Lumafield etc). Hardware innovation will require deeper investigation, experimentation, and understanding of business/revenue models to fit customer needs as well as to maximize value for the innovating company. Strong and experienced management teams work hard to minimize sales friction, inventory costs etc, but also maximize gross margins, customer stickiness, and ability to upsell during the life of a contract.

339

37 Comments

🚨 Is California’s AI Regulation Going Too Far? SB 1047 is making waves across Silicon Valley, and the entire tech world is paying attention. While it's clear we need safety regulations to keep AI from causing harm, is this bill overdoing it? 🤔 The big tech companies are raising alarms about the potential for increased legal liabilities ⚖️ and how these regulations could stifle innovation 🛑. They’re concerned that heavy-handed rules will slow down progress and put the U.S. behind in the global AI race. And it’s not just the tech giants feeling the squeeze—smaller startups are also on edge. For them, vague testing standards are like an added weight that could make it harder to innovate and compete. 🚀 Of course, there’s the other side. AI safety is no joke. I’m all for responsible AI, but regulations like SB 1047 seem like a step too far. Are we at risk of over-regulating AI in a way that might protect us now but choke creativity and progress in the future? What do you think? Should we push for more balanced regulation that keeps AI development alive and well while still protecting people? Drop your thoughts in the comments!👇 #AI #Innovation #SB1047 #TechRegulation #SiliconValley #AIsafety #AIregulation #CaliforniaAI #ResponsibleAI #AIethics #TechInnovation #AIoversight #AIrisk

14

4 Comments

Incredible how API unifiers are taking off right now. If I had to build a business with $1M ARR in less than 12 months, I'd start as an API unifier 👇 Nearly all agentic application need roughly the same set of integrations. Calendar (Google Cal, Outlook), communications (slack, teams, email), transcriptions (zoom, otter, fireflies) cloud drive, etc. I can't imagine how much code is being duplicated right now by startups each creating their own integration. There's a huge need to unify into a single API and data schema which agents can work on top of. I'm not the only one to realize this. iLife - API's for logistics Kombo - API's for future of work TravelPOrt - API's for travel Makini - API's for manufactorers Twilio - API's for voice and text messaging Recall.ai - API's for all transcription services TruePill - APIs for telehealth services and prescription fulfillment As AI coding gets better a lot of this will become commoditized or generated on the fly. Long term, the ones who succeed will build a moat but value add services on top of the data they unified.

49

24 Comments

Watching Vinod Khosla talk with Rocky Yu about exponentials at the AGI House. Vinod said you could see the future of the Internet in the 1970s if you watched carefully. So, here are my guesses on what new fields AGI will revolutionize in what order: 1) Creative industries: image generation, music generation and movies 2) Robotics: foundational models will enable much better robotics to be deployed much more widely 3) Biotech: the creation of new drugs, better therapies etc…

34

1 Comment

There you have it! BREAKING NEWS! New model comparison feature up on all Ask Sage, Inc. tenants! This feature enables you to compare prompt results across LLMs on a single screen with RAG, live, file attachment options. Did you know we can support any commercial and OSS models and deploy on any environment (air gapped, cloud, on premise... ?) Great convenient feature!

111

7 Comments

Key insight from Satya Nadella on the Bg2 Pod that's not getting enough attention: SaaS applications as we know them will collapse in the AI era. Why? The business logic layer is moving entirely to AI agents. The future architecture: • Agents will be multi-repo CRUD, not tied to single databases • Business logic shifts from apps to the AI tier • Backend systems become primarily data stores This isn't just a data transformation - it's a fundamental shift in how business logic and communication flows through enterprise systems. Think supply chains operating like real-time trading desks rather than quarterly review cycles. The architectural implications here are massive and under-appreciated. And to make these AI agents from different vendors collaborate with each other, we need agentic connectors - something we have been thinking about solving for a while. Worth a listen -->Agentic discussion starts around the 30 min mark: https://lnkd.in/gvCqyQau). The future is truly agentic.

26

5 Comments