Sal Huante

AI Just Got Leaner, Faster, and More Affordable – What Does That Mean for Your Biz?? Meta’s latest release, Llama 3.1 8B, is a game-changer in AI efficiency. This model runs with 50% fewer parameters while maintaining 98% of the accuracy of the previous model. How? Through smarter design, like 2:4 sparsity patterns and GPU-optimized architecture. While everyone else was distracted by Gpt, Meta doubled down on efficiency innovations. For businesses considering AI, the implications are becoming clear: 30% higher throughput and 1.8x lower latency mean faster AI responses and streamlined workflows. Lower computing costs and reduced infrastructure requirements make AI adoption more accessible. Specialized tasks like coding, data analysis, or customer service can now be powered by AI that’s efficient without sacrificing capability. Picture this: your enterprise scales customer support without additional servers, analyzes massive datasets in half the time, or deploys AI-powered marketing campaigns faster than ever—all while keeping costs down. Efficiency in AI is no longer a luxury; it’s becoming a standard. As technology moves toward leaner, high-performance models, businesses that adopt these solutions will operate smarter, faster, and more cost-effectively. I dunno, just another way to think about how a 30% efficiency boost could transform operations.

1

Example of how increased inference speed creates qualitatively new opportunities: Have a conversation with Vapi at https://vapi.ai. Thanks to ultra-fast inference of things like Llama-3 running on Groq, you can have a back-and-forth conversation that feels very human. You can even interrupt Vapi mid-conversation and it will just handle that seamlessly. Pretty awesome.

3

The cost of training top models is increasing 2-3X per year, and at this rate will reach $1 billion per model by 2027. The findings are from Epoch AI and follow their recent analysis of increases in compute for foundation models running at 4-5X per year. They've also provided some interesting data on the relative breakdown of cost components like staff, hardware, and energy. A few thoughts after reading this: - I would assume staff costs won't increase as much as hardware and energy costs, so the relative contribution of human labor to future AI models will decrease, in yet another sign of capital's increasing dominance over labor in the industry (just look at NVIDIA's ever-rising stock price). - If compute is increasing 4-5X per year but cost is increasing 2-3X, is it safe to assume that training runs are getting more efficient? - Some people, like Anthropic CEO Dario Amodei, have said we'll see $1 billion training runs in the next year. Does this mean the 2-3X pace is about to increase? That Epoch is wrong in its estimate? Or that Amodei is overstating the cost, possibly to put fear into competitors who can't keep pace? - Only a few companies are going to be able to train future models. We're likely going to see further consolidation. Perhaps we get to a future with only two large general purpose models, like the Windows/MacOS and Android/iOS duopolies we've seen in the past. Article here: https://lnkd.in/g5SejXen

7

I think people are missing the arbitrage opportunity for ChatGPT Pro tier: To create "human wrappers" around ChatGPT. ChatGPT Pro costs $200/month and offers unlimited o1 Pro (near superhuman intelligence) and unlimited advanced voice mode (unlimited conversational AI). Yes, $200/month is beyond what many companies might pay per user, at least initially, because they might not have sufficient need. But those companies might pay a consultant $20,000 for a valuable task that the consultant could use ChatGPT Pro to complete in less than a day.

19

13 Comments

Ed Zitron’s opinion piece is really interesting. I had similar thoughts that I could never put together as effectively as he did. I remember thinking that Raghavan’s placement over Ads and Search was the end of Google’s original soul. A public affirmation there was **officially** no wall between search and ads.  Having experienced the poor quality of Google's results over the past few years, it's hard not to believe there was a reason for it. Do I need to read through 12,000 words of "content" to find the pancake recipe at the bottom of a site in some weird network of lifestyle yoga bloggers? However, I think his inability to see the upcoming storm (also mentioned in the emails) and allowing another company (OpenAI / Microsoft)to beat you (Google) to death with your own invention (transformers) is the much graver issue. This piece by Matt G. Southern is a good overview. https://lnkd.in/eVYCsHiD

78

13 Comments

As the Google anti-trust lawsuit continues, it made me think a bit about disruptors in the industry. I started at Google in 2007. At the time, Mapquest was the top directions app - Google Maps didn't exist. It was built while I was there and rose to prominence because it was that good. At the time, iPhones and Blackberrys were the main phones if you wanted something "smarter". I had a decked out flip phone. While there, they launched the first Android device - the G1. It only worked on one network and they gave all Googlers one as a holiday gift to increase usage. Within a few years, and integration with Verizon's CDMA network, Android because a force in the market and Blackberry didn't pivot to catch up. At the time, Microsoft was the main choice for workspaces with Word, Excel and Powerpoint. There was a cheap knockoff version, but that was about it in terms of choices. While at Google, they created Docs, Sheets, Drive, etc. I remember hating having to use the original versions of Sheets and Slides and begging to keep Powerpoint. Now, I'm begging to keep the Google versions as Sharepoint (Microsoft's pivot to stay relevant) just doesn't work as well. Other things that weren't around that I can think of: Photos, Earth, Chat, Meet, all of Google's social efforts (which completely bombed), and even Gmail was still in "beta" when I started. I think there is still plenty of room for disruptors - just make a product that people want.

1

1 Comment

Generative AI ROI : A $1 Trillion Bet—But Is It Paying Off? Goldman Sachs recently reported that enterprises are set to spend $1 trillion on generative AI in the coming years, with little to show for it. Jim Covello, Head of Global Equity Research at Goldman Sach, argues: “The substantial cost to develop and run AI technology means that AI applications must solve extremely complex and important problems for enterprises to earn an appropriate return on investment (ROI).” I respectfully disagree, on multiple levels. 1️⃣ Cost-Effective Implementation: Contrary to popular belief, adopting AI doesn't always break the bank. Most enterprises don't need a custom-built LLM; they just need a solid generative AI platform. It's like choosing between building a car from scratch or leasing a reliable model—why reinvent the wheel? 2️⃣ Value Over Complexity: AI doesn't need to solve quantum physics to be useful. Take customer service: automating level 1 support might seem simple, but it can save companies 2% to 15% of their revenue. Conservatively, 50% of customer service is spent on level 1 support. That's not pocket change—it's a game-changer. 3️⃣ ROI That Speaks for Itself: While some predict doom and gloom, real businesses are seeing real results. We know this because our customers are consistently achieving over 500% ROI. When the investment is reasonable and the problem solved is valuable, outstanding returns naturally follow. Don't let fear, uncertainty, and doubt cloud your judgment. Companies are already reaping the benefits of generative AI. The future isn't just coming—it's here, and it's profitable. #generativeAI #customerservice #ROI

15

Check out our curated list of #MarTech and #AdTech startups that have received funding. You can access the Marketing and Advertising funded startups list at any time! Link to the list https://lnkd.in/ehwz2_tg #startup #startups #vc #venturecapital #investmentbanking #privateequity #managementconsulting #b2b #founders #entrepreneurs

1

Google’s UGC Push: Sullivan Explains The Shift In Search Results from Matt Southern on Search Engine Journal My thought: Maybe as SEOs purge all that "non-performing" content from Websites they'll mistakenly remove some of the crap they've been creating "for SEO" and help clean up the indexed Web. But I'm not holding my breath. #seo #webmarketing #searchengineoptimization #google #searchengines #digitalmarketing https://lnkd.in/ezu42CAg

2

Meta Llama 3, the next generation of our state-of-the-art open source large language model with 8B and 70B parameters that can support a broad range of use cases. Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. Give Meta AI a try - https://meta.ai . Read more here - https://lnkd.in/gEE5w9ZG Meta #metaai

9

https://lnkd.in/ecuZDqKz. https://lnkd.in/ecuZDqKz Liquid AI Introduces New Class of Foundational Models, Advancing AI Performance and Efficiency Boston, September 30, 2024—Liquid AI, a pioneer in artificial intelligence innovation, is set to redefine the AI landscape with the launch of its Liquid Foundational Models (LFMs). These models bring unprecedented levels of performance and efficiency to generative AI, offering powerful solutions for industries looking to harness artificial intelligence more effectively on device or on server. A technical whitepaper about model performance will be available on September 30, 2024, at 11 am EST. Key Highlights: ● LFMs Introduce a New Scaling Law for Generative AI: Markedly improving quality, cost-effectiveness, and power efficiency compared to current GPT models. ● Innovative Architecture: Leveraging dynamical systems and advanced mathematics, LFMs achieve superior performance with fewer parameters. ● Initial Model Release: The lineup includes models with 1B, 3B, and 40B parameters, each setting new benchmarks in their respective categories. ○ LFM-1B: A 1.3-billion-parameter model achieving the highest scores across benchmarks in its class. ○ LFM-3B: A 3.1-billion-parameter model that outperforms many 7B and 13B models, making it ideal for edge deployment and resource-constrained environments. ○ LFM-40B: A 40.3-billion-parameter Mixture of Experts (MoE) model designed for complex tasks, offering performance comparable to much larger models while remaining cost-effective. Real-World Applications: • Enterprise: LFMs enable the design of more powerful AI assistants and analytics tools that can run efficiently on existing infrastructure, reducing operational costs. • On-Device: LFMs prioritize privacy, low latency, security and offline accessibility, making them suitable for mobile and edge deployments. These models have demonstrated impressive results on standard LLM evaluation benchmarks. Additionally, LFMs maintain near-constant inference time and memory complexity, allowing for longer input sequences without significant increases in computational resources. Advancements in AI Architecture: LFMs introduce a new design space for foundational models by employing adaptive linear operators rooted in dynamical systems and signal processing. This approach unifies and extends existing computational methods in deep learning, providing a systematic way to explore architectural possibilities. The models are highly adaptable, optimized for various hardware platforms including NVIDIA, AMD, Qualcomm, Cerebras, and Apple devices.

46

🤯 Imagine the CEO of the world’s hottest company says the next trillion-dollar opportunity is something your startup has already been delivering results with for over a year in stealth! #JensenHuang and #MarkBenioff have the world buzzing about #AIAgents. 🐝  But at #CalvinBallTech, we've been quietly delivering game-changing results with them for over a year!  🤫 Don't get caught up in the hype 🙅‍♂️ Stay tuned. CalvinBall Tech is about to come out of stealth, and we're bringing proven AI Agents that have already solved critical problems for real businesses. 💪 Here’s a quick couple of questions for anyone showing you fancy roadmaps and dazzling demos: Do they have proven results already? Can they start delivering tomorrow? Gurnoor Dhillon Ratu Nadiah KhairunnisaSahal ZainEva ZuliyaFaris -Yoga Wigardo Dharnesh GordhonAntoine de CarbonnelFrançois Walewski #AICanSolveThat #AISuperAgents #AIRevolution #FutureOfWork #EnterpriseAI #CalvinBallTech #StartupJourney  #CalvinballerCode

17

Truly impressed by OpenAI’s decision to include Sora for all Plus subscribers, bundled with their existing offerings. There’s been plenty of discussions around which tool gives the most value for $20—whether that’s Claude Pro, GPT Plus, or even Perplexity (for better search). But now, with Sora available and DALL·E 3 already included, GPT Plus is pretty much a no-brainer. Can't wait to try it ! #Sora #OpenAI #GenerativeAI

8

Episode 1 of the "Worthy Successor" series on The Trajectory is with none other than, Nick Bostrom. Themes: -- Nick's Worthy Successor criteria -- How AGI might be able to preserve some degree or "human values" -- What Nick hopes for (and doesn't hope for) around AGI governance See the full episode on YouTube, and be sure to comment with your thoughts: https://lnkd.in/eXCSJvnC Subscribe to Apple podcasts: https://lnkd.in/eqps-YaN The Trajectory newsletter (never miss an interview) http://bit.ly/TrajectoryTw I respect Nick tremendously. It is safe to say that, 13 years ago, there were 3 driving forces that lead me to dedicating myself to the cause of a worthy successor: - Emerson's essays - Kurzweil's "How to Create a Mind" - The early papers of Bostrom He's a genius. I'll be bringing some of the insights from these discussions into my upcoming talks / meetings with the UN / OECD.

7

2 Comments

AI people freaked out about Meta's new AI models this week If you don't really get why, no worries, here's your cheat sheet: ▪️ What's different about Meta's model vs ChatGPT vs Google's Gemini, etc. Two words: open source ▪️ Why is open source important? We wrote about this in our newsletter 2 weeks ago, but it comes down to two big things: 1. Any company can host open source AI themselves and use it (WITHOUT sharing their data with a 3rd party) 2. Any company can build on open source AI and improve it (AND own the improvements) ▪️ Why is Llama 3 important? Open source AI models have traditionally been significantly worse than their paid/closed source alternatives (ChatGPT, Google Gemini, Anthropic Claude) But the early benchmarks are showing: - The small version of Llama 3 beats other open source leaders like Mistral - The larger version of Llama 3 beats closed source leaders like Google's Gemini Pro 1.5 and Anthropics' 2nd best Claude 3 model ▪️ What will happen next? Every time a new open source AI model is released, a ripple effect happens. AI developers will start tinkering, they'll figure stuff out, and new variants will emerge that are better and better. You'll also see costs drop across the board as people have more, cheaper options to craft their AI solutions. Big things coming. Like/repost if this was helpful! And comment any questions you have on this or other parts of shaping your AI strategy 👇 p.s. if you want to try Llama 3 out yourself, it's free at: https://www.meta.ai/

147

27 Comments

OpenAI and Google have been slugging it out in the open to be at the forefront of GenAI, but there’s a new giant stomping into the ring: Amazon. Last Thursday Amazon CEO Andy Jassy wrote in an open letter to shareholders that, on top of leveraging what’s already out there, Amazon is getting to building foundational AI models. In an all-hands call, Jassy reportedly told employees that Amazon’s going to be a leader in AI. If this feels like a scramble to catch up, that’s because it probably is. Because of that, it could go badly. After all, Google rushed Gemini out the door and suffered PR disasters because of it. There’s no reason to think Amazon won’t make the same mistakes. But both Google and Amazon have money to burn, and a guns-blazing approach to throwing money into innovation is something that Amazon’s well known for. They can afford to fail until they succeed. That marks another major player in the genAI space to keep an eye on! Are there any other upcoming genAI projects you're excited for? Check out this writeup about Amazon's letter to shareholders for more info: https://lnkd.in/eD_pShwZ #ArtificialIntelligence 

17

What Modularizing DSPs Taught Me About AI and RAG Integration Ad tech taught me how to solve big, complex problems at speed and scale. DSPs are designed to process billions of transactions in milliseconds, matching advertisers to ad space with precision. But DSPs aren’t just ad delivery engines—they’re blueprints for scaling intelligent systems. What if we applied DSP modularization to power AI agents and Retrieval-Augmented Generation (RAG)? These technologies are reshaping data retrieval, problem-solving, and personalization. Here’s what I’ve learned: 1. Scalability Wins Every Time DSPs handle billions of impressions daily while scaling gracefully. AI and RAG must deliver insights or decisions at scale without sacrificing speed or accuracy. Lesson: Build systems that adapt to volume surges seamlessly. 2. Real-Time Optimization is Essential DSPs continuously refine outcomes in milliseconds. AI and RAG need to optimize outputs instantly to meet user expectations. Lesson: Systems must process feedback instantly or risk irrelevance. 3. Modularity Drives Flexibility Modular DSPs integrate independently functioning components like bidding and targeting. AI and RAG should follow the same approach, allowing fast upgrades and new tools without disruption. Lesson: Modularity future-proofs your system. 4. Data is Key, But Privacy is Paramount DSPs succeed because they handle data responsibly. AI-driven systems must go beyond compliance and prioritize trust and transparency. Lesson: Trust must be baked into system design. 5. It’s Always About the User DSPs align advertiser intent with user behavior. AI and RAG systems must similarly focus on delivering what users truly need. Lesson: User-centricity is the foundation for success. Where Do We Go From Here? DSPs are blueprints for innovation. Their principles—scalability, real-time optimization, modularity, and user focus—can transform AI and RAG systems. The question isn’t whether DSP lessons can shape the future. The question is: Will we apply them fast enough? #ArtificialIntelligence #AdTech #Innovation #MachineLearning #DataPrivacy #ModularDesign #DSP

2

1 Comment

I enjoyed learning about the Reflection technique from Andrew Ng, where you ask AI to do a task ("Summarize the Acme deal"), then ask it to reflect on its answer ("Check your answer carefully and highlight any potential hidden challenges"). This extra "Reflection" step can dramatically improve the AI answers -- even allowing older models to outperform newer models. I'd be very interested to see someone apply this technique to a  #BuiltWithEinstein example with an Einstein Copilot Action and an extra Reflection Action that improves the original answer. Full talk from Andrew Ng can be watched here ->

13

2 Comments

A novel loss function and a way to aggregate multimodal input data are key to the dramatic improvements: https://lnkd.in/eZKurFYs #AmazonScience #ComputerVision #GenAI #ML

24

2 Comments