Yannis L.

I am currently working with some of the best Machine Learning Engineers in the US who are looking for new roles. Here are some of the common themes why they are on the market: ➕ Losing confidence in the technical direction of their company ➕ Wanting to move towards a GenAI-focused role ➕ 4+ years, RSUs have vested and want a new challenge ➕ Big tech work is repetitive; they want more high-impact work ➕ Still a lot of restructures and instability ➕ Lack of long-term company vision. Short-term panic affecting work Generally, candidates are very open to startups or large companies. As long as the ML work has a high impact If you are a candidate open to new roles. Get in touch; I have clients actively interviewing this week. If you are a hiring manager, now is a good time to pick up some of the best talent. Get in touch with People In AI 🙂 📧 [email protected]

20

I’ve been fortunate enough to do AI shows with Sam Altman, Yann LeCun & Arthur Mensch. But this show today with David Cahn at Sequoia Capital is the best of all of them: 🧠 AI’s $600BN question 🏃 Why no one will never train a frontier model on the same data center twice 🏗️ Servers, steel & power: The 3 pillars of the future of AI Not one to be missed, my 8 key takeaways 👇 1. The Oligopoly That is Blowing Billions on AI - Microsoft, Azure, & Google now represent $7T market cap. - Of course they will spend billions to protect their oligopoly - What is most interesting is Meta is the only incumbent that is financing this not with cloud being their cash cow. 2. How Startups Could Win with the Cost of Compute Coming Down - Big tech companies are producers of compute & startups are consumers of compute. - Overproduction of compute = lower prices.  - This gives startups higher margins & makes them more valuable. 3. Is Compute the Currency of the Future or Do the Costs Come Down and It Is Commoditised? - We are forgetting “compute” is a euphemism for a $2BN data center in the middle of nowhere. - They have GPUs & liquid cooling systems that need to be constantly optimized & upgraded. - We cannot simply build 15 years worth of compute all at once.  No one's ever going to train a frontier model on the same data center twice. 4. Data Centre Teams and Model Teams Need to Be Coupled - You cannot have separate teams running data centers & building models. - Elon & Zuck control all their data centres. - It will not work as these models get better & better. 5. Why the Data Center Is the Most Important Asset - There is not much difference between these models today. - AI researchers are jumping from one lab to another. - Scaling laws will become more dominant as these models get bigger. 6. Servers Steel and Power: The Three Bottlenecks of AI Three things: 1. Servers: NVIDIA, AMD, Broadcom… 2. Steel: Construction & real estate developers for data centers. 3. Power: All the energy required for AI. - More factories will be built in 12 months for AI and we will see another industrial revolution. 7. You Have to Have a Cash Machine to Play the Big Model Game - Amazon has AWS - Meta has Instagram - Microsoft has Azure - You need a cash machine to compete and it CANNOT be the AI business. 8. There is One Definition of Success in Venture: Are you a Slugger? - If you are generating billion dollar gains, you are a slugger. - Doug Leone, Pat Grady, Andrew Reed are all sluggers. - You cannot be good at this business without being a slugger. (links in comments) #founder #funding #business #investing #vc #venturecapital #entrepreneur #startup #seed #funding #sequoia

190

26 Comments

If you've ever worked with ML models for medical imaging then you know how long the computation time will be. Here's a quick tip help with that. If you deal with CT or MRI scans of hundreds of slices then your model will take minutes to finish. Now imagine having multiple of these models working together for a certain application! Inference time can quickly get out of hands! In such a case, you most likely have a model orchestrator which is a piece of code that sends requests to your models APIs and then collect the outputs and order them. What you should do is leverage asynchronous processing. Meaning, run all of your models in parallel at the same time without waiting for one model to finish before calling the second one. Of course, this supposes that each model is deployed separately in its own API. But what if some models depend on other models outputs? You can still use asynchronous processing, but you'll be picking and choosing which models need to run first and which models need to wait a little bit. A great package to do this in Python is asyncio.

32

If you're day-trading $NVDA (like everyone else I know), you probably want to dig into the numbers they just reported. Fortunately, we've gone ahead and produced a full historical data sheet with every earnings release going back 5 years, with all the non-GAAP granularity you could possibly care about. Download it (link in comments) and check it out yourself. Some highlights: --4.3x YoY sales growth in the key Data Center segment, an acceleration from last quarter --Gross margins hit 78.9% (quite impressive for a hardware company) beating their guidance from last quarter by 190bps, and suggesting better pricing as the quarter evolved --Free Cash Flow of nearly $15bn in the quarter, bringing the LTM total to $39.3bn. With cash generation so high, they will have an unmatched ability to reinvest in their business Wouldn't it be great if you could get this level of detail in minutes on any company, public or private? If you build models for a living, get in touch with us to learn more about how Understory can help.

83

5 Comments

Notion vs Glean reminds me a lot of Tesla vs Waymo... Just like I think Tesla is going to solve the 'software' problem of autonomous vehicles before Waywmo solves the 'hardware' problem... I think that Notion is going to solve 'enterprise knowledge search' before Glean solves the problem of document and knowledge creation... Why? A couple reasons: 1) Just like Tesla, Notion has a huge advantage when it comes to the amount of underlying data and documents to train on... They already have 100M+ users, and likely 10x that in terms of the number of documents, files, etc. in team's workspaces. 2) It's very hard to change people's behaviour re: where/how they create 'work'. It's why a lot of companies have come and gone in that category... As exciting as Glean's AI writing assistants might seem, most people are just fine (or even prefer) a good old fashioned editor/writer. I could be way off here, but this will be an intersting one to follow. Thoughts? Where am I getting it wrong? #ai #enterprise #knowledge #management #notion #glean

3

1 Comment

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

29

Exploring AI Autonomy: testing Claude's Computer Use Building upon my recent experience with AutoGen Magentic-1, I had the opportunity to test Anthropic's Claude 3.5 Sonnet's "Computer Use" feature. This capability allows Claude to interact with computer interfaces, performing tasks such as moving the cursor, clicking buttons, and typing text, effectively automating actions that typically require human intervention. Key Observations: - Task Execution: Claude's ability to navigate and operate within a computer environment showcases a significant advancement in AI autonomy. It can perform complex sequences of actions, enhancing efficiency in tasks like data entry and web navigation. - Integration Potential: This feature opens new avenues for integrating AI into daily operations, potentially reducing the need for human involvement in routine tasks. Comparative Insights: - AutoGen: Excels in orchestrating multiple Large Language Models (LLMs) to collaborate on complex tasks, leveraging their combined strengths. - Claude's Computer Use: Focuses on automating user interface interactions, streamlining workflows that involve repetitive manual tasks. As AI continues to evolve, these advancements underscore the importance of selecting the right tools to enhance efficiency and productivity. Allie K. Miller did a great job to write a blog "How to Use Claude Computer Use to Build an AI Agent for Non-Engineers"; give it a try: https://lnkd.in/dUdQdyeE Prompt I Used: "You are an AI Agent tasked with finding a training wheel for a 14-inch bike within a $20 budget, including shipping. Browse the web naturally to avoid CAPTCHA challenges and ask the user for any necessary details like location, login credentials, or preferred shopping platforms. Identify suitable options, ensuring they meet the requirements, and provide a concise list with item names, prices, and purchase links. Confirm the options with the user to ensure satisfaction before finalizing. Add the selected item from the user to the cart." #AI #Automation #AutoGen #Claude #Anthropic #MachineLearning

9

4 Comments

🎙 Episode 43 of Machine Minds with Bob Rogers, CEO & Co-Founder of Oii.ai is live! Bob's background speaks so itself so imagine what the insights from the episode are like! Background: - PhD in Physics - Expert in residence at UCSF who helped build the first FDA-cleared AI product that saves lives every day - Member of the Board at Harvard IACS - Led Data Science and AI for Intel - Was the Co-founder and Chief Scientist of an acquired healthcare AI company - Co-founded and ran a quantitative hedge fund - Is currently a Chief Scientific Advisor for two companies and a Co-Founder of one of them - Authored 4 books... And I'm sure there's more I'm missing. Check out the full episode wherever you get your podcasts to hear all about Oii as they just came out of stealth! #machineminds #techpodcast #ai #supplychain #startups

10

2 Comments

ML Engineer: "Why should I explain what real-world application I used *sentence piece tokenization* with? The ML team hiring me knows" Human: "Because humans do not know why *RAG* or *Embeddings* matter" To all y'all engineers. Learn the impact of your work. Articulate it succinctly Came across the website copy of threesai.com, what a breath of fresh air In just 23 words It clearly communicates the **why**

4

Finding hiring ML/AI Engineers difficult? Yes + it's expensive. Here are some great start-up/scale-up stats (from Pave's Matt Schulman) from last week #freshcut ▶️ 1 in 5 engineers are getting a signing bonus when hired ▶️ 1 in 4 AI/ML professionals get a signing bonus when hired ✨ Good heads up for candidates who are looking; but also a good non-recurring cash solution to lock in that special candidate (with some retention clauses 😊 ). #signingbonuses #spotbonuses #pave #compensation #scaleups

11

4 Comments

A very useful checklist to follow when you have to put in production Transformer based models

5

Foundational models, or large language models (LLMs), train at the scale of the entire internet to acquire the vocabulary, syntax, and narrative structure of human language. Fine-tuning involves using a smaller, domain-specific dataset to grant additional specificity to a pre-existing foundational model that has already acquired patterns and features from massive underlying training datasets. By utilizing the existing knowledge embedded in a pre-trained language model, we unlock high-performance language understanding, but fine-tuning helps to get the right answer - not just an answer that sounds right - with a minimum of hallucinations or incorrect info. In healthcare, out-of-the-box, foundational models are often too generic to sufficiently answer the precision of a healthcare query. Foundational language models are trained on a massive scale of data, which includes jokes, incorrect information, misunderstandings, and misinformation, along with the sum total of human knowledge. The problem is, if you ask a large language model to give you factually correct information when it has been trained on BOTH accurate and inaccurate information - you’re playing a bit of a dangerous game. Enter fine-tuning language models. In the old days, we used to build a single model for a single task - for example, classifying procedures into evidence-based or not as laid out by clinical practice guidelines, for patients with particular disease markers. Each disease would have its own model, which took a long time to build and didn’t scale particularly well, and we struggled with the sparsity and inherent high-dimensionality of healthcare data. By fine-tuning LLMs, however, we're able to specialize those foundational models to achieve greater domain specificity and specialization, leverage the unique predictive and planning capabilities of LLMs, and cut down on the incorrect information that such models inevitably produce, while still preserving the underlying language prowess of the LLM. If improved accuracy and specificity wasn't enough of a selling point, there's another notable upside to fine-tuning foundational models for use in a clinical context. In terms of computational efficiency, using LLMs out of the box on specific tasks can require massive compute (and spend) that amounts to computational overkill for the problem. Some LLMs charge per query, some by compute load or time - either way, as a business running LLMs in a production environment, minimizing the number of LLM calls means optimizing your compute expenditures, too. When LLMs fail at scale in healthcare, it's usually along the fault line of failure to specify adequately. Fine-tuning large language models along domain-specific lines is one way to effectively productize genAI for healthcare use cases. #genAI #LLM #ai #healthcareAI #healthtech #largelanguagemodels #generativeAI #chatGPT

61

2 Comments

Some of you know that #observability is an old passion of mine [https://lnkd.in/gD9abpjT], and I am really excited to be working with Samuel Colvin and the rest of the Pydantic team. Samuel has earned his stripes with the Pydantic open-source data validation library which is a beloved product, and he is applying the same knack for #DX and design to observability. Logfire is the easiest way to add monitoring to a #python app. And if you are already using Pydantic, it's a no-brainer!

93

6 Comments

LLMs have become more capable with better training and data. But they haven’t figured out how to “think” through problems at test-time. The latest research from Scale finds that simply scaling inference compute–meaning, giving models more time or attempts to solve a problem–is not effective because the attempts are not diverse enough from each other. 👉 Enter PlanSearch, a novel method for code generation that searches over high-level "plans" in natural language to encourage response diversity. PlanSearch enables the model to “think” through various strategies before generating code, making it more likely to solve the problem correctly. The Scale team tested PlanSearch on major coding benchmarks (HumanEval+, MBPP+, and LiveCodeBench) and found it consistently outperforms baselines, particularly in extended search scenarios. Overall performance improves by over 16% on LiveCodeBench from 60.6% to 77%. Here’s how it works: ✅ PlanSearch first generates high-level strategies, or "plans," in natural language before proceeding to code generation. ✅ These plans are then further broken down into structured observations and solution sketches, allowing for a wider exploration of possible solutions. This increases diversity, reducing the chance of the model recycling similar ideas. ✅ These plans are then combined before settling on the final idea and implementing the solution in code. Enabling LLMs to reason more deeply at inference time via search is one of the most exciting directions in AI right now. When PlanSearch is paired with filtering techniques—such as submitting only solutions that pass initial tests—we can get better results overall and achieve the top score of 77% with only 10 submission attempts. Big thanks to all collaborators on this paper including: Evan Wang, Hugh Zhang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen H., Sean Hendryx, Summer Yue 👉 Read the full paper here: arxiv.org/abs/2409.03733

180

2 Comments

The Information explores how OpenAI's reasoning models could help counteract potentially slowing progress from pretraining of larger more sophisticated models with more incremental gains over earlier models. It also notes that "relatively few developers" are using o1 models. While I haven't seen the full-fledged o1 models (and am certainly not a developer), the current models sometimes think hard about how to break code -- have seen them replace functional authentication with placeholders that you have to go back and update later, or swap out the correct variables with wrong ones -- for instance upper- or lowercasing something you didn't ask for, or adding an underscore where you didn't want one. At present, I think it still is better to have tighter feedback loops with the user -- otherwise, the genAI is thinking long and hard -- while a game of telephone can ensue where mistakes sometimes increase in the process of reflection -- about something that might not be what you wanted. A snippet from the paywalled article. The Information has done a lot of unique reporting on the subject.

4

1 Comment

$2 H100s: How the GPU Bubble Burst Last year, H100s were $8/hr if you could get them. Today, there's 7 different resale markets selling them under $2. What happened? The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and decline in new foundation model co’s. Rent instead. https://lnkd.in/gpQEidmC

2

1 Comment

Anthropic Claude just announced major upgrades to their AI lineup, and it's a game-changer for the industry. 𝗞𝗲𝘆 𝗵𝗶𝗴𝗵𝗹𝗶𝗴𝗵𝘁𝘀: - An upgraded Claude 3.5 Sonnet with significant improvements in coding capabilities - now achieving 49% on SWE-bench Verified, leading all publicly available models - Introduction of Claude 3.5 Haiku - matching Claude 3 Opus's performance while maintaining cost-effectiveness and speed - Most notably: The launch of computer use capabilities in public beta - Claude can now navigate computers like humans do, using cursors, clicking, and typing! Early adopters like Asana, Canva, and DoorDash are already exploring these capabilities, with some tasks involving hundreds of steps. GitLab reports up to 10% stronger reasoning across DevSecOps use cases with zero added latency. What excites me most is how this bridges the gap between AI potential and practical implementation. I see this being used in cases where you are daily copying data between multiple systems, developers context switching between applications, juggling multiple tools etc. We're moving from theoretical capabilities to real-world applications that can transform how we work. #ai #claude

9

"At the application layer, economies of scale aren’t a viable mechanism for defensibility either — because computational costs are an order of magnitude lower compared to the model layer. Your ability to pay for an OpenAI’s API or compute for your app isn’t a sustainable advantage over a future competitor. Some applications like Character.ai have attempted to avoid this problem by vertically integrating, i.e. building their own customized models. Here again, the efficacy of economies of scale is questionable for the same reasons I mentioned earlier. This leaves network effects and switching costs as the only realistic modes of defensibility for most applications. So how do you create real, meaningful network effects with an AI application? Based on what we’ve talked about so far, the app needs to have an AI-enabled multiplayer interaction which involves a lot more than AI-generated output created by another user. These applications use AI to enable a (higher friction) multiplayer interaction that was previously impossible." https://lnkd.in/gW4zPGGp

3