Claire Froessel

🚀 Looking to streamline fine-tuning for models like Llama 3, Mistral, and Mixtral? Our latest blog shows how Anyscale simplifies LLMs fine-tuning with Ray: 🔑 Key takeaways: • Scalable fine-tuning with LLM-Forge • Model Deployment with RayLLM Read more: https://lnkd.in/gQwD4A2U

34

🚨 Big News from Meta Connect! 🚨 Meta just dropped its Llama 3.2 models on Watsonx, including the first multimodal 11B and 90B Vision models! Key Highlights: - Image-in, text-out support for tasks like document understanding and image captioning. - High-res image support (up to 1120x1120) for classification, OCR, and more. - Lightweight Llama 3.2 models (1B & 3B) run on most devices with low latency and enhanced privacy. - New multimodal safety model: Llama Guard. Meta continues leading with efficient, open-source AI models! Groq now support Llama 3 models for even faster AI Performance #AI #Meta #Llama32 #TechInnovation #MultimodalAI #Watsonx #genai

26

Building Optimized RAG with LlamaIndex + DSPy 📈 We’re excited to announce a comprehensive set of integrations with DSPy that let you combine DSPy’s PyTorch-esque syntax and optimization capabilities with the comprehensive set of data+orchestration tools around RAG/agents that LlamaIndex offers. If you prefer to use DSPy Signatures to define your prompt+LLM input/outputs, you can do that. If you prefer to use LlamaIndex prompt modules and leverage DSPy optimization capabilities, you can do that. If you want to repurpose existing optimized DSPy predictors as LlamaIndex prompts, you can do that too 🔥 The three integrations are as follows: 1️⃣ Build and optimize Query Pipelines with DSPy predictors  2️⃣ Build and optimize Query Pipelines with LlamaIndex Prompts, but use DSPy to optimize 3️⃣ Port over DSPy-Optimized Prompts to any LlamaIndex Module with the `DSPyPromptTemplate` syntax. A bonus ✨: Learn how to define custom evaluator functions using LlamaIndex evaluators that still plug into the DSPy optimizer Thanks to both Omar K. and Arnav Singhvi for the help with this. Full cookbook here: https://lnkd.in/gCZjqui4 

412

13 Comments

Why CoT Works: Iterative Computation for Serial Problems Ever wondered when to use fancy Chain of Thought (CoT) prompting, and when regular ChatGPT will do? Here's the key: Serial problems NEED CoT. Turns out, without CoT, we can only solve problems that have fast, parallel solutions. Let's get practical: - Travel Planning: Imagine an AI travel agent. CoT lets the AI figure out your destination, considering your preferences and budget. Then, it factors in dates (weather and your schedule), books your stay, and suggests activities based on the destination's attractions and your interests – each step builds on the last, ensuring a perfect trip. This is a classic example of a serial problem – each step depends on the outcome of the previous one, making it impossible to ask five people to do these steps in parallel. - Sentiment Analysis: This one ISN'T serial. This is a parallel task! A regular transformer can analyze a review for positive/negative/neutral vibes all at once. Imagine it like asking multiple people to independently assess the sentiment of the text simultaneously. They don't need to know the results of each other's analysis or follow any specific order. The Tech Deep Dive (Simplified!) CoT basically lets transformers tackle way more complex problems. Without CoT, they're limited to AC0 problems (solvable by constant-depth circuits). But add CoT, and now we're talking problems solvable by log-depth circuits (NC1 complete problems). Want proof? Check out the paper "Chain of Thought Empowers Transformers to Solve Inherently Serial Problems" (https://lnkd.in/giCaSGA5). Author: Zhiyuan Li, Hong Liu, Denny Zhou , and Tengyu Ma The Takeaway: Next time you've got a problem for your transformer, ask: Is this serial (step-by-step), or can it be done in parallel? That'll tell you if you need the power of CoT. #ChainOfThought #Transformers #CoT #LLM #AI

26

5 Comments

On the last Scala Space Podcast we have explored Smithy IDL with Olivier Melois (@meloisolivier), the author of Smithy4s toolset and principal engineer at Disney. 1/2 We have discussed the philosophy of Smithy, its genesis, compared it with other IDLs, talked about what kinds of problems it solves best and who can benefit from adopting it the most. Watch it here: https://lnkd.in/g_NdzPSV 2/2

6

1 Comment

Easily generate one billion embeddings in under a day... and with limited budget! How? With Ray Data from Anyscale and Pinecone! Cheng Su and Roy Miara present this can't-miss session during the #DataAISummit tomorrow. Catch it in-person or online.

43

2 Comments

GRAPHRAG: NEW TOOL FOR COMPLEX DATA DISCOVERY NOW ON GITHUB The challenge and opportunity for LLMs is solving problems with data they haven't been trained on. This allows for new possibilities in data analysis, like identifying themes and concepts. In this post, XUAN ANH NGUYEN and Bien Vo from our engineering team provide insight into GraphRAG introduced by Microsoft Research, a significant advancement in enhancing LLM capabilities. - GraphRAG is a data pipeline and transformation suite designed to extract meaningful, structured data from unstructured text using the power of Large Language Models (LLMs). - RAG (Retrieval-Augmented Generation) combines retrieval-based and generation-based methods to continuously improve the performance and accuracy of language models that require up-to-date external knowledge. - Traditional RAG utilizes document retrieval through vector similarity search. - GraphRAG uses LLMs to extract information, build graphs, and summarize communities to provide context. - Process Overview: User Query → Document Retrieval (Search and Indexing) → Contextual Integration (Selection and Formatting) → Response Generation (Using Language Model) → Generated Response. For more details, check out the official announcement: https://lnkd.in/gvRsrPn3 https://lnkd.in/gvYHw49f #dwarves #software #graphrag #LMS — Dwarves Notes (https://memo.d.foundation/) combine our team’s collective know-hows, R&Ds, and operation approaches. Connect and learn alongside other tech fellows: - Discord: discord.gg/dwarvesv - Github: https://lnkd.in/gZZ2eZMu - Website: https://d.foundation

3

🚀 Innovating Ethereum's Liquidity 🚀 Eigenlayer revolutionizes Ethereum by enhancing security and reducing costs for innovation layers. Key points: 🔹 Blockchain Evolution: Ethereum separates innovation from trust, enabling token creation without starting from scratch. 🔹 Re-Staking Purpose: Eigenlayer’s Re-Staking shares security between service and protocol layers. 🔹 AVS Explained: Active Validation Services (AVS) are crucial for system security. 🔹 Eigenlayer's Solution: Pooled security via restaking and free-market governance. #Ethereum #Eigenlayer #Blockchain #DeFi https://lnkd.in/gbtP8_89

1

I couldn't be more proud to write my first official 🤗 blogpost 🎉! I explore the often overlooked yet crucial topic of profiling LLM deployments, specifically with TGI's Benchmarking Tool. It’s a vast area but so vital for understanding the performance nuances of our models, especially for different use-cases. Have you ever encountered surprises or challenges while profiling LLMs? I’d love to hear your experiences and insights! Check out the blog and let’s discuss! 📈 https://lnkd.in/gwK56EXa #LLM #GenAI #TGI #performance

205

7 Comments

🚀 Anthropic ‘s Model Context Protocol (MCP) is revolutionizing how AI connects to your data sources! Gone are the days of endless custom integrations and repetitive coding. With MCP, developers get a unified protocol to seamlessly connect LLMs like Claude to data sources—from GitHub to databases—through a single integration. 🔗 Key Highlights: • Simplified Integration: One protocol for all your data connections. • Built-In Security: Keep data control with the server—no API key sharing. • Boosted Productivity: Connect Claude to GitHub, create repos, and make PRs in minutes. MCP simplifies AI integration, empowering developers to skip the setup and focus on innovation. 🌟 #AI #DeveloperTools #Productivity #Anthropic #aiassistants #aiagents https://lnkd.in/dUj6EzVx

7

Today, we’re launching Substrate. We’re also announcing our $8M Series Seed led by Lightspeed. We believe the most robust, reliable, and productive uses of AI are when many models are used in coordination with each other, with regular programming mixed in. Most people building with AI already know this. But the available tools are suboptimal, unless you work for Google. For the rest of us, we’re left creating an unwieldy mess of chained API calls to multiple providers, with many roundtrips in between. We took a hard look at this state of affairs, and recognized how much it is stifling progress. Building large multi-step AI workloads requires sophisticated high-performance tooling and infrastructure. Nobody wants to deal with more tooling and infrastructure… but everyone would benefit from simple, intuitive interfaces that abstract away a powerful system underneath. No tooling, no infrastructure – just elegant abstractions.

27

1 Comment

The best part of investing at the earliest stages of company building is every once in a while you meet founders who can see the future. Ben Guo & Rob Cheung are two of those founders and after a year of building in stealth they're launching ꩜Substrate, the first inference API optimized for multi-step AI workloads. 👉 https://substrate.run With ꩜Substrate, you connect nodes from a curated library that includes optimized ML models, built-in file and vector storage, a code interpreter, and logical control flow. By simply connecting nodes, you describe a graph workflow, which Substrate then analyzes and runs as fast as possible. Entire graphs of many nodes will often run on a single machine, with microsecond communication between tasks. We are thrilled to participate in their 💵 $8M Seed led by Lightspeed (Guru Chahal/Nnamdi Iregbulem) alongside our friends at South Park Commons (Arian Agrawal), Craft Ventures, Red Swan Ventures (Eric Rubin/Sandy Cass), joined by amazing 🧠 founders and 👨‍🏭 operators like Guillermo Rauch, Immad Akhund, William Gaybrick, Chris Best, Gokul Rajaram, Shreyas Doshi, Jana Messerschmidt, Michael Manapat, and Matthew Hartman.

23

1 Comment

OpsMonsters Typhoons : Why LLMs are not Good for Developers ? Large Language Models (LLMs) like ChatGPT have revolutionized how we interact with technology, effortlessly understanding and generating human-like text. But when it comes to coding, these AI marvels still have a lot to learn. Here’s why: 1. Language Barrier Different Worlds: LLMs are trained on text, which is very different from code. Text is full of variety, while code follows strict rules. Token Trouble: LLMs break text into pieces (tokens) to understand it. This works well for text, but code often has patterns and structures that tokens miss. For example, code indentation is super important, but tokenizers often ignore it. 2. Short-Term Memory Limited Focus: LLMs can only "remember" a small amount of text at a time. Code often has connections between different parts, which LLMs can miss. Big Picture Problem: Long codes with complex structures are hard for LLMs to handle because they can't see the whole thing at once. 3. Learning to Predict One-Way Street: LLMs learn to predict the next word based on what came before. This is great for writing, but code often needs to look both forward and backward. Better Training: New models are learning to look at code from both sides, which helps them understand and generate code better. Final Thoughts While newer iterations of GPT models show improved coding capabilities, they do not necessarily address the core issues directly. Typically, these models use the traditional encoder-decoder transformer architecture and are pre-trained on code bases to develop a strong prior for human-like coding patterns. Task-specific fine-tuning with smaller datasets further enhances their performance. However, despite promising results from fine-tuning and integrating additional components like the ChatGPT code interpreter, some researchers advocate addressing these challenges fundamentally. This approach aims to evolve LLMs beyond relying solely on maximum likelihood estimations to adopting performance-aware code generation strategies. #LLM #ChatGPT #COPILOT #OpsMonsters Don't miss a Geek! Follow us for your daily dose of tech news, insights, and more...

4

2 Comments

Thank you, Sion Smith, for having us! 🎙 In this talk, Yingjun Wu introduces RisingWave, a scalable streaming database that utilizes #S3 as its primary storage. From underlying technology to real-world scenarios, Yingjun illustrates how RisingWave supports heavy traffic workloads in enterprise-grade production environments. #risingwave #streamingdatabase

3

Exciting News! Grok 2 is Here: Revolutionizing AI Interaction! 1. Real-Time Updates and Image Generation: Grok 2 integrates with X (Twitter) for instant data and image creation through Flux 1, enhancing both functionality and user experience. 2. Superior Performance and Accessibility: It surpasses GPT-4, Claude 3.5, and Llama 3 in coding, logic, and math, available through Premium ($8/month) and Premium+ ($16/month) subscriptions. 3. Versatile and Recognized: Offering a sleek new interface and two model options (Grok 2 and Grok 2 Mini), it's already ranked #3 on the LMSYS leaderboard. Grok 2 is your all-in-one AI assistant for questions, writing, and coding. #grok2 #generativeai #llm ##stateofart https://x.ai/blog/grok-2

17

1 Comment

We have ONE final open role at Pod: ⚙️ Senior #ML Engineer In the words of our current ML lead Guillaume: “Stop bugging me for a quote, we just need someone who gets LLMs and wants to build cutting edge stuff”. Appy here 👉 https://lnkd.in/d6cezyzK Tag/forward to someone you know! #Pod #salesai #hiring #ai #salescoaching

12

Machine Learning Engineer - Founding Team - London £200k If you answer yes to 4/5 of these questions, then you want to check out this job (link in comments) 1️⃣ Do you have experience with quantization techniques (e.g., PEFT, DeepSpeed, TensorRT) to optimize ML models? 2️⃣  Have you ever managed or worked with a multi-node HPC cluster? 3️⃣ Are you familiar with Deep Learning using PyTorch Library? 4️⃣ Do you have experience with Kubernetes container orchestration? 5️⃣  Have you worked with some of the recent Large Language Models (e.g., OpenAI, LLaMA)? #LLM #ArtificialIntelligence #Quantization

3

2 Comments

⏰Join us at 11am ET TOMORROW for the next YugabyteDB Community Open Hours.⏰ In this episode, Yugabyte experts Franck Pachot and Denis Magda will be joined by Yugabyte Senior Full-Stack Engineer, Brett Hoyer, who will share how to build a multimodal generative AI application using PostgreSQL pgvector. ➡️ First, they'll create an application prototype that takes an image and text as input to generate a detailed image description. ➡️ Next, they'll use an embedding model to convert these descriptions into multi-dimensional vector arrays. ➡️ Finally, they'll demonstrate how to use the generated embeddings and Postgres pgvector to search for recipes stored in a database. Bring your questions to the chat and join the discussion.🚀 Register now!➡️https://hubs.la/Q02Kc6fY0 Join the YugabyteDB community!➡️ https://hubs.la/Q02Kc6pm0 #YugabyteDB #cloudnative #distributedsql #Postgresql #community #opensource #AI #pgvector

6

1 Comment