Tony Zhao, Ph.D

Can Synthetic Data Unlock the Future of Quantum Computing? Google’s Willow made headlines, but DeepMind’s recent innovation in quantum error correction might be just as groundbreaking—if not more. Researchers at DeepMind used machine learning to detect and fix errors in qubits, with their model, AlphaQubit, delivering remarkable results: • Outperforming other state-of-the-art methods. • Generalizing to longer distances and more complex error-correction cycles. • Offering potential for future improvements. What caught our attention? The use of synthetic data. Unlike the typical reliance on real-world data, AlphaQubit uses synthetic data for pretraining, generating large volumes to simulate noise patterns. Real-world data comes in only during fine-tuning for specific devices, requiring relatively few samples. The result? Synthetic data provides the bulk of performance gains. As noted in the study: “Even without fine-tuning, AlphaQubit achieves parity with traditional methods at code distance 5 after pretraining with synthetic data.” This demonstrates synthetic data’s role as a strong foundation for the next wave of AI and quantum computing models. Beyond performance, synthetic data delivers benefits like cost-efficiency, privacy, and streamlined licensing—essential for enterprises building safe and scalable systems. DeepMind’s breakthrough underscores why synthetic data is a powerful tool, not just for quantum computing, but for solving complex problems across industries. Link to paper in the comments:

19

3 Comments

Earlier this year we saw some really exciting research around Mamba, a selective SSM model architecture. So it's no surprise that we're starting to see some additional model architectures that promise to rival transformers in performance and efficiency. Researchers from Microsoft Research and Tsinghua University have developed YOCO (You Only Cache Once), a decoder-decoder architecture for large language models. This architecture appears to show some really exciting performance and efficiency gains over traditional transformers. Here are the top three takeaways: 1. Memory Efficiency: YOCO dramatically reduces GPU memory demands by caching key-value pairs just once. This means these models can handle long sequences without exhausting GPU memory, even managing up to 1M tokens efficiently! 2. Speed & Performance: The design significantly cuts down prefill latency and boosts throughput. YOCO slashes prefilling time by up to 30.3x and enhances throughput by 9.6x, demonstrating extremely fast model response times. 3. Scalability: YOCO excels at scaling up model sizes, training tokens, and context length. It performs competitively with traditional transformers across various metrics. It will be really exciting to see how new architectures like Mamba and YOCO can help in reducing training costs and improving performance. Read the full paper here: https://lnkd.in/g838mSPy #AI #MachineLearning #TechInnovation HOPPR

16

Missed the May 2nd AI, Machine Learning and Data Science Meetup? Check out the playbacks, abstracts, Q&A and resources links, all in one place - https://lnkd.in/gbF5ugi9 Talks included: * Who needs RLHF When You Have SFT? - Srishti Gureja at Georgia Institute of Technology & Writesonic * Develop a Legal Search Application from Scratch using The Milvus Project and DSPy! - Mert Bozkir - LLM Engineer * Making LLMs Safe & Reliable - Shiv Sakhuja at Athina AI (YC W23) #computervision #ai #artificialintelligence #machinevision #machinelearning #datascience #robotics

6

2 Comments

PyTorch leads the model training space, with 63% adoption rate 🚀 New research from The Linux Foundation, LF AI & Data Foundation & Cloud Native Computing Foundation (CNCF) shows how open source tools and frameworks play a critical role in #GenAI model building and inference. For more insights, download the research report on shaping the future of generative AI: https://lnkd.in/eGTGcFXT #OpenSource #AI #ML #PyTorch #GenAI #Inference #ModelBuilding

126

3 Comments

What's next for AI agentic workflows with Andrew Ng Did you know AI is evolving beyond just processing data?🤯 Andrew Ng, the renowned AI expert, shares his insights on the next frontier of AI: Agentic Workflows. Watch the full video here: #AI #ArtificialIntelligence #MachineLearning #AIWorkflows #AgenticAI #FutureofAI #AndrewNg #AIFund #Sequoia #Innovation #Technology https://lnkd.in/gJfudX_J

2

🎉 Score 2nd place in the VN1 challenge with a few lines of code in under 5 seconds using TimeGPT We present a fully reproducible experiment demonstrating that Nixtla's TimeGPT can achieve the 2nd position in the VN1 Forecasting Accuracy Challenge with zero-shot forecasting. Here’s how TimeGPT compares to the official competition scores: 🏆 1st place: 0.4637 ⚡ TimeGPT: 0.4651  🥈 2nd place: 0.4657 🥉 3rd place: 0.4758 No training. No exogenous variables. Just 5 seconds of inference. 🤯 How we did it 1️⃣ We processed the historical sales data 2️⃣ We leveraged TimeGPT’s zero-shot forecasting capabilities — no training, tuning, or adjustments required. 3️⃣ Inference time? Just 5 seconds to forecast for all series. This experiment highlights the potential of foundational models like TimeGPT to deliver top-tier forecasts with minimal effort. Unlike traditional approaches where most of the work goes into model design, training, and tuning, here the focus shifts to data preparation. And while the VN1 Challenge required open-source solutions, it was still a great opportunity to demonstrate TimeGPT's power as a closed-source API solution for time series forecasting. Want to reproduce it? We’ve made it easy to follow along. Check out the complete pipeline in our GitHub repo: link in comments. This isn't just a win for TimeGPT — it's a glimpse into how forecasting is evolving, from model-first to data-first workflows. We’re excited to see where this shift takes us. 🚀 Special thanks to Nicolas Vandeput #HappyForecasting #timeseries #forecasting #llms #genai #foundationmodels

177

11 Comments

Exciting update! Google has just released Gemma 2, their latest open AI models. It's all about top performance and efficiency, making advanced AI more accessible. Built for developers and researchers, Gemma 2 integrates seamlessly into your workflows with broad compatibility across major AI frameworks like Hugging Face Transformers, JAX, PyTorch, and TensorFlow. Ready to see what Gemma 2 can do for you? #AI #Gemma2 #TechNews

16

Hands-On with Andrew Ng's AI Suite! AI Suite: Simplifying Multi-Model Integration for Next-Gen AI Development I recently explored the AI Suite package by Andrew Ng and went a step further to create real-life demos showcasing how you can leverage this powerful tool for next-gen AI development. 𝑾𝒉𝒂𝒕’𝒔 𝒊𝒏𝒔𝒊𝒅𝒆: A practical look at integrating multi-model capabilities Real-world use cases you can adapt for your projects Insights into why AI Suite is a game-changer for developers 𝑾𝒉𝒚 𝒊𝒕 𝒎𝒂𝒕𝒕𝒆𝒓𝒔: Multi-model integration is key to unlocking the true potential of AI in solving complex, real-world problems. These demos aim to inspire and guide developers to make the most of this cutting-edge package. 𝑳𝒆𝒕’𝒔 𝑪𝒐𝒏𝒏𝒆𝒄𝒕: Have you used AI Suite or similar tools? I’d love to hear about your experiences, challenges, or ideas. Drop a comment or send me a message! #AI #AndrewNg #AIDevelopment #MultiModelIntegration #TechInnovation https://lnkd.in/ex8w5qXJ

2

We’ve recently contributed FP8 support to vLLM in collaboration with Neural Magic -- with this feature, you can see up to a 1.8x reduction in inter-token latency, with >99% accuracy preservation! A common concern with FP8 is whether users will experience accuracy degradation. To address this, Neural Magic has produced many checkpoints for key models with >99% accuracy preservation across a wide range of benchmarks (https://lnkd.in/gTimN5dZ), including: - Llama3-70b - Mixtral 8x7b - Llama3-8b You can easily try this out on vLLM, and read more about the feature here -- https://lnkd.in/gzKJqerB

146

3 Comments

FP8 quantization is now available in vLLM - check it out! Quantized inference is one of the best ways to reduce the costs of LLM deployments.

5

👩‍💻 Calling all AI developers interested in agentic workflows. We’re excited to announce that our new course, ‘Build Long Context AI Apps with Jamba’, built in partnership with Andrew Ng's DeepLearning.AI, is now live - and it’s currently free! Why join? Long contexts are critical for agentic workflows. But, there’s a challenge: 🔹 Most LLMs today are built using a #Transformer architecture, which is powerful, but computationally expensive for long contexts. 🔹The alternative, state space models like #Mamba, offer cost efficiency, but lack the accuracy needed for enterprise AI use cases. Our solution: #Jamba, AI21’s cutting-edge hybrid model with the #1 longest effective context window on the market. By combining the best of both Transformers and Mamba, Jamba delivers both efficiency and accuracy, making it the ideal choice for agentic workflows where long-context processing is mission critical. In this course, Chen Wang and Chen Almagor from AI21 Labs will walk you through how to use Jamba for: ✨ Long-context prompting ⚙️ Tool calling 📄 Analyzing long documents 🛠️ Conversational RAG Plus, you’ll dive deep into the Jamba architecture and its unique design. Upskill and join a global developer community in building AI apps with Jamba. Questions? Drop them below. 👉 Start learning today: https://lnkd.in/e95KGpu2 #DeeplearningAI #AICourse #AgenticAI #DevCommunity

307

24 Comments

🧠 What’s a Semantic Cache in RAG? Here’s What We’ve Learned Semantic cache is a time and cost saving method for Retrieval Augmented Generation (RAG). Instead of always running a full vector search, a semantic cache stores embeddings of frequently asked queries and their responses. This speeds up your RAG system by skipping redundant searches for similar queries. How do you implement it? Step 1: Store history of answered questions, their embeddings and the LLM response in a semantic cache table. Step 2: When a new query comes in, compare it against the cache table using vector similarity. Step 3: If there’s a match with score over a certain threshold, serve the cached response. If not, run a fresh retrieval and add it to the cache. Now, let’s talk vector search methods: 1️⃣ Cosine Similarity: Measures angle between vectors, works well for normalized data. 2️⃣ Euclidean Distance: Measures straight-line distance but can be skewed by vector magnitude. 3️⃣ Dot Product: Our favorite—it factors in vector size and performs best for our use cases. Want to see how semantic caching powers a RAG system in real life? Check out our demo at https://lnkd.in/g2jFuHTT. 🚀

2

Managing the limitations of LLMs 🤔  https://lnkd.in/gtzFapVd The retrieval augmentation and internal reasoning of large language models has certain limitations when it comes to topical inference, particularly when trying to bring together relationships across very disparate datasets, according to Persistent Systems CTO Pandurang Kamat. “You have to augment it. We typically augment with vector databases by vectoring information. But what graph datasets and knowledge graphs bring to the table is they allow us to pull in data from disparate sources within the enterprise and build out those relationships so you can pull a deeper context to give to the LLM,” Kamat tells #theCUBE. Get the full story here: https://lnkd.in/gtzFapVd

2