Xiangyi Li

🚀 Looking to streamline fine-tuning for models like Llama 3, Mistral, and Mixtral? Our latest blog shows how Anyscale simplifies LLMs fine-tuning with Ray: 🔑 Key takeaways: • Scalable fine-tuning with LLM-Forge • Model Deployment with RayLLM Read more: https://lnkd.in/gQwD4A2U

35

In the lead-up to #Raysummit, we have a guest blog on Building a RAG Batch Inference Pipeline with Anyscale and Union.ai 🚀 #Flyte is an open-source orchestrator facilitating production-grade data & ML pipelines. When combined with Ray’s distributed computing power, it unites data, platform, infrastructure, and ML engineers to boost productivity and scale across evolving use cases. Union.ai platform makes orchestrating these pipelines seamless. In the RAG Batch Inference pipeline exactly there are actually 2 pipelines that Union orchestrates: 🔹 Embedding Generation Pipeline 🔹 Batch Inference Pipeline #Flyte Deck on Union.ai lets you preview responses without downloading, ensuring instant validation. Anyscale’s Ray platform optimizes the execution of these pipelines, helping deliver leading performance and cost efficiency. Read more here 👉 https://lnkd.in/gjDKRPmu

47

Groq is leading the pack! - Up to 18x faster speeds compared to other inference API providers for open source LLMs. - Easy integration with frameworks like LangChain and LlamaIndex - Flexible pricing, as low as $0.04 per million tokens Read more from Aayush Mittal here: https://hubs.la/Q02_CyhF0

25

4 Comments

Want to learn how to work smarter and get more done with AI this holiday season? 🎄✨ Here’s a tool that’s been a game-changer for us—making things easier and faster! Groq #AI #Productivity

1

Check out our latest blog on Meta's Llama 3.1 available on Friendli Suite! Llama 3.1 8B and 70B can now be utilized for inference on Friendli Serverless Endpoints, with Llama 3.1 70B delivering impressive performance at well over 100 tokens per second. Also, Friendli Dedicated Endpoints support fine-tuning and inference for Llama 3.1 models. Read the full blog here 👉 https://lnkd.in/devRe9xj Experience the blazing-fast generation with Llama 3.1 70B on Friendli Serverless Endpoints 👉 https://suite.friendli.ai See Llama 3.1 405B in action 👉 https://lnkd.in/dm8DafjV #FriendliAI #Friendli #Llama31 #LLM #FineTuning #Inference

15

"Females can act like glue within a team, contributing to the overall success the same as men, while injecting a different perspective." - Jian Cheng, Senior Full Stack Developer at Edge Signal. We had the pleasure of speaking with Jian and other female leaders within our ecosystem in celebration of #IWD2024. With 15+ years of software programming experience, Jian is in a unique situation – being a senior developer, female, Chinese, an immigrant, and currently the only female on a small start-up team that is tasked to build an AI-powered edge computing platform, Edge Signal. Read the article to learn more: https://lnkd.in/eUyBJrVg. #tech #femalesintech #womenintech #edgesignal

31

Discover why vector databases are transforming search. By unifying Sparse-BM25 and semantic search, these purpose-built systems outperform traditional setups, revolutionizing applications from enterprise search to multimodal AI. Explore this shift in Jiang Chen’s article. #AI #Databases

1

We’ve recently contributed FP8 support to vLLM in collaboration with Neural Magic -- with this feature, you can see up to a 1.8x reduction in inter-token latency, with >99% accuracy preservation! A common concern with FP8 is whether users will experience accuracy degradation. To address this, Neural Magic has produced many checkpoints for key models with >99% accuracy preservation across a wide range of benchmarks (https://lnkd.in/gTimN5dZ), including: - Llama3-70b - Mixtral 8x7b - Llama3-8b You can easily try this out on vLLM, and read more about the feature here -- https://lnkd.in/gzKJqerB

146

3 Comments

FP8 quantization is now available in vLLM - check it out! Quantized inference is one of the best ways to reduce the costs of LLM deployments.

5

Check out our recent blog post collaboration: Building a RAG Batch Inference Pipeline with Anyscale and Union.ai! If you have any questions find some of us this week at #RaySummit! 🔗 https://lnkd.in/gjDKRPmu

7

Want to learn how to fine-tune open source LLMs that rival closed source providers? At Observe this year, Arnav Garg (Predibase) will show you how to do just that. 🚀 Join us in San Francisco on July 11 for this and other hands-on sessions: https://lnkd.in/gpEitQb5 More about this session: Most teams start with a commercial LLM like GPT-4 and the appeal is pretty clear: users gain instant access to a large language model with trillions of parameters that shows great performance out of the box and beats almost all open source alternatives. However, the story changes once open-source LLMs are fine-tuned, where they not only consistently beat GPT-4 by 4-15 points on downstream tasks, but do it while being 250x smaller and with a 100x cheaper inference cost. Learn how to efficiently fine-tune and serve OSS LLMs using open source packages like Ludwig and LoRAX with a focus on core fine-tuning and inference ideas that enable faster, cheaper, smaller and better models compared to closed source providers. Don’t miss this year’s premier event for building AI-powered applications and improving quality and performance once in production. Register here: https://lnkd.in/gpEitQb5

14

Label Studio Customized Backend for Semiautomatic Image Segmentation Labeling - Customized backend; GCP Deployment; Data Versioning with GCS Integration by Yuhan Yao

3

Easily generate one billion embeddings in under a day... and with limited budget! How? With Ray Data from Anyscale and Pinecone! Cheng Su and Roy Miara present this can't-miss session during the #DataAISummit tomorrow. Catch it in-person or online.

44

2 Comments

Conquering prompt-based model with DSPy 🌟 Jibby Nguyen Check this out: https://lnkd.in/gTgiGsSJ In this seminar, we will discuss a framework called DSPy developed for optimizing language model (LM) prompts and weights within more complicated pipelines. With DSPy, we can no longer function without breaking down problems into steps, writing prompts for each step, and then adjusting everything by hand to accommodate changes. We also get our hand dirty by handing on real code to see how to use it. About us: 📚 We share knowledge about AI through detailed seminars, covering various domains such as Reinforcement Learning, LLM, MLLM, Vector Search, Computer Vision, Web Crawling, and more. 🌍Our goal is to foster an educational community eager to learn and apply AI technologies across different sectors. 🌟 We definately can make mistake as it's human-being so we are really appreciated reading your feedbacks, feel free to express your opinion!!

1

🚨 New Session Alert 🚨 Tianyu Liu, Research Scientist Meta #PyTorch team, will take you through Torch Titan, the production-ready PyTorch native solution powering the Llama 3.1 family of LLMs. Register now to secure your spot: https://buff.ly/3NHGt5I #meetup #ai #machinelearning #deeplearning #llm #gpu #modeltraining #cloud #compute #storage #finetuning

3

LangChain Vs LlamaIndex: A Detailed Comparison LangChain and LlamaIndex are advanced frameworks designed to enhance the capabilities of large language models (LLMs). LangChain focuses on building complex workflows and interactive applications, while LlamaIndex emphasizes seamless data integration and dynamic data management. This article provides a comprehensive comparison between these two frameworks, exploring their unique features, tools, and ecosystems. Detailed sections cover LangChain’s definition, core features, tools, and ecosystem, followed by a similar examination of LlamaIndex. Additionally, a dedicated section compares the code implementations of both frameworks, highlighting their differences in approach and functionality. Finally, the article summarizes the main distinctions between LangChain and LlamaIndex, offering insights into their respective strengths and suitable use cases, and guiding developers and data scientists in selecting the right framework for their specific needs. ➡ Do not miss the 60% discount on To Data & Beyond yearly subscription: https://lnkd.in/gmwFUqYr

3

👋🏻 We are happy to welcome Eliran Eliassy for the 3rd time at NG-DE! 🤔 You don't know him, yet? 🤗 No problem. 📼 Here, you can find his deep dive about #angular's iterable differs:

2

1 Comment