Feng Shi

Torch Compile is here to optimize inference in large language models (LLMs) with minimal overhead. In Benjamin Marie's latest article, explore how adding just one line of code can improve decoding performance. #Llama LLM

14

Mixture-of-Agents as an Event-Driven System 🤖☎️ This paper by Wang et al. shows you how to ensemble smaller LLMs to create a system that can outperform state-of-the-art larger models. We’ve implemented this paper in a fully async, event-driven workflow system thanks to Ravi Theja Desetty - treat each “small LLM” as an event-driven step that can process incoming events and respond to it, independently and in parallel. ✅ Take full advantage of processing an entire batch of requests  ✅ Cleaner, readable code LlamaPack: https://lnkd.in/gRuqQGk2 Learn more about workflows: https://lnkd.in/gqWwz7Xs

311

9 Comments

Aligning Large Language Models with Diverse User Preferences Using Multifaceted System Messages: The JANUS Approach Quick read: https://lnkd.in/gQAsT95M Paper: https://lnkd.in/gkBAuWiC

19

The parameters of compiler optimizations interact in complex and non-trivial ways, and finding optimal combinations remains an open research challenge. There are several approaches to discovering effective combinations of these optimization parameters. Recently, Michael Canesche (Cadence Design Systems) and Gaurav Verma (Stony Brook University) have had promising results using auto-tuning with coordinate descent. To visualize this process, imagine that each optimization represents a 'dimension' in a search space. The combination of all optimizations and their respective parameters forms a vector in this space. Now, consider that each point in this space is mapped to a real number representing the performance of a kernel (assuming a fixed input). Coordinate descent is a search technique that assumes the performance curve is convex. Based on this assumption, it aims to find the optimal point by varying one optimization parameter at a time. Explore as a Storm, Exploit as a Raindrop: On the Benefit of Fine-Tuning Kernel Schedulers with Coordinate Descent: https://lnkd.in/dgxxfdYR The Droplet Search Algorithm for Kernel Scheduling: https://lnkd.in/dBfQid9z

77

Optimization of Language Models for Efficient Inference and Performance Using Mixed Architectures via #TowardsAI → https://bit.ly/4dMGxfp

9

Optimization of Language Models for Efficient Inference and Performance Using Mixed Architectures via #TowardsAI → https://bit.ly/4dMGxfp

2

Optimization of Language Models for Efficient Inference and Performance Using Mixed Architectures via #TowardsAI → https://bit.ly/4dMGxfp

4

Optimization of Language Models for Efficient Inference and Performance Using Mixed Architectures via #TowardsAI → https://bit.ly/4dMGxfp

4

Let's evaluate Llama3's RAG performance on an arXiv paper (about RAG evaluation, this is a very meta notebook, puns intended)! In this tutorial, we build on our work earlier this week in combining Unstructured's API for pdf document preprocessing with OpenAI's GPT-4o + ragas for evaluation, Hugging Face for Llama3, and LangChain to integrate all of these systems to evaluate Llama3. Llama3's metrics on RAG for our quick example: context precision: 0.9867, faithfulness: 0.8297, answer relevancy: 0.8643, context_recall: 0.9733 Try this out with your favorite unstructured data by swapping the arXiv pdf URL in the notebook! Colab Notebook: https://lnkd.in/gSndzTuC

26

1 Comment

Easily generate one billion embeddings in under a day... and with limited budget! How? With Ray Data from Anyscale and Pinecone! Cheng Su and Roy Miara present this can't-miss session during the #DataAISummit tomorrow. Catch it in-person or online.

43

2 Comments

Read our latest #OpenAccess research! 💡 Authored by Amin Jafarimoghaddam, the paper "The Indirect Bang-Singular Algorithm (IBSA) for Singular Optimal Control Problems with State-Inequality Constraints," presents a novel algorithm aimed at addressing complex singular #OCPs. The IBSA offers a general systematic and symbolic computational framework, with capabilities that make this algorithm particularly suitable for solving advanced problems such as those encountered in commercial #aircraft #TrajectoryOptimization 🛩️ . 🔗 Find the link to the journal paper on our website: https://bit.ly/48krnwr #Aviation #Algorithm #Science Aerospace Engineering UC3M

4

🚀 Looking to streamline fine-tuning for models like Llama 3, Mistral, and Mixtral? Our latest blog shows how Anyscale simplifies LLMs fine-tuning with Ray: 🔑 Key takeaways: • Scalable fine-tuning with LLM-Forge • Model Deployment with RayLLM Read more: https://lnkd.in/gQwD4A2U

34

🚀 Check out the latest Medium post by our student Tianyunxi Yin: "Quantization Fundamentals for Model Compression". Explore the essentials of quantization and learn how it helps in compressing machine learning models for better efficiency without compromising performance. Read more here: https://lnkd.in/gHbbR_J4 #USFMDS #DataScience #MachineLearning #ModelCompression #Quantization

15

𝗣𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿-𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴 (𝗣𝗘𝗙𝗧) PEFT is an optimization technique for fine-tuning large language models (LLMs) by only modifying a small subset of parameters, instead of adjusting the entire model. This significantly reduces computational resource requirements while maintaining high performance. Popular PEFT techniques include 𝗟𝗼𝗥𝗔 (𝗟𝗼𝘄-𝗥𝗮𝗻𝗸 𝗔𝗱𝗮𝗽𝘁𝗮𝘁𝗶𝗼𝗻), 𝗣𝗿𝗲𝗳𝗶𝘅 𝗧𝘂𝗻𝗶𝗻𝗴, 𝗮𝗻𝗱 𝗣-𝗧𝘂𝗻𝗶𝗻𝗴. Common PEFT techniques: 1 .𝗟𝗼𝗥𝗔 (𝗟𝗼𝘄-𝗥𝗮𝗻𝗸 𝗔𝗱𝗮𝗽𝘁𝗮𝘁𝗶𝗼𝗻): LoRA works by "freezing" most of the original model’s parameters and only training a pair of low-rank matrices. This reduces the number of trainable parameters from tens of thousands to just a few thousand, saving resources while maintaining good performance on specific tasks like text classification and language translation. 𝟮. 𝗣𝗿𝗲𝗳𝗶𝘅 𝗧𝘂𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗣-𝗧𝘂𝗻𝗶𝗻𝗴: These techniques add parameters into different layers of the model or within embeddings. They are optimized for tasks such as Natural Language Understanding (NLU) without the need to fine-tune the entire neural network. https://lnkd.in/gTM7r_tf #PEFT #FineTuning #LLMs

1

In this advanced use case, #VeniceDB is used to merge batch and stream inputs, and to transport the data to serving nodes which load it into GPUs, thus updating their ML model in near real time. Check out the paper for more details!

16

1 Comment

From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, Huaming Chen University of Sydney 2024 https://lnkd.in/gVZ3r8G7 "As LLMs have demonstrated significant capabilities in tasks such as code generation and vulnerability detection, researchers are increasingly interested in their potential applications. However, the study highlights the limitations of LLMs, including their lack of autonomy and self-improvement, which LLM-based agents aim to address. These agents represent a promising advancement towards achieving Artificial General Intelligence (AGI) by integrating LLMs for decision-making and action-taking. Despite the growing body of research on LLMs in software engineering, there remains a lack of clarity distinguishing between LLMs and LLM-based agents, as well as a need for standardized benchmarks to evaluate these technologies. The survey systematically investigates current practices and solutions related to both LLMs and LLM-based agents, focusing on six key areas: requirement engineering, code generation, autonomous decision-making, software design, test generation, and software maintenance. By reviewing and contrasting the contributions of LLMs and LLM-based agents across these topics, the survey provides insights into their respective tasks, benchmarks, and evaluation metrics. It also discusses the models and benchmarks utilized in this domain, offering a comprehensive analysis of their effectiveness and applications in software engineering. Ultimately, the authors aim to illuminate the path forward for LLM-based agents in software engineering, paving the way for future research and development in this promising area."

2

🌐 Exciting Announcement! We are thrilled to host Fangru Lin for a discussion on her groundbreaking work, "Graph-enhanced Large Language Models in Asynchronous Plan Reasoning," recently presented at ICML 2024! 🎉🧠 Join us on August 24, 2024, at 10:00 AM (EST) as Fangru Lin walks us through the innovative "Plan Like a Graph (PLaG)" approach. This method revolutionizes LLM performance by breaking down tasks into sub-tasks, forming an execution graph that is either solved in parallel or sequentially by the LLM. This structured approach not only enhances results but also opens new pathways for future research in prompt engineering. 📜 Paper Abstract: Planning is a fundamental property of human intelligence. Reasoning about asynchronous plans is challenging since it requires sequential and parallel planning to optimize time costs. Can large language models (LLMs) succeed at this task? Here, we present the first large-scale study investigating this question. We find that a representative set of closed and open-source LLMs, including GPT-4 and LLaMA-2, behave poorly when not supplied with illustrations about the task-solving process in our benchmark AsyncHow. We propose a novel technique called Plan Like a Graph (PLaG) that combines graphs with natural language prompts and achieves state-of-the-art results. We show that although PLaG can boost model performance, LLMs still suffer from drastic degradation when task complexity increases, highlighting the limits of utilizing LLMs for simulating digital devices. Our study is an exciting step towards using LLMs as efficient autonomous agents. Our code and data are available at https://lnkd.in/e49YXamU. Check out the full paper: https://lnkd.in/eCdsdhE4. 📅 Save the Date: 🗓️ August 24, 2024 🕙 10:00 AM (EST) 📍 Location: 🌐 Virtual Event Join via Zoom: https://lnkd.in/eSq8PKNe 🚀 Don't miss this opportunity to gain insights into the future of LLMs and their application in complex tasks! 🌟 #ICML2024 #LLMs #MachineLearning #GraphLearning #AIResearch Looking forward to your participation! Spread the word and let's dive into the future of LLMs together! 🚀

13