Linde C.

Foundational models, or large language models (LLMs), train at the scale of the entire internet to acquire the vocabulary, syntax, and narrative structure of human language. Fine-tuning involves using a smaller, domain-specific dataset to grant additional specificity to a pre-existing foundational model that has already acquired patterns and features from massive underlying training datasets. By utilizing the existing knowledge embedded in a pre-trained language model, we unlock high-performance language understanding, but fine-tuning helps to get the right answer - not just an answer that sounds right - with a minimum of hallucinations or incorrect info. In healthcare, out-of-the-box, foundational models are often too generic to sufficiently answer the precision of a healthcare query. Foundational language models are trained on a massive scale of data, which includes jokes, incorrect information, misunderstandings, and misinformation, along with the sum total of human knowledge. The problem is, if you ask a large language model to give you factually correct information when it has been trained on BOTH accurate and inaccurate information - you’re playing a bit of a dangerous game. Enter fine-tuning language models. In the old days, we used to build a single model for a single task - for example, classifying procedures into evidence-based or not as laid out by clinical practice guidelines, for patients with particular disease markers. Each disease would have its own model, which took a long time to build and didn’t scale particularly well, and we struggled with the sparsity and inherent high-dimensionality of healthcare data. By fine-tuning LLMs, however, we're able to specialize those foundational models to achieve greater domain specificity and specialization, leverage the unique predictive and planning capabilities of LLMs, and cut down on the incorrect information that such models inevitably produce, while still preserving the underlying language prowess of the LLM. If improved accuracy and specificity wasn't enough of a selling point, there's another notable upside to fine-tuning foundational models for use in a clinical context. In terms of computational efficiency, using LLMs out of the box on specific tasks can require massive compute (and spend) that amounts to computational overkill for the problem. Some LLMs charge per query, some by compute load or time - either way, as a business running LLMs in a production environment, minimizing the number of LLM calls means optimizing your compute expenditures, too. When LLMs fail at scale in healthcare, it's usually along the fault line of failure to specify adequately. Fine-tuning large language models along domain-specific lines is one way to effectively productize genAI for healthcare use cases. #genAI #LLM #ai #healthcareAI #healthtech #largelanguagemodels #generativeAI #chatGPT

61

2 Comments

- SwiftKV is a novel model transformation and distillation procedure (by folks at Snowflake) designed to optimize large language model (LLM) inference. Very cool findings in the paper (as presented in the vLLM meetup on 11/13). - SingleInputKV: Prefills later layers' KV cache using earlier layer outputs, reducing computation on prompt tokens. The key point is that the first few layers of attention in LLMs are similar (nice find), allowing SwiftKV's SingleInputKV mechanism to skip computations in later layers during inference. By using a single earlier layer's output to compute the KV cache for subsequent layers, it significantly reduces both computation (by 50%) and memory usage (by 62.5%) without sacrificing model performance. - AcrossKV: Merges KV caches of neighboring layers to minimize memory usage and increase batch size for higher throughput. - For models like Llama-3.1-8B and 70B, SwiftKV reduces prefill compute by 50%, memory usage by 62.5%, and achieves up to 2× higher throughput and 60% lower time per token. https://lnkd.in/guhEBW5G

62

SCRUB the video to appreciate the many examples, I didn’t include only Einstein. 👇 The latest research from #ByteDance, called #Loopy, works exceptionally well at synthesizing #emotions for #autonomous #animation. I have included multiple examples in the clip below for you to see. I think the length of the piece with Einstein is an accomplishment in temporal consistency worthy of review, because this isn’t a 4s-10s bit, it’s a lengthy and complex emotional monologue. The later bits are different styles, such as animation in 2D and 3D styles. There are tons more examples on the project page, it honestly boggles the mind. NO, you cannot get access to the code, but it's evident they're crushing it, and if you think they're just doing this for #TikTok videos or #videogames, I have a bridge I can sell you, too. If you're interested in an #American based company that is capable of doing this, and holds a family of patents to boot (it's arguable that this work from ByteDance infringes on theirs, but that's for a court to decide), then you should check out #MetaSoul: https://metasoul.one/ Project page: https://lnkd.in/grKhnnit Research Paper: https://lnkd.in/gWU8Gqjv _ #genai #emotionsynthesis #avatar #avatars

8

ModernBERT is recently released by (like BERT, it is also an encoder-only model) by Answer.AI and LightOn, building upon BERT. ModernBERT incorporates several advancements from recent LLM research: - Modernized Architecture: It adopts elements from Transformer++ (as used in Llama 2), including rotary positional embeddings (RoPE) for improved handling of long sequences, GeGLU activation functions in the MLP layers, removal of unnecessary bias terms, and an additional normalization layer after embeddings. - Efficiency Optimizations: ModernBERT leverages Flash Attention 2 for speed improvements. It also employs "Alternating Attention" (global attention every 3 layers, local attention with a sliding window in between) to reduce computational complexity for long sequences. "Unpadding" and "Sequence Packing" are used to minimize wasted computation on padding tokens, maximizing hardware utilization. - Enhanced Training Data: ModernBERT was trained on 2 trillion tokens from a diverse set of English sources, including web documents, code, and scientific articles, with minimal repetition of training examples. This diverse training data particularly enhances performance on code-related tasks. - Training Process: The training uses a three-phase approach: initial training at a sequence length of 1024, a long-context adaptation phase at 8192, and final annealing. The stable checkpoints from the first two phases are publicly released to facilitate further research and fine-tuning. Training optimizations include batch size warmup and weight initialization via tiling (for the larger model), leveraging pre-trained weights from the base model. ModernBERT achieves state-of-the-art performance across various benchmarks (GLUE, ColBERT for retrieval, and StackOverflow-QA for code). https://lnkd.in/gJzgxxvW

17

If only you could have fixed hallucinations of a pre-trained LLM through contextually relevant prompts. It would have been a solved problem by now! Contextual relevance (RAGs) improve domain specificity, not hallucination! Nice article. #llms #chatgpt4 #aiml #mlops #llmops #ml #rag https://lnkd.in/d44hdifr

11

LLMs have become more capable with better training and data. But they haven’t figured out how to “think” through problems at test-time. The latest research from Scale finds that simply scaling inference compute–meaning, giving models more time or attempts to solve a problem–is not effective because the attempts are not diverse enough from each other. 👉 Enter PlanSearch, a novel method for code generation that searches over high-level "plans" in natural language to encourage response diversity. PlanSearch enables the model to “think” through various strategies before generating code, making it more likely to solve the problem correctly. The Scale team tested PlanSearch on major coding benchmarks (HumanEval+, MBPP+, and LiveCodeBench) and found it consistently outperforms baselines, particularly in extended search scenarios. Overall performance improves by over 16% on LiveCodeBench from 60.6% to 77%. Here’s how it works: ✅ PlanSearch first generates high-level strategies, or "plans," in natural language before proceeding to code generation. ✅ These plans are then further broken down into structured observations and solution sketches, allowing for a wider exploration of possible solutions. This increases diversity, reducing the chance of the model recycling similar ideas. ✅ These plans are then combined before settling on the final idea and implementing the solution in code. Enabling LLMs to reason more deeply at inference time via search is one of the most exciting directions in AI right now. When PlanSearch is paired with filtering techniques—such as submitting only solutions that pass initial tests—we can get better results overall and achieve the top score of 77% with only 10 submission attempts. Big thanks to all collaborators on this paper including: Evan Wang, Hugh Zhang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen H., Sean Hendryx, Summer Yue 👉 Read the full paper here: arxiv.org/abs/2409.03733

180

2 Comments

Models like OpenAI's CLIP combine text and images for search, but they have limitations, particularly with longer texts and complex relationships. This article talks about JINA's new model that improves on CLIP by integrating both text and image embeddings in one model, making it faster and more efficient for multimodal search tasks. Experiments and Findings: → Search with Both Text and Images: Combining both embeddings leads to better and more diverse search results, improving relevance and variety in results. → Image-Driven Diversification: By using images for diversification, Jina-CLIP v1 can present a broader range of results, offering users more varied options. Results Comparison: → Using both text and image embeddings (via averaging) consistently outperformed searches based solely on text or images, providing more precise and diverse retrieval across both broad and fine-grained categories. https://lnkd.in/guNRQpBU

15

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

28

Quentin H. and Yanwei (Wayne) Zhang on the Faire Data team recently did some great work fine-tuning Llama3 model to enhance search relevance. Here are some key insights from the work: • While prompt engineering is valuable, LLM fine-tuning is essential for a domain specific task like understanding semantic search relevance. • Fine-tuned open-source LLMs like Llama3 can deliver impressive performance, scalability, and cost-efficiency compared to using proprietary models. • Effective application of a fine-tuned LLM demands a well-defined problem, robust, high-quality labeled data, and ongoing model refinement. Ground-truth labeled data is crucial for maximizing model performance. • Fine-tuned Llama3-8b achieves performance on par with the larger Llama2–13b model. Read the blog post to dive deeper into these findings! https://lnkd.in/ecP_3E7S We are also hiring (https://lnkd.in/dHfRZcPi) #LLM #AI #Finetuning #LLAMA3 #SearchRelevance #Hiring

92

3 Comments

As a parent, it is bittersweet to watch your children grow up and gain independence. This is what is happening now with my second pandemic baby, PyTorch Forecasting — a #Python package featuring state-of-the-art #AI models for forecasting problems. I'm thrilled to announce that sktime will be taking over and driving further the development of PyTorch Forecasting. It's rare for an #OpenSource project to be adopted by an organization, and this milestone is a testament to the quality and relevance of the package and the strength of the community. I started #PyTorchForecasting in 2020, and it quickly gained traction, reaching nearly 4k stars on #GitHub. It has become one of the most popular packages for #TimeSeries forecasting, with more than 1M downloads so far. It powers use cases from #FinancialForecasting to #RetailPricing, to #Inventory, and even #FoodDelivery optimization. PyTorch Forecasting makes the very complex world of advanced #DeepLearning-based time-series modelling accessible to many. This success proves that user experience and convenience are paramount. The team at sktime, particularly Franz Király, have done a stellar job since I began collaborating with them. The project is in great hands, and I'm excited to see where they take it next. Looking forward to the future and the continued success of PyTorch Forecasting which is now housed here: https://lnkd.in/e_Qr5kMR #DataScience #MachineLearning #PyTorch

244

11 Comments

🔮RAG + KG: Powerful Combination for Translation Multi-AI Agents Large language models (LLMs) have shown great promise in machine translation, but they still struggle with contextually dependent terms, such as new or domain-specific words. This leads to inconsistencies and errors that are difficult to address. Existing solutions often depend on manual identification of such terms, which is impractical given the complexity and evolving nature of language. While RetrievalAugmented Generation (RAG) could provide some assistance, its application to translation is limited by issues such as hallucinations from information overload. =================== In this paper, researchers propose CRAT, a novel multi-agent translation framework that: 》 leverages RAG and causality enhanced self-reflection to address these challenges. This framework consists of several specialized agents: ✱ the Unknown Terms Identification agent detects unknown terms within the context, ✱ the Knowledge Graph (KG) Constructor agent extracts relevant internal knowledge about these terms and retrieves bilingual information from external sources, ✱ the Causality enhanced Judge agent validates the accuracy of the information, ✱ Translator agent incorporates the refined information into the final output. =================== This automated process allows for more precise and consistent handling of key terms during translation. Their results show that CRAT significantly improves translation accuracy, particularly in handling context-sensitive terms and emerging vocabulary. --- If your company is looking for an AI consultant to accelerate business growth, or if you are looking to build your career in AI, send me a message and I can help

4

Spatial Reasoning did not ‘emerge’ spontaneously in Large Language Models (LLMs) the way so many reasoning capabilities did. Humans have specialized, highly capable spatial reasoning capabilities that LLMs have not replicated. But every subsequent release of the major models — GPT, Claude, Gemini- promises better multimedia support, and all will accept and try to use uploaded graphics along with texts. #llms #chatgpt4 #gemini #llmops #aiml #ml https://lnkd.in/dFSWKqKs

2

$2 H100s: How the GPU Bubble Burst Last year, H100s were $8/hr if you could get them. Today, there's 7 different resale markets selling them under $2. What happened? The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and decline in new foundation model co’s. Rent instead. https://lnkd.in/gpQEidmC

2

1 Comment

🎉 Check out my new Medium article! 🎉 In this post, I dive into how to use BigQuery ML and Vertex AI’s Gemini model to analyze bird feeder images with just SQL and BigQuery. 🐦💻 ✨ Build a remote BigQuery ML model with Vertex AI 🔍 Use SQL to identify bird species in images stored in Google Cloud Storage 🤖 Automate species identification with AI-powered insights Whether you're into AI, machine learning, or simply love birds, this guide is for you! https://lnkd.in/gU5AeknV #BigQueryML #VertexAI #AI #MachineLearning #Birdwatching #GoogleCloud #DataEngineering

3

Could RAFT (Retrieval-Augmented Fine-Tuning) be the next evolution of RAG architecture? I am following this with great interest. https://lnkd.in/eHDx3rHz

13

🌟 Simplifying Predictions with Elastic Net Regression Struggling with too many variables or correlated data? Elastic Net Regression combines the strengths of Ridge and Lasso Regression to handle complex datasets effectively. Why Elastic Net Works: Balances Penalties: Uses a mix of Ridge (L2) and Lasso (L1) penalties to shrink coefficients and simplify models. Handles Multicollinearity: Keeps variables with overlapping effects in check. Feature Selection: Eliminates irrelevant variables by shrinking their impact to zero. Hyperparameter Tuning: Adjust Alpha for regularization strength and Lambda to balance Ridge and Lasso penalties. Perfect For: Retail: Predicting sales with correlated variables like promotions and ads. Healthcare: Analyzing patient data with interdependent factors. Finance: Modeling stock trends while filtering noise. 🔗 Read more: https://lnkd.in/gAP3SpHF

4

2 Comments