Ayush Parolia

Check out this highly recommended course for those interested in optimizing their models: - Introduction to Quantization - Symmetric and Asymmetric Quantization Modes - Hands-on Implementation from Scratch using PyTorch - covering Per Tensor, Per Channel (Axis), and Per Group techniques - Weights Packing: Exploring 2 or 4 Bits Quantization Methods https://lnkd.in/gMAuWXHm

43

Foundational models, or large language models (LLMs), train at the scale of the entire internet to acquire the vocabulary, syntax, and narrative structure of human language. Fine-tuning involves using a smaller, domain-specific dataset to grant additional specificity to a pre-existing foundational model that has already acquired patterns and features from massive underlying training datasets. By utilizing the existing knowledge embedded in a pre-trained language model, we unlock high-performance language understanding, but fine-tuning helps to get the right answer - not just an answer that sounds right - with a minimum of hallucinations or incorrect info. In healthcare, out-of-the-box, foundational models are often too generic to sufficiently answer the precision of a healthcare query. Foundational language models are trained on a massive scale of data, which includes jokes, incorrect information, misunderstandings, and misinformation, along with the sum total of human knowledge. The problem is, if you ask a large language model to give you factually correct information when it has been trained on BOTH accurate and inaccurate information - you’re playing a bit of a dangerous game. Enter fine-tuning language models. In the old days, we used to build a single model for a single task - for example, classifying procedures into evidence-based or not as laid out by clinical practice guidelines, for patients with particular disease markers. Each disease would have its own model, which took a long time to build and didn’t scale particularly well, and we struggled with the sparsity and inherent high-dimensionality of healthcare data. By fine-tuning LLMs, however, we're able to specialize those foundational models to achieve greater domain specificity and specialization, leverage the unique predictive and planning capabilities of LLMs, and cut down on the incorrect information that such models inevitably produce, while still preserving the underlying language prowess of the LLM. If improved accuracy and specificity wasn't enough of a selling point, there's another notable upside to fine-tuning foundational models for use in a clinical context. In terms of computational efficiency, using LLMs out of the box on specific tasks can require massive compute (and spend) that amounts to computational overkill for the problem. Some LLMs charge per query, some by compute load or time - either way, as a business running LLMs in a production environment, minimizing the number of LLM calls means optimizing your compute expenditures, too. When LLMs fail at scale in healthcare, it's usually along the fault line of failure to specify adequately. Fine-tuning large language models along domain-specific lines is one way to effectively productize genAI for healthcare use cases. #genAI #LLM #ai #healthcareAI #healthtech #largelanguagemodels #generativeAI #chatGPT

61

2 Comments

When it comes to "Retrieval" phase of RAG, choosing the right strategy makes all the difference. Sparse retrieval (like BM25/TF-IDF) is great for finding exact keyword matches, while Dense retrieval (like BERT) understands the meaning behind words for more accurate results. Hybrid retrieval combines both, offering the best of both worlds. Each method has its strengths. Understanding when to use them is key to better search results. 

10

4 Comments

LangChain's modularity and flexibility enable the seamless orchestration of various LLM components, making it an ideal framework for constructing intricate conversational flows. Paired with LangGraph's visual design capabilities for managing these flows, development processes are notably more intuitive and streamlined. Read my take. #machinelearning #LLM #Langchain #Langgraph

13

Ever wondered how random numbers work in NumPy and why setting a random seed is important? 🌟 Check out my latest blog to learn more about true vs. pseudo-randomness and the role of random seeds. #Randomness #PseudoRandomness #NumPy #PythonProgramming #Algorithms #Computing #Cryptography #ScientificComputing

21

6 Comments

Many of ML practitioners might have heard the term "Extending context length" for LLMs. In case if you want to dive deeper into this topic along with its code implementation, give this blog a read. I have described about Position Interpolation, NTK aware and Dynamic NTK method, which are widely used for extending context length.

69

💡 Enhancing Retrieval Accuracy with Contextual RAG As GenAI systems go beyond demos and are deployed at production scale, improving retrieval accuracy remains a top priority for building robust Retrieval-Augmented Generation (RAG) systems. One promising technique that Anthropic recently discussed on their blog? Contextual RAG. The idea is to run LLMs on document chunks first to create context before running them through the standard embeddings pipeline. This method alone can reduce failure rates by 35%. And adding additional techniques like BM25 indexing and Reranking - techniques that are already well researched - improves failure rates by 67% 🤯 As token costs continue to drop and smaller models improve, these kinds of approaches become increasingly viable for large-scale use cases. You could, for example, use a smaller LLM for context creation while leveraging a larger, more powerful model for generating responses. Combining models in this way can help strike a balance between performance and cost. #AI #LLMs #RAG #ContextualAI #ScalableAI

25

3 Comments

If you've ever worked with ML models for medical imaging then you know how long the computation time will be. Here's a quick tip help with that. If you deal with CT or MRI scans of hundreds of slices then your model will take minutes to finish. Now imagine having multiple of these models working together for a certain application! Inference time can quickly get out of hands! In such a case, you most likely have a model orchestrator which is a piece of code that sends requests to your models APIs and then collect the outputs and order them. What you should do is leverage asynchronous processing. Meaning, run all of your models in parallel at the same time without waiting for one model to finish before calling the second one. Of course, this supposes that each model is deployed separately in its own API. But what if some models depend on other models outputs? You can still use asynchronous processing, but you'll be picking and choosing which models need to run first and which models need to wait a little bit. A great package to do this in Python is asyncio.

32

Excited to share our interactive BlogApp on Generalized Linear Models (GLMs)! In collaboration with Venkatachalam Subramanian Periya Subbu, Satya Lasya N, and Pooja Baralu Umesh, we've developed an intuitive tool that makes understanding GLMs both engaging and accessible. If you are an aspiring Data Scientist, Actuary, Statistician, Economist, Epidemiologists or an Engineer, you definitely need to check it out at: https://lnkd.in/gMqf3M5Y View the source code: https://lnkd.in/g9HUm_7r Special thanks to Professor Cody Carroll for the guidance throughout this project. Let's discuss GLMs! What's your experience with these powerful statistical tools? Share your thoughts below. #DataScience #Statistics #MachineLearning #GLM #Analytics #Programming #USF #MSDS

34

2 Comments

GPT-4o says "9.11 is bigger than 9.9". It is extremely fascinating how AI models which can understand quantum mechanics, have read books from Tolstoy to Principia, and are now achieving super-human scores on knowledge benchmarks can fail at something as trivial as interpreting the decimal point system. It would almost be like if a top scientist made groundbreaking discoveries in physics and then said 1x1=2 *. These issues are at the heart of why AI models are considered unreliable. To build a world where AI models are ubiquitous, we need to figure this out ASAP. I think the most important step here is understanding WHY these mistakes happen. To this end, the recent interpretability work done by Anthropic will be crucial. https://lnkd.in/gvMKkGXX Further thoughts in comments. * Any resemblances to actual people are purely coincidental 😜 #AI #Anthropic #LLMs

58

28 Comments

Ran a quick test on the new #OpenAI #o1mini to solve IIT-JEE Advanced 2024 maths questions. It accurately solved 13 out of 17 questions with an impressive accuracy of ~76%. Quick thoughts: 1. The hidden chain of thought is quite detailed and extensive. Breaks the problem, solves and verifies and iterates multiple times. This might increase the output token cost by a big margin. 2. Seems there are tools integrated for calculations. Much needed as #gpt4o does a bad job at it. 3. Latency is quite high. Some questions took more than 50 seconds to get answered. This needs to come down for production user-facing use cases. 4. Hallucination still exists. Check the 1st question in the link, it had the correct step-by-step reasoning but hallucinated at generating the final answer. 5. Overdoing self-thought. Saw a couple of instances where it did generate correct answers in the process but eventually made wrong answers. Would give a clearer picture once the rate limits are increased and benchmarking is done on a bigger sample. Let me know any interesting findings and thoughts 😀 https://lnkd.in/giSCezyG #GenAI #OpenAI #PhysicsWallah

139

5 Comments

🌟 **Decoding Vector vs. Graph Databases for RAG & RAFT – Which One Wins?** When it comes to enhancing large language models through Retrieval Augmented Generation (RAG) or Retrieval Augmented Fine-tuning (RAFT), the choice of database—vector or graph—plays a pivotal role. 🔹**Vector Databases:** Data is partitioned into chunks and each chunk is transformed into a vector using a language model. These vectors are then indexed in a vector database, facilitating quick nearest neighbor searches to find the most relevant information chunks in response to a query. 🔸**Graph Database:** This database shines in storing relationships between entities. By parsing and indexing these connections, graph databases excel at constructing detailed subgraphs for specific queries, ensuring that only the most relevant, contextually connected information is retrieved. It’s about understanding the network of relationships.An LLM is good at extracting the below kind of triplet information. [Entity A] -> [Relationship/Context] -> [Entity B] Both have their strengths—vector databases are great for fuzzy matching large data sets quickly, while graph databases excel in precise context and relationship extraction. 🚀 **The cutting edge? Combining both!** : This hybrid model takes advantage of the quick searching capability of vector databases with the deep relational insights of graph databases. For RAG or RAFT, this means not only retrieving the most relevant information quickly but also understanding its context within a broader knowledge base. For example, if a model is answering a query about "healthy eating," a vector database can quickly fetch documents mentioning health and nutrition. A graph database then elaborates on how these concepts connect with diet plans, medical advice, and recipes, providing a comprehensive, context-rich response. #GraphRAG #RAG #RAFT #GenerativeAI #GraphDB #VectorDB

13

Resharing it: This is a great summary shared by the LinkedIn engineering team about their #llmops and LLM deployment work and their key learnings. #llms #chatgpt4 #gemini #aiml #mlops #llmops https://lnkd.in/d2bH5GRR

2

Excited to share insights from my recent conversation with Shaan Mukhtar and Scott Cecil about the open-source library RAY (https://lnkd.in/gE6sShCe) ⚡️ This discussion opened my eyes to the potential of Ray for accelerating data science workflows. Here's what I learned: - Ray simplifies data ingestion, feature engineering, and deployment, especially on unstructured data. - Ray Serve makes deployment a breeze with just one line of code. - Scaling up and down is effortless, utilizing both CPUs and GPUs. - KubeRay enables seamless integration with Kubernetes for scaling. - Ray is a game-changer for LLMs, providing efficiency and lower latency. - While SQL remains essential, Ray can complement and even replace Spark in certain scenarios like unstructured data. I'm eager to explore Ray further and put these learnings into practice. Thanks to Shaan and Scott for the insightful conversation and personalized connection! 🙏 #Ray #DataScience #MachineLearning #LLMs #Deployment #Scaling

27

4 Comments

Why do attention models require positional encoding while RNN or CNNs do not? How does it work? I have provided a detailed explanation in my recent blog. #LLMs #AttentionModels #GenaI #PositionalEncodings https://lnkd.in/dHzQfDsi

18

A good explanation of advantages of using MLFlow

8