Sauradeep Debnath

Quantized embeddings offer significant advantages for NLP models, including reduced storage requirements, faster computations, lower energy consumption, and improved model interpretability. They are becoming increasingly important for enabling efficient and scalable NLP on resource-constrained devices and cloud-based applications. Read my latest article below

6

To CoT or not to CoT? Chain-of-thought (CoT) prompting is the most widely used prompting technique to improve LLMs' reasoning. In recent times CoT data have been used for training newer LLMs or for conducting extensive fine-tuning of LLMs on CoT. To understand the importance of CoT for today's LLMs, ChatGPT-o1 was trained on CoT data and uses it for reasoning: "Similar to how a human may think for a long time before responding to a difficult question, o1 uses a chain of thought when attempting to solve a problem. Through reinforcement learning, o1 learns to hone its chain of thought and refine the strategies it uses. " source: https://lnkd.in/dM7TmFVx But is CoT really this panacea for all the ills of LLMs? Does it really help to improve reasoning? According to a recently published study, CoT only gives benefits for tasks involving math and logic. For other reasoning tasks (such as common sense), however, it is useless. Moreover, according to the authors instead of using CoT one can use LLM with external tools with better results and more efficiency. In other words, better to use an LLM to create a plan and then use tools to execute it. CoT for the authors is unnecessarily expensive in most cases, it should be used only when needed. "CoT is unnecessary for many problems where it is widely employed: there exist more efficient prompting strategies that yield similar performance for much lower inference cost." article source: https://lnkd.in/dHuFvMtP my takes: https://lnkd.in/drajhkes #AI #LLM #promptengineering #CoT

5

🌿 SandalQuest: Digitizing Knowledge of Sandalwood Cultivation with AI 🌿 I’m excited to share our project SandalQuest, created as part of the ML-Fiesta: AI/ML Hackathon organized by International Institute of Information Technology Bangalore! Our team developed an AI-powered pipeline to preserve and provide easy access to the indigenous knowledge surrounding sandalwood cultivation in Karnataka. 🔑 What We Built: - An Automatic Speech Recognition (ASR) model tailored for colloquial Kannada, capturing informal language and dialect variations. - A speech-based question-answering system that allows users to ask questions and retrieve relevant answers from a transcribed audio database. - A searchable audio database that helps users access traditional knowledge through speech. 💻 Tech Stack: - Python, PyTorch for AI & Machine Learning - Whisper for Speech Recognition - Hugging Face Transformers for NLP - MongoDB for database management 🔍 Impact: We’re helping preserve valuable knowledge on sandalwood cultivation and making it easily accessible for farmers, researchers, and enthusiasts. 🔗 Check out the project: https://lnkd.in/dMxRP76s 🚀 Team Name: Code Wizards Team Members: - Shani Sinojiya (AI/ML & Backend Developer) - Mohammad Anas Africawala (AI/ML Engineer) - Tisha Patel (Full Stack Developer) It’s been an amazing experience working on this project, and I’m proud of what we’ve achieved. Looking forward to applying these insights to real-world applications! #AI #MachineLearning #SpeechRecognition #Kannada #ColloquialSpeech #Sandalwood #DataScience #NaturalLanguageProcessing #AIForGood #Hackathon #TechForGood #GoogleCloud #AIResearch #DeepLearning

10

2 Comments

Researchers have developed a novel "Hidden Chain-of-Thought" (HCoT) framework that significantly speeds up large language model inference while preserving multi-step reasoning capabilities. HCoT (Hidden Chain-of-Thought) differs from CoT (Chain-of-Thought) by compressing the multi-step reasoning process into a specialized token representation, allowing for faster inference while maintaining reasoning capabilities. Here are the key steps for implementing the Hidden Chain-of-Thought (HCoT) framework: 1. Data Preparation   - Construct HCoT training samples from original datasets.   - For math reasoning tasks, use GPT-4 to generate separate fields for thoughts and contents.   - For question answering, utilize existing fields from datasets like ScienceQA.   - For agent invocation tasks, employ fields generated through frameworks like ReAct. 2. Auxiliary CoT Model Training   - Extract thought processes from HCoT training samples.   - Insert special [CoT] tokens between content and thought segments.   - Train the model to maximize the likelihood of generating accurate thought representations.   - Incorporate symmetric contrastive loss to enhance thought compression. 3. HCoT Model Training   - Replace thought processes in original samples with [CoT] tokens.   - Fine-tune the model to generate content and [CoT] tokens.   - Freeze parameters of the auxiliary CoT model during this stage. 4. Inference Process   - The HCoT model generates content until encountering a [CoT] token.   - The auxiliary CoT model produces a compressed thought representation.   - The compressed representation is fed back into the HCoT model to continue generation. 5. Performance Evaluation   - Compare the HCoT model against baselines like zero-shot CoT and full CoT training.   - Evaluate accuracy across various tasks (math reasoning, science QA, agent invocation).   - Measure speedup in terms of sequence length and wall-clock inference time. 6. Optional CoT Recovery   - When full reasoning steps are needed, allow the auxiliary CoT model to generate the complete thought process.

11

1 Comment

🚀 Handling Unknown Words in NLP: Strategies for Success 🌟 In natural language processing (NLP), encountering an out-of-vocabulary (OOV) word can be a challenge. Here are some effective strategies to manage this issue: Using a Special Token (e.g., <UNK>): Many models use a special token like <UNK> to represent all unknown words. This token is included in the vocabulary and stands in for any OOV word. This approach allows the model to handle unseen words, though it loses specific information about each OOV word. Subword Tokenization (e.g., Byte-Pair Encoding, WordPiece): Subword tokenization techniques break words into smaller units (subwords) that are part of the vocabulary. This enables the model to represent OOV words as sequences of known subwords. This method reduces the number of OOV words and retains some information about their structure. Character-level Models: Some models operate at the character level, treating each character as a separate token. This allows any word, whether in the vocabulary or not, to be processed. This approach retains complete information about the word but may require processing longer sequences. Contextual Embeddings (e.g., BERT, GPT-3): Contextual embedding models like BERT or GPT-3 can generate embeddings for OOV words based on their context. These models, pre-trained on large corpora, can handle a wide range of words, including many that are not in the specific vocabulary used for fine-tuning tasks. Each of these methods has its pros and cons. The choice depends on the specific application and requirements of your NLP task. Stay tuned for more insights on NLP and AI advancements! 💡 #NLP #AI #MachineLearning #DataScience #LanguageProcessing #TechInnovation Feel free to connect and share your experiences or ask any questions! 🙌

9

Be aware of using LLMs in low-probability situations even if the tasks seem very similar. For example, they may work well for sorting alphabetically but fail often when sorting reverse alphabetically. Or they might do well on ROT-13, a task that is 60 times more commonly found in internet text than ROT-2. Here’s my review of a really fun paper I read recently on similar LLM gotchas and some questions AI practitioners must ask when building LLM applications.

19

3 Comments

Interested in quantization? Arun Nanda's article introduces this technique and sets the stage for exploring the journey from simpler models like BERT to advanced 1.58-bit LLMs. #LLM #ML

23

I was shocked to see the below graph 🤯. I saw this Google Search trend graph on Chat GPT VS Perplexity VS Claude. I now feel like I am in a bubble surrounded by tech folks who use claude etc and to me it feels normal. But as you can see in the graph, the actual use of Claude & Perplexity is very very minute compared to Chat GPT. Especially Perplexity feels like its non-existent. And now a month back Chat GPT Search is also launched which will definitely take a share of this audience. Looking at this graph also makes me optimistic about building, It clearly shows that there is a huge gap in the market, and similar products can be built. What do you guys think ? #ai #startups #chatgpt #claude #perplexity

20

4 Comments

💪 How about supercharging your RAG by using 32x lesser Memory leading to up to 45x Speedup while maintaining ~96% accuracy? Try Binary Quantization! 👇 🚀 But what is Binary Quantization? Here are some details: - Converts float32 values to 1-bit values, resulting in a 32x reduction in memory and storage usage. - Uses Hamming Distance for efficient retrieval, with just 2 CPU cycles. (super fast) - Rescoring step can boost performance. 🚀 Can't compromise on accuracy but still want a boost up? Then try out Scalar (int8) Quantization: - Maps float32 values to discrete int8 values, representing 256 distinct levels. - Uses a large calibration dataset of embeddings to compute the range and steps. - Can be combined with binary quantization for optimal performance. - Can provide up to 4x memory saving and 4x speedup while maintaining 99% accuracy! 🔗 Article: https://lnkd.in/dCSGuYJy #ai #ml #rag #datascience #quantization #machinelearning #llm #embeddings

22

3 Comments

Evaluating Large Language Model (LLM) performance in production involves monitoring various metrics to ensure the model meets business requirements and user expectations. Here are key metrics to evaluate LLM performance: 1. _Accuracy_: Measures the model's ability to generate correct responses. 2. _F1-score_: Evaluates the model's balance between precision and recall. 3. _Perplexity_: Assesses the model's ability to predict the next token in a sequence. 4. _ROUGE score_: Measures the quality of generated text compared to reference text. 5. _BLEU score_: Evaluates the model's ability to generate fluent and coherent text. 6. _Latency_: Monitors the model's response time to ensure it meets performance requirements. 7. _Throughput_: Measures the model's ability to handle multiple requests simultaneously. 8. _Error rate_: Tracks the frequency of errors or exceptions. 9. _User satisfaction_: Collects feedback through surveys, ratings, or user interviews. 10. _Business metrics_: Aligns with business objectives, such as revenue, engagement, or conversion rates. Additionally, consider: 1. _A/B testing_: Compares the performance of different models or configurations. 2. _Continuous monitoring_: Regularly tracks metrics to detect performance degradation. 3. _Model interpretability_: Analyzes feature importance, attention weights, or visualizations. 4. _Adversarial testing_: Evaluates the model's robustness to edge cases or attacks. 5. _Data quality_: Monitors the quality and relevance of training data. By evaluating these metrics, you can ensure your LLM model performs optimally in production, meets user expectations, and drives business value.

4

Hello Everyone Your learning - #learnwithvanama ---- 🔍 Exploring Best Practices in Retrieval-Augmented Generation (RAG) 📄 I recently delved into the insightful paper "Searching for Best Practices in Retrieval-Augmented Generation" by Xiaohua Wang et al., which offers a comprehensive survey on RAG techniques. This innovative approach merges large language models with information retrieval, enhancing content accuracy and relevance by tapping into external sources. Key Findings: - Mitigating Hallucinations: RAG proves effective in reducing inaccuracies and improving response quality, especially in specialized fields. - Multimodal Enhancements: Techniques for multimodal retrieval enhance question-answering related to visual inputs and expedite the generation of multimodal content. - Challenges: - Implementation complexities and extended response times are notable hurdles. - Integration Potential: Combining RAG with reinforcement learning or few-shot learning could yield more robust AI systems. Best Practices: - Continuous Evaluation: Regularly assess outputs to identify areas for enhancement. - Transparency: Clearly explain how outputs are generated to build user trust. - Personalization: Use customer integration data to tailor AI features effectively. The paper serves as a significant step forward in refining retrieval-augmented generation. While challenges persist, the proposed strategies and best practices offer a solid foundation for optimizing RAG and advancing AI capabilities. Embracing these insights can drive the development of more knowledgeable and contextually-aware AI systems. Link : https://lnkd.in/e3YJfq4q #AI #MachineLearning #Research #DataScience #Innovation #TechTrends

22

#Burnout is a challenge many professionals face, especially in high-demand industries like data science and #AI. In this post, Sasibhushan Rao Chanthati explores how #LLMs can offer a solution to this growing issue by optimizing workflows, enhancing productivity, and providing relief to overburdened teams. https://lnkd.in/g3D4YUyK #DataScience #BurnoutPrevention #WorkplaceWellness

1

Sarvam AI Unveils Sarvam-1: A Game-Changer in Indian AI! India’s Sarvam AI, co-founded by Vivek Raghavan and Pratyush Kumar, is headquartered in Bangalore and has launched Sarvam-1, a cutting-edge language model designed specifically for Indic languages. And it’s already outperforming the big players like Gemma-2 and Llama-3.2! Here’s why Sarvam-1 is making waves: ✅ Supports 10 Indian Languages + English: From Hindi, Bengali, Tamil, Telugu, Kannada, Malayalam, Marathi, Punjabi, Gujarati, Oriya, to English—this model bridges the language gap in AI-powered communication. ✅ Outshines Larger Models: With only 2 billion parameters, Sarvam-1 excels across benchmarks like ARC-Challenge and IndicGenBench. On TriviaQA, it achieved 86.11 accuracy across Indic languages, towering over Llama-3.1 8B’s score of 61.47. For translation, summarization, and cross-lingual tasks, Sarvam-1’s chrF++ score of 46.81 on the Flores dataset is unmatched. ✅ 4-6x Faster Inference: Its speed makes it ideal for edge devices with limited computing power. ✅ Efficient Tokenization for Indian Scripts: Traditional LLMs require more tokens per word for Indian languages, but Sarvam-1’s tokenizer brings this down to 1.4–2.1 tokens, similar to English’s efficiency. ✅ Trained on the Sarvam-2T Dataset: This 2-trillion-token corpus contains a rich blend of technical, scientific, and programming data across the 10 languages, enhancing its ability to handle complex reasoning tasks. Built on Yotta’s Shakti cluster with NVIDIA’s NeMo framework, the model was trained on 1,024 GPUs in just five days, showing incredible efficiency. Developers can now explore Sarvam-1 on Hugging Face, opening new doors for Indic NLP applications—from conversational AI to advanced translations. With Vivek Raghavan and Pratyush Kumar leading the charge, Sarvam AI is at the forefront of AI innovation in India, setting new standards in multilingual technology. The future of AI-powered conversations is here, and it’s proudly made in Bangalore! I am eagerly waiting for Sarvam's API to test it and develop meaningful use cases. Are you as well? Please feel free to comment below!" #AI #SarvamAI #MultilingualAI #TechForIndia #HuggingFace

37

How do you get to a quantized model when you start out from a standard model? Arun Nanda looks into the nitty-gritty details of several quantization approaches for neural network models.

6

One more session alert by Vidvatta for people interested in prompt engineering. Topic - Efficient Prompt Engineering and Tuning for Language Models! 🗓️ Date: 26th May 2024 🕒 Time: 11 AM IST ⏳ Duration: 1 hour 30 mins 🔗 Register Here https://lnkd.in/gurGZmeV ✨ Join us in this webinar where you will have a unique experience of doing hands on together with mentor in this session and get live feedbacks when you are stuck. Are you interested in Large Language Models (LLMs) and their potential? Curious about how to leverage advanced prompting techniques for various tasks? Our upcoming webinar is perfect for you! ✨ What to Expect: A brief overview of LLMs and the importance of prompting. Prompting and Task Complexity- Classification of tasks: Easy, Medium, Hard. Advanced Prompting Techniques: * Zero-Shot and Few-Shot Learning: Definitions and applications. * Task Decomposition: Simplifying complex tasks. * Chain and Tree of Thought: Enhancing reasoning capabilities. Interactive Q&A Session: Open floor for attendee questions and sharing experiences. Future trends in LLMs and prompt engineering. Don't miss this opportunity to delve into the intricacies of LLMs and enhance your skills. Register now and be part of the conversation shaping the future of prompt engineering! In case of any queries contact [email protected] #Webinar #LLM #PromptEngineering #MachineLearning #ArtificialIntelligence #FutureTech #LearningCommunity

13

1 Comment

Text chunks in retrieval may risk losing context. The below article introduces contextualizing each chunk using an LLM. The added cost and latency required for contextualizing is partially addressed through a lightweight LLM (haiku) and prompt caching. Along with other known techniques (BM25, reranking, etc.), a significant reduction in retrieval failure is achieved. Overall, a good article to read. https://lnkd.in/g_5fgAN8

10

🚀 What Happens to a Contextual Embeddings Model if a Word is Not Present in the Vocabulary? 🌟 In contextual embeddings models like BERT, GPT-3, etc., handling out-of-vocabulary (OOV) words is crucial for maintaining performance and accuracy. These models typically address OOV words by breaking them down into subword units using techniques like WordPiece or Byte-Pair Encoding (BPE). This means that even if the entire word isn't present in the vocabulary, its parts likely are. Here's how it works: 1. Subword Tokenization: When a word is not in the vocabulary, it is split into smaller subword units. For example, the word "newword" might be broken down into "new" and "word", or even into smaller units like "ne", "##w", "##word" in the case of BERT. This approach ensures that the model can still generate embeddings for parts of the word, capturing some of its meaning and allowing it to be processed effectively. 2. Retaining Semantic Information: By breaking down OOV words into known subwords, the model can retain and leverage semantic information from the subwords. This allows the model to handle a wide range of words, even those it hasn't seen during training. For instance, the subwords "new" and "word" in "newword" carry meaningful information that the model can use to infer the context and meaning of the OOV word. 3. Handling Complex Words and Morphology: Subword tokenization is particularly useful for handling complex words and morphological variations. It reduces the number of OOV words and improves the model's ability to generalize across different forms of words. For example, the word "unhappiness" might be broken down into "un", "##happi", and "##ness", allowing the model to understand its structure and meaning more effectively. 4. Contextual Understanding: Contextual embedding models like BERT and GPT-3 are pre-trained on large corpora, which helps them understand and generate embeddings for subwords based on their context. This allows them to handle OOV words dynamically and accurately. Even if a word is rare or newly coined, the model can use its context to generate meaningful embeddings and produce relevant responses. By leveraging subword tokenization techniques, contextual embeddings models can effectively manage OOV words, ensuring robust performance and accurate language understanding. Stay tuned for more insights on NLP and AI advancements! 💡 #NLP #AI #MachineLearning #DataScience #LanguageProcessing #TechInnovation #BERT #GPT3 Please follow my posts for more updates: Tejas Nanaware on LinkedIn #machinelearning #deeplearning #artificialintelligence #datascience #statistics Feel free to connect and share your experiences or ask any questions!

5

Did you know - bad Statistical inference can derail your MMM project? After all the hard work of collecting the data, building the models, the last thing you would want is faltering at the model inference stage. Read on to know the common misinterpretations of MMM outputs in this substack post. https://lnkd.in/gcn-2qmt

6