Shahab Mofidi

Can LLM preference tuning degrade Multi-Agent system performance? In a recent tweet, Andrej Karpathy questioned why all LLMs sound similar, prompting Sebastian Rascha to speculate about the uniformity of alignment datasets. Aligning user preferences is a last-mile step in training large language models. Regardless of the pre-training datasets used, could the preference datasets used during alignment compromise the effectiveness of LLMs? This issue could adversely affect multi-agent systems incorporating multiple LLMs in their design.

7

1 Comment

We are running out of data to train our models, no joke. Jack Clark confirms what we already know: foundation models have been trained on meaningful percentages of all Internet data. Researchers from Epoch estimate that we will exhaust 1. Low-quality language data by 2032 2. High-quality language data by 2024 3. Image data between 2030 and mid-2040s Source: 2024 AI Index Report

10

6 Comments

There is no need to hold your breath any longer! My next article in the series, 'The ABCs of AI,' is ready. This time, we delve into the world of Artificial Neural Networks. 🤖✨ Understanding these digital neurons is key to unlocking AI's potential. From their structure to real-world applications, let’s explore how they’re reshaping our tech landscape. Dive in and share your thoughts—let’s spark a conversation! #ArtificialIntelligence #NeuralNetworks #MachineLearning #TechInnovation #AITrends

7

A popular DS interview question: MLE vs. EM — What’s the difference? Maximum likelihood estimation (MLE) and expectation maximization (EM) are two popular techniques to determine the parameters of statistical models. Due to its applicability in MANY statistical models, it's asked in many DS interviews (especially the distinction). Here's how they work. 1) MLE: MLE determines the parameters of a statistical model fitted on a LABELED dataset. The process is pretty simple. ↳ Step 1) Start by assuming a data generation process. This reflects our belief about the distribution of the true label (y), given an input (X). ↳ Step 2) Define the likelihood (L) of observing the data, parameterized by θ. ↳ Step 3) Obtain optimal parameters (θ) by maximizing the likelihood (L). This gives the parameters that most likely generated the data (X, y). MLE Done! -- But what if we don’t have true labels? For instance, say the data comes from multiple distributions (a mixture). We don't know the true labels but wish to estimate the parameters of every distribution. MLE, as you may have guessed, will not be applicable. The true label (y), being unobserved, makes it impossible to define likelihood. In such cases, techniques like expectation maximization (EM) are helpful. -- 2) EM: EM is an iterative technique to estimate model parameters, especially for UNLABLED data. Here's the core idea: ↳ Step 1) Make a guess about the initial parameters (θ) ↳ Step 2.1) E step: Compute the posterior probabilities of the unobserved label (let’s call it ‘z’) using θ. - Here, ‘z’ is also called a latent variable, which means hidden or unobserved. - Relating it to our case, we know that the true label exists in nature. But we don’t know it. - Thus, we use a latent variable ‘z’ instead to estimate its posterior probabilities using the guessed parameters. - These probabilities tell the belongingness of a point to a distribution. This is in contrast to MLE, where belongingness is certain due to presence of true labels. ↳ Step 2.2) Define L: Now we have a proxy (not precise, though) for the true label, we can define an “expected likelihood” function. ↳ Step 3) M step: Now we have a likelihood function to work with. Maximizing it with respect to θ gives us a new estimate for the parameters (θ`). ↳ Step 4) Go back to step 2, and recompute the posterior probabilities using updated parameters (θ`). Next, update L, and maximize it, to obtain another estimate for the parameters. Repeat until convergence. As explained above, in EM, we repeatedly iterate between the E and the M steps until the parameters converge. A good thing about EM is that it always converges. But, at times, it might converge to a local extrema. The image below summarizes the differences. To learn more, check this recent issue of Daily Dose of Data Science: https://lnkd.in/gMu7VWTC. 👉 Over to you: Can you add more models for each MLE and EM?

167

1 Comment

A popular DS interview question: MLE vs. EM — What’s the difference? Maximum likelihood estimation (MLE) and expectation maximization (EM) are two popular techniques to determine the parameters of statistical models. Due to its applicability in MANY statistical models, it's asked in many DS interviews (especially the distinction). Here's how they work. 1) MLE: MLE determines the parameters of a statistical model fitted on a LABELED dataset. The process is pretty simple. ↳ Step 1) Start by assuming a data generation process. This reflects our belief about the distribution of the true label (y), given an input (X). ↳ Step 2) Define the likelihood (L) of observing the data, parameterized by θ. ↳ Step 3) Obtain optimal parameters (θ) by maximizing the likelihood (L). This gives the parameters that most likely generated the data (X, y). MLE Done! -- But what if we don’t have true labels? For instance, say the data comes from multiple distributions (a mixture). We don't know the true labels but wish to estimate the parameters of every distribution. MLE, as you may have guessed, will not be applicable. The true label (y), being unobserved, makes it impossible to define likelihood. In such cases, techniques like expectation maximization (EM) are helpful. -- 2) EM: EM is an iterative technique to estimate model parameters, especially for UNLABLED data. Here's the core idea: ↳ Step 1) Make a guess about the initial parameters (θ) ↳ Step 2.1) E step: Compute the posterior probabilities of the unobserved label (let’s call it ‘z’) using θ. - Here, ‘z’ is also called a latent variable, which means hidden or unobserved. - Relating it to our case, we know that the true label exists in nature. But we don’t know it. - Thus, we use a latent variable ‘z’ instead to estimate its posterior probabilities using the guessed parameters. - These probabilities tell the belongingness of a point to a distribution. This is in contrast to MLE, where belongingness is certain due to presence of true labels. ↳ Step 2.2) Define L: Now we have a proxy (not precise, though) for the true label, we can define an “expected likelihood” function. ↳ Step 3) M step: Now we have a likelihood function to work with. Maximizing it with respect to θ gives us a new estimate for the parameters (θ`). ↳ Step 4) Go back to step 2, and recompute the posterior probabilities using updated parameters (θ`). Next, update L, and maximize it, to obtain another estimate for the parameters. Repeat until convergence. As explained above, in EM, we repeatedly iterate between the E and the M steps until the parameters converge. That said, EM might converge to a local extrema. The image below summarizes the differences. I covered this in detail in the Daily Dose of Data Science newsletter. Find it in the comments. 👉 Over to you: Can you add more models for MLE and EM? -- 👉 Get a free data science PDF (530+ pages) with 150+ core DS and ML lessons. https://lnkd.in/gzfJWHmu

132

1 Comment

STGformer: Efficient Spatiotemporal Graph Transformer for Traffic Forecasting ((SUSTech, October 2024) Paper: https://lnkd.in/gQRkPQgq Abstract: "Traffic forecasting is a cornerstone of smart city management, enabling efficient resource allocation and transportation planning. Deep learning, with its ability to capture complex nonlinear patterns in spatiotemporal (ST) data, has emerged as a powerful tool for traffic forecasting. While graph neural networks (GCNs) and transformer-based models have shown promise, their computational demands often hinder their application to real-world road networks, particularly those with large-scale spatiotemporal interactions. To address these challenges, we propose a novel spatiotemporal graph transformer (STGformer) architecture. STGformer effectively balances the strengths of GCNs and Transformers, enabling efficient modeling of both global and local traffic patterns while maintaining a manageable computational footprint. Unlike traditional approaches that require multiple attention layers, STG attention block captures high-order spatiotemporal interactions in a single layer, significantly reducing computational cost. In particular, STGformer achieves a 100x speedup and a 99.8\% reduction in GPU memory usage compared to STAEformer during batch inference on a California road graph with 8,600 sensors. We evaluate STGformer on the LargeST benchmark and demonstrate its superiority over state-of-the-art Transformer-based methods such as PDFormer and STAEformer, which underline STGformer's potential to revolutionize traffic forecasting by overcoming the computational and memory limitations of existing approaches, making it a promising foundation for future spatiotemporal modeling tasks."

3

[research update] use a Transformer's multi-head attention activations to fetch RAG style, to augment multi faceted problem breakdown for LLMs in ."Multi-Head RAG: Solving Multi-Aspect Problems with LLMs": 1. Enhances LLMs by integrating document retrieval (RAG) into the LLM context. 2. Challenge: Existing RAG solutions struggle with queries needing multiple documents with varied contents due to distant embeddings. 3. Multi-Head RAG (MRAG): Introduces a novel approach using Transformer's multi-head attention activations to fetch diverse documents. 4. Activation Utilization: Different attention heads capture varied data aspects, improving complex query retrieval. 5. Improved Relevance: MRAG shows up to 20% improvement in relevance over standard RAG baselines. 6. Evaluation Methodology: Provides metrics, synthetic datasets, and real-world use cases to demonstrate effectiveness. 7. Seamless Integration: MRAG can be integrated with existing RAG frameworks and benchmarking tools like RAGAS. 8. Multi-Aspect Problem Solving: Designed specifically to handle multi-aspect queries more effectively. 9. Data Stores Compatibility: Compatible with various data store classes. 10. Potential Applications: Benefits scenarios requiring diverse information retrieval, enhancing LLM responses. Experiment The paper introduces MRAG and evaluates it using synthetic datasets and real-world data. It compares MRAG's performance against standard RAG models. The experiment focuses on the model's ability to retrieve relevant documents across various query aspects, demonstrating up to a 20% improvement in relevance over baselines. Use Case MRAG is tested on real-world scenarios like customer support, where queries often require information from multiple documents with diverse contents. The model's capability to retrieve and integrate multi-aspect data enhances the quality of responses.

81

5 Comments

Myths about Large Language Models (LLMs) like #Chatgpt #Gemini #llama and others - LLMs can understand like humans: LLMs work on pattern matching using algorithms. LLMs cannot understand the questions as humans do and so the response might not be always 100% as expected. - Accuracy of LLMs:While models try for accuracy, the information given might be incorrect or misleading. Model does not verify facts of the response - More data will improve response of LLMs:While more data can improve quality to some extent, fine tuning, transfer learning, tunning some parameters, modifying architecture of data flow, etc. can improve performance of LLMs. - LLMs can be creative like humans: LLMs can generate text, images or videos. While LLMs can augment human creativity but it cannot replace complexity of human creativity. - LLMs are invulnerable to errors or manipulation: LLMs can make mistakes and can be manipulated with adverse inputs to generate misleading information. - LLMs will surpass human intelligence: Artificial general intelligence (AGI) is still far off from reality and hence LLMs are far from achieving human-like intelligence. * views expressed are personal

42

3 Comments

LLM as a solver Software involving Combinatorial optimization techniques is ubiquitous. How does LLM fare compare to a brute force method? We prompt the LLM with, 1. A distance matrix, the distance between the cities 2. Starting node/city. And expect the shortest path between the cities starting from the given start city and the distance covered. https://lnkd.in/enMkT92W 1. Create a python dataclass to describe the input and out of a TSP problem. 2. Write a prompt template to describe the input, output and few-shot examples. 3. A quick brute force TSP problem generator for n-shot examples. 4. Prompt the LLM for answer.

17

4 Comments

Llama 3.2 is really mind blowing with multimodal capabilities and edge deployment options. This blows off some difficult use cases in document extraction and object detection. Check out this comprehensive summary post below from Armand Ruiz https://lnkd.in/dix6E4Ne

9

Machine Learning in Machine tools Thrilled to share a recent successful engagement with a Machine tool manufacturer by using Temporal Fusion Transformer (TFT) models for time series forecasting enabling a new level of precision and insight in their demand forecasting. Why TFT over Traditional Models? Traditional models often struggle with complexity, especially when faced with changing seasonality, trends, and multiple covariates. TFT addresses these challenges with: Multivariate Forecasting: TFT handles multiple variables simultaneously, capturing complex relationships between features, which enhances prediction accuracy. Interpretability: Unlike black-box models, TFT provides interpretability, allowing us to understand the impact of each feature on the forecast. Handling Missing Data: With in-built capabilities to manage missing data, TFT minimizes preprocessing, saving time and resources. Adaptability to Changing Patterns: TFT dynamically learns from both short- and long-term temporal patterns, adapting to shifts in data trends. For our Client, this means improved forecast accuracy, reduced inventory costs, and better resource allocation. This exercise has been transformative in enhancing decision-making, optimizing supply chain processes, and aligning production with demand. #AI #MachineLearning #TimeSeriesForecasting #TemporalFusionTransformer Do you think Machine learning models such as TFT-driven forecasting is transforming the manufacturing industry? Would like to know your thoughts on how the industry is approaching data-driven decision making?

16

2 Comments

Week 10 NeuroNuggets 🧠 : Prompt Engineering Using LLMs - Part 3   In this week's NeuroNugget, we will go over a second set of prompting techniques to improve an LLM's performance:   1) Chain of Thought ⛓: Chain-of-Thought (CoT) prompting, contrasts with the standard few-shot 💉 prompt approach by encouraging the language model (LLM) to reveal its reasoning process. Instead of just providing examples and expecting direct answers, CoT prompts guide the LLM to unfold its thought process. This method often leads to answers that are not only more accurate but also easier to understand and interpret. It's like having a detective 🕵‍♀️ mentor who not only shows you evidence but also teaches you how to think like a detective to crack the case on your own.   2) ReAct: Reason & Action 🏹(ReAct) prompting is a method that helps language models do two things at once: explain their thinking and decide what to do next. This approach lets them think through problems and make plans that they can change and improve as they go. It's like having a smart friend 👧 who not only gives you advice but also helps you understand the reasons behind it, making your plans more organized and effective.   3) Directional ↗ Stimulus: Directional Stimulus Prompting (DSP) introduces a small set of tokens called "directional stimulus" into the prompt. These tokens act like hints or clues, guiding the language models (LLMs) towards the right direction. For instance, in a summarization task, the directional stimulus could be specific keywords that should be included in the summary.   4) Meta Prompting: A Meta Prompt serves as a generic, structured template 🎶 that outlines how to approach a category of tasks without specific examples. It guides LLMs in reasoning through tasks by providing a framework that can be adapted with specific details. This method enhances efficiency and precision in utilizing LLMs, focusing on the methodology of problem-solving rather than the specific content. For eg, a meta prompt to create a product description would outline sections like key features, specifications, use cases, and conclusion, emphasizing structured content generation over specific product details.   This would be my last post in this series ;) Hope these posts have been helpful! Thank you for learning with me 💌  

3