Ricardo Mokhtari

Brain Teaser #11: What mathematical arguments can explain how **Layer Normalization** enhances training stability in large language models (LLMs) and Transformer architectures, particularly in mitigating issues like vanishing and exploding gradients? Answer: Training destabilization in neural networks, particularly due to gradient explosion or vanishing, typically occurs because of the high condition numbers of weight matrices during backpropagation. These large condition numbers lead to unstable gradient scaling, either amplifying gradients excessively or diminishing them, hindering effective learning. How Layer Normalization overcomes and brings stability in training is briefly discussed here: https://lnkd.in/gmWm42tQ #MachineLearning #DeepLearning #Transformers #LargeLanguageModels #LayerNormalization #GradientStability #AIResearch #DataScience #NeuralNetworks #AIInsights

16

5 comentarios

Google "Quantum error correction below the surface code" unedited version available in Nature: https://lnkd.in/dqMWEYUB Older version, with more supplemental material (and open access), in arxiv:

13

1 comentario

The Information explores how OpenAI's reasoning models could help counteract potentially slowing progress from pretraining of larger more sophisticated models with more incremental gains over earlier models. It also notes that "relatively few developers" are using o1 models. While I haven't seen the full-fledged o1 models (and am certainly not a developer), the current models sometimes think hard about how to break code -- have seen them replace functional authentication with placeholders that you have to go back and update later, or swap out the correct variables with wrong ones -- for instance upper- or lowercasing something you didn't ask for, or adding an underscore where you didn't want one. At present, I think it still is better to have tighter feedback loops with the user -- otherwise, the genAI is thinking long and hard -- while a game of telephone can ensue where mistakes sometimes increase in the process of reflection -- about something that might not be what you wanted. A snippet from the paywalled article. The Information has done a lot of unique reporting on the subject.

4

1 comentario

🚀 Advancing Information Retrieval with APEER: Automating Prompt Engineering for LLMs! 🚀 I recently delved into a fascinating paper on APEER (Automatic Prompt Engineering Enhances LLM Reranking), a method transforming the way we approach relevance ranking in Information Retrieval (IR) using Large Language Models (LLMs). 🌟 🔍 Challenges in IR with LLMs: Current IR methods with LLMs often rely heavily on human-crafted prompts for zero-shot relevance ranking. This process is time-consuming, subjective, and lacks scalability. The complexities of integrating query and passage pairs and conducting comprehensive relevance assessments further complicate the effectiveness of existing methods. 💡 APEER: A Game-Changer APEER addresses these challenges by automating prompt engineering through iterative feedback and preference optimization. Here's how it stands out: Feedback Optimization: Continuously refine prompts based on performance metrics. Preference Optimization: Improves prompts by learning from positive and negative examples. 🌟 Key Highlights: Reduced Human Effort: APEER minimizes the need for manual prompt crafting, making the process more efficient and less reliant on human expertise. Enhanced Performance: Demonstrates significant improvements in LLM performance on IR tasks, with notable gains in metrics like nDCG@10. For example, APEER achieved an average improvement of 5.29 nDCG@10 on eight BEIR datasets over manual prompts on the LLaMA3 model. Better Transferability: Shows consistent outperformance across diverse datasets and LLM architectures, including GPT-4, LLaMA3, and Qwen2. 📊 Robust Validation: APEER has been rigorously tested on multiple datasets such as MS MARCO, TREC-DL, and BEIR, ensuring its robustness and effectiveness in various IR scenarios. 🌐 Implications: This advancement represents a significant step forward in optimizing LLM prompts for complex relevance ranking tasks. By reducing manual intervention and enhancing the efficiency of LLMs, APEER paves the way for more scalable and accurate applications in real-world IR tasks. 🛠️ Conclusion: APEER's automated approach to prompt engineering is a major leap towards more effective and scalable IR systems. It highlights the potential of integrating iterative feedback and preference optimization to refine LLM performance, offering promising avenues for future research and practical applications. Looking forward to seeing how APEER influences the landscape of Information Retrieval and beyond! #InformationRetrieval #LLM #AI #MachineLearning #NLP #DataScience #Research

1

[𝐌𝐚𝐬𝐭𝐞𝐫𝐢𝐧𝐠 𝐌𝐋]: 𝐁𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐚 𝐌𝐚𝐜𝐡𝐢𝐧𝐞 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐏𝐢𝐩𝐞𝐥𝐢𝐧𝐞 𝐟𝐨𝐫 𝐁𝐫𝐞𝐚𝐬𝐭 𝐂𝐚𝐧𝐜𝐞𝐫 𝐏𝐫𝐞𝐝𝐢𝐜𝐭𝐢𝐨𝐧 I'd like to share a project where I trained some machine learning models to predict whether a tumor is malignant or benign using the Breast Cancer Data Set from Kaggle (https://lnkd.in/dT2ET7qA). Here's a breakdown of the process: 1. 𝐈𝐦𝐩𝐨𝐫𝐭 & 𝐃𝐚𝐭𝐚: I started by importing essential libraries, loading the dataset, and transforming it into a Pandas DataFrame for easier manipulation. 2. 𝐃𝐚𝐭𝐚 𝐀𝐧𝐚𝐥𝐲𝐬𝐢𝐬 & 𝐄𝐱𝐩𝐥𝐨𝐫𝐚𝐭𝐢𝐨𝐧: I explored the features, checked data types, identified and handled missing values, removed unnecessary features, and analyzed data distribution and variable correlations. 3. 𝐎𝐮𝐭𝐥𝐢𝐞𝐫 𝐀𝐧𝐚𝐥𝐲𝐬𝐢𝐬: I identified potential outliers visually and used z-scores to spot them. Then I built a new DataFrame without these outliers to compare the ML model based on the performance metrics. 4. 𝐃𝐚𝐭𝐚 𝐏𝐫𝐞𝐩𝐫𝐨𝐜𝐞𝐬𝐬𝐢𝐧𝐠: I separated the target variable from the predictors, applied label encoding to the target, and scaled the data — both for the original DataFrame and the one w/o outliers. 5. 𝐇𝐞𝐥𝐩𝐞𝐫 𝐅𝐮𝐧𝐜𝐭𝐢𝐨𝐧𝐬: I developed functions to streamline predictions, including those for splitting data into training and testing sets, performing PCA, and plotting model metrics. 6. 𝐏𝐫𝐞𝐝𝐢𝐜𝐭𝐢𝐨𝐧 𝐌𝐨𝐝𝐞𝐥𝐬: I tested a range of models, including Naive Bayes, Logistic Regression, SVM, KNN, Decision Tree, Random Forest, and XGBoost. 7. 𝐌𝐨𝐝𝐞𝐥 𝐀𝐧𝐚𝐥𝐲𝐬𝐢𝐬 & 𝐂𝐨𝐦𝐩𝐚𝐫𝐢𝐬𝐨𝐧: Finally, I compared the performance metrics of all models to determine the most effective approach for this classification task. You can check out the full project, including the Python Notebook, on GitHub: https://lnkd.in/dk6Q2bsQ. I'm also writing a detailed blog post to walk through the entire process and share insights on training machine learning models. Stay tuned. https://lnkd.in/dk6Q2bsQ

34

2 comentarios

Your LLM isn’t doing math—it’s using a small circuit of neurons with simple pattern-matching, not real algorithms. Researchers found LLMs rely on ~200 neurons per layer using pattern-matching heuristics, not algorithms, to achieve 96% faithfulness in arithmetic tasks. Key points: ▸ Each neuron applies simple heuristics to identify numerical patterns. ▸ 200 neurons per layer handle arithmetic, only 1.5% of the model. ▸ Ablating individual heuristics reduces accuracy by 29%. ▸ Mechanism appears early in training and persists across models. ▸ Faithfully reproduces model behavior with 96% accuracy. Paper: https://lnkd.in/gm5qYUzw ↓ Are you technical? Check out https://AlphaSignal.ai to get a daily summary of breakthrough models, repos and papers in AI. Read by 200,000+ devs.

978

45 comentarios

I believe that the latest high performance LLMs would even demand external knowledge in order to achieve nearly perfect accuracy as long as they rely on the current popular architectures like transformer. The knowledge graph is one form of external information that could help LLMs’ inference. Those graphs are sometimes processed by GNNs. The latest eye-catching topic regarding GNN is GNN-RAG. Prior to going over GNN-RAG, I would like to run over one of the related works, RoG (Reasoning on graphs). RoG is intended for question-answering tasks. More often than not, using only LLMs does not achieve high accuracy. One reason is lack of information. RoG has LLMs create a set of relation paths based on the question and retrieve subgraphs based on the paths. For example, when you want to know who are someone's uncles, you might first search for her/his mother and father and then their brothers unless you can directly find his/her uncles. RoG works like that. RoG predicts that the relation paths such as “mother of, brother of” or “father of, brother of” help in finding useful information from the knowledge graphs for the question “Who are his/her uncles?“ You need to adjust the LLMs to the knowledge graphs you use when you employ RoG. And, the LLMs might not always guess helpful relation paths. Although, RoG improves accuracy compared to the conventional approaches, such as only using LLMs or extracting subgraphs from knowledge graphs without planning. https://lnkd.in/dfuqmAj6

14

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

29

Here’s a blogpost that breaks down our work on Attribute Controlled Fine-tuning for Large Language Models . We introduced a new method that trains an auxiliary model to control a specific attribute (in this case toxicity). The approach regularizes the LLM's training by penalizing deviations from the desired (non-toxic) distribution, using the auxiliary model trained alongside the core LLM. This work, published at #EMNLP2024, was led by our intern Tao Meng (UCLA) together with our collaborators Ninareh Mehrabi, Palash Goyal, PhD, Anil Ramakrishna, PhD, Aram Galstyan, Richard Zemel, Kai-Wei Chang and Rahul Gupta. Blogpost: https://lnkd.in/eRzvKSKi Paper: https://lnkd.in/eHNwgWAt

34

Neural networks are frequently employed to solve Partial Differential Equations (PDEs) across various domains. However, nonlinear PDEs with multiple solutions present a significant challenge for existing neural network techniques. Function learning strategies attempt to learn the solution function but often falter due to an ill-posed problem. Conversely, operator learning methods, such as Physics-Informed Neural Networks (PINN) and DeepONet, aim to approximate the mapping between parameters and solutions. Introducing NINO: a novel method designed to tackle nonlinear PDEs with multiple solutions through operator learning techniques. NINO enhances traditional Newton methods by integrating them into a superior network architecture and formulating problems more effectively. NINO surpasses conventional Newton methods and neural operator techniques in terms of execution speed. It employs two distinct training methodologies: one using Mean Squared Error Loss and another that combines supervised and unsupervised learning. In experiments, the neural operator method efficiently learned the Newton operator, requiring minimal supervised data. https://lnkd.in/edM5VRWh

1

There is a new LLM & RAG framework in town that you must know about, as it might replace LangChain and LlamaIndex ↓ The new RAG framework targets to be the PyTorch library for LLM applications. It prioritizes: - simplicity - modularity - robustness - a readable codebase ...over out-of-the-box one-liners that are hard to understand and extend. ...which we often encounter when working with LangChain and LlamaIndex. . 𝗦𝗼, 𝘄𝗵𝗼 𝗶𝘀 𝘁𝗵𝗲 𝗻𝗲𝘄 𝗸𝗶𝗱 𝗶𝗻 𝘁𝗼𝘄𝗻? → 𝘓𝘪𝘨𝘩𝘵𝘙𝘈𝘎 It was initiated by Li Yin, who started implementing LLM solutions before they were cool and realized how limited one is when building custom solutions. She highlighted that each use case is unique in its data, business logic, and user experience. Thus, no library can provide out-of-the-box solutions. Ultimately, LightRAG aims to provide a robust and clean codebase you can 100% trust, understand and extend. With the ultimate goal of quickly customizing your own LLM, RAG, and agent solutions. . If you are into LLMs, RAG and agents, consider checking it out on GitHub and support it with a ⭐️ (or even contribute) ↓ 🔗 𝘓𝘪𝘨𝘩𝘵𝘙𝘈𝘎: https://lnkd.in/d-fcDZ3A #machinelearning #mlops #datascience . 💡 Follow me for daily content on production ML and MLOps engineering.

391

29 comentarios

Excited to be sharing another blog post, this time diving into the topic of regularisation in neural networks! Some popular regularisation techiniques can initially seem quite bizarre, but after reading about their central importance in deep learning I wanted to dig deeper for myself. I give both experimental and theoretical justifications for some common regularisation techniques, which I hope makes the topic both interesting and accessible. Excited to hear your thoughts! https://lnkd.in/d-6prCCJ

19

1 comentario

Our new paper on quantum error correction is up on Arxiv https://lnkd.in/egBAyigt. It illustrates new circuit design algorithms for Clifford logical operations on the family of Hypergraph Product codes, which are one of the most popular code families of interest in near term intermediate scale quantum devices.

40

3 comentarios

We are thrilled to announce that our ICML paper, titled "Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation," is now available on arXiv. This paper presents one of the first automated, interpretable, task-specific evaluation methods for Retrieval-Augmented Generation (RAG) in Q&A contexts. For a summary of our contributions, check the following 🧵 from my co-author Laurent Callot on 𝕏: https://lnkd.in/gBF5iXaR. https://lnkd.in/g6r3Xv27 #ICML #LLMEvaluation #AmazonScience #RAG

138

5 comentarios

🚀 Demystifying GraphRAG 🌐 I spent my weekend to understand in-depth, what exactly is the underlying mechanism and algorithm of GraphRAG - A groundbreaking approach that combines the strengths of retrieval-augmented generation (RAG) and advanced graph-based indexing (KG) to tackle complex query which gives additional context, story behind the scene. ⚙️ How Graph RAG Works: 1️⃣ Entity Knowledge Graph Creation: Graph RAG begins by processing source documents to build a comprehensive entity knowledge graph. This graph includes nodes representing entities, edges depicting relationships, and covariates that provide additional context. This structured representations helps answer more complex queries, to uncover relationships between multiple entities are involved. 2️⃣ Community Detection with Leiden: A key feature of GraphRAG is its use of the Leiden algorithm, which efficiently detects communities within the graph. This algorithm identifies groups of closely related nodes, enabling a modular structure that enhances summarization capabilities. It works by iteratively moving nodes between communities to optimize a quality function. 3️⃣ Query-Focused Summarization: When a user asks a query, GraphRAG generates partial responses from each community summary in parallel. These responses are then aggregated into a final global answer, providing users with comprehensive insights tailored to their questions. 🔑 Benefits of Graph RAG: Enhanced Comprehensiveness: By leveraging community summaries, GraphRAG ensures that answers cover all relevant aspects of a query, leading to richer insights. Improved Diversity: The approach captures varied perspectives by summarizing information from multiple communities, resulting in well-rounded responses. Scalability: Designed to handle large datasets efficiently, GraphRAG can process millions of tokens without losing critical context. Modular Structure: The use of the Leiden algorithm allows for effective partitioning of information, making it easier to navigate complex datasets. full paper here - https://lnkd.in/dyczAj8D! Checkout the side by side comparison I created on how GraphRAG's "Global Search" is effective in unlocking connected entities for broader understanding. #GraphRAG #NLP #DataScience #LLM #Summarization #LeidenAlgorithm

55

To first get the elephant out of the way, the graph says no such thing. The y-axis is "attention on RL (reinforcement learning)", and not any indicator of the strength or intelligence of the models. I am now seeing a lot of posts about "RLHF is barely RL", and that we should try to make the models "more RL". This is part true but subtly incorrect. What we want is more search, not sparse scalar rewards. By search I obviously mean the algorithmic sense, not "search engine" sense. Search is about exploring the solution space to find satisfactory or best possible solutions. RL means using search to optimize sparse scalar reward feedback. RL in the classical sense doesn't scale, because any reasonable reward signals aren't informative enough to learn about the world. That's why RL systems generally have all sorts of hacks from proxy reward functions, like intrinsic rewards and differentiable critics, to world models which learn by self-supervision. Over the history of RL, we have actually done less and less RL, and more and more self-supervises learning, to get more information and training feedback from the world. We have also swapped naive rewards to depictions of goal states (for controllability), or to preferences (e.g. DPO, to sidestep the impossible challenge of defining a good reward signal). May I also point out how terribly bad an idea it is to set a single reward objective to a highly intelligent entity? It will make a huge mess, no matter how smart you think you were when you set it to be rewarded by profit, or by closed sales. It will not only make a mess, but it will lead to a monomaniacal specialist AI, not a generalist one. It will be weak and highly exploitable by AIs not encumbered by the same dysfunctions. What we need is more search over open-ended non-imitative objectives, not "more RL".

8