Yonghua Yin

Exciting developments! 🐍 ML-Mamba’s linear complexity and efficient handling of long sequences could be a game-changer for multimodal tasks. The fact that it achieves comparable performance to top models like TinyLaVA and LLaVA, while using fewer parameters and being faster, is impressive. Looking forward to diving into the details and seeing how this innovation shapes the future of MLLMs! #MLMamba #AI #MultimodalAI #Transformers #MachineLearning #TechInnovation #LLMWatch #ArtificialIntelligence

1

Abacus Embeddings: Enhance Transformer Accuracy in Arithmetic Tasks The challenges transformers face in arithmetic tasks, particularly their struggle to track digit positions within large numbers. "Abacus Embeddings," a novel positional embedding method that encodes the relative position of each digit. This approach significantly improves accuracy in addition tasks, enabling models trained on 20-digit numbers to generalize to problems with up to 120 digits – a sixfold improvement over previous benchmarks. This accomplishment represents a significant leap in length generalization for learned addition. Architectural modifications, such as input injection and looped transformer architectures, demonstrating their ability to enhance performance when combined with Abacus Embeddings. Notably, these enhancements result in near-perfect generalization on the tested addition problems. The applicability of these techniques to other algorithmic tasks, including multiplication and sorting, demonstrating promising length generalization in these domains. This research provides valuable insights into improving algorithmic reasoning in transformers and opens avenues for future exploration in multi-step reasoning tasks. #Research #Transformers #AlgorithmicReasoning #Arithmetic #AbacusEmbeddings #ArtificialIntelligence #MachineLearning #LinkedinPost #NeuralNetworks #MathematicalReasoning #AI

7

💡 Great insights on fine-tuning LLMs using Sparse Mixture of Experts (MoE)! This method optimizes performance by activating only specific parts of the model, which can significantly enhance efficiency and cost-effectiveness. 📊💸 The study’s focus on training efficacy, runtime metrics, GPU utilization, and cost modeling is crucial for managing cloud expenses and improving resource allocation. 🔍🚀 It’s exciting to see advancements that help us get more out of our compute resources and budget accurately. A valuable read for anyone involved in AI research and development! 🙌 #AI #MachineLearning #LLM #MixtureOfExperts #TechInnovation #CostEfficiency #AIResearch

2

“LlamaIndex is an advanced orchestration framework designed to amplify the capabilities of LLMs like GPT-4. While LLMs are inherently powerful, having been trained on vast public datasets, they often lack the means to interact with private or domain-specific data. LlamaIndex bridges this gap, offering a structured way to ingest, organize and harness various data sources — including APIs, databases and PDFs.“ https://lnkd.in/gEQup-9r

1

RAG using tabular data with function calling - my talk from @pydatalondon is now available on youtube: The talk is scattered with references to Monty Python's Quest for the Holy Grail (tabular data is the holy grail of RAG for Enterprises). https://lnkd.in/dAkSUMiE

13

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

29

🚀 Apple has just released a groundbreaking tool for evaluating Large Language Models (LLMs) on real-world tasks: ToolSandbox! This new open-source benchmark is set to transform how we assess LLMs by focusing on their tool-use capabilities in complex scenarios. 🔧🧰 Key Features of ToolSandbox: Stateful Tool Execution: Unlike traditional benchmarks, ToolSandbox evaluates how well LLMs handle multi-step processes with stateful dependencies between tools, providing a more realistic assessment of their capabilities. Implicit State Dependencies: Tests the model's ability to manage and utilize tool outputs effectively in subsequent steps, mimicking real-world task execution. Built-In User Simulator: Allows for on-policy conversational evaluation, crucial for understanding how LLMs perform in interactive, back-and-forth dialogues while using tools. Key Findings: Performance Gap: A significant difference was observed between open-source and proprietary models, highlighting variations in tool-use capabilities. Complex Task Challenges: Even top-tier LLMs face difficulties with complex tasks such as state dependency, canonicalization, and situations with incomplete information. Evaluating LLMs is complex, but ToolSandbox brings us closer to understanding their performance in realistic, multifaceted scenarios. This tool promises to advance our capabilities in developing and deploying more effective AI models. 📬 Want to stay updated on the latest LLM research? Join my newsletter with 35k+ readers for in-depth insights and updates: llmwatch.com 💡 #ToolSandbox #LLM #AI #MachineLearning #Benchmarking #StatefulExecution #AIResearch #TechInnovation #OpenSource #ModelEvaluation 🔧🧰💡

1

Complexity science meets deep learning! 🧠🤯 Kolmogorov-Arnold Networks (KANs) are bridging the gap, offering higher accuracy & interpretability for tackling complex problems. Researchers: explore the theory! Practitioners: unlock new possibilities! #ComplexityScience #DeepLearning #AI #KANs #NeuralNetworks

4

https://lnkd.in/e6w5az6A Food 4 Thought 🤔📍 It's definitely not "Common Sense" to interpret Euler's Conditions in complex mathematical Equation's with trigonometrical expressions of Cos and Sine (Phi) Phi Vs (Teta, Beta or Mu).....is the 21st letter of the Greek alphabet, used to represent the "ph" sound in Ancient Greek. In the system of Greek numerals, it has a value of 500. Point is there is a need to recognize Euler's Rules & conventions even in Economics particularly when a THESIS needs to state or make reference to these conditions particularly when it relates to the "Law of Diminishing Marginal Utility"!! Diminishing Marginal Utility has absolute value Vs Euler's 2 period models of the Marginal Utility being equal rather than decreasing with the rate of interest also decreasing with the rate of Consumption. "The more red apple 🍎🍏 I eat I am more inclined to green apples and vice versa whether it's a point on a Tangent or 3 or 4 Dimensional Sphère"!! Question ❓🤔 Does all this make Sense !? Is it required to make Sense or is it to be substituted by an Evolving Theory which in Practice makes perfect sense with E Links (It's neither Teta, Beta, Pi, Phi or Mu) but "Virtual Interface" and e-banking e-Solutions that can authenticate and validate Transactions without having to divulge financially sensitive information like the 3 Digit CSV CVV Security Verification Code for Digital Payment's in the Fintech Space.

1

Interesting paper about the limitations of the generative AI. Actually Yann LeCun talks about this limitation as inherited problem in the existing models for long time. Knowing the limitation helps to align expectations. https://lnkd.in/dCid9B_J #generativeai #llm #artificailintelligence #ai

7

3 Comments

Hallucination is Inevitable in LLMs An important new paper out that shows that even with perfect data and an idealized architecture “LLMs will always hallucinate”. I suspect that this will over time be linked to Gödel’s Incompleteness Theorem [1] and Turing’s Halting Problem [2]. Every technology has trade-offs. We love LLMs but they need to be used in the right way. [1] Good pop science depiction in Oppenheimer (Gödel was Einstein’s walking companion) or also see Rebecca Goldstein’s book Incompleteness [2] Turing is depicted in The Imitation Game and also the progenitor of the famed Turing Test.

36

Refreshing to see other approaches showing promise beyond just RAG (Retrieval-Augmented Generation) with this new paper from BlackRock and NVIDIA. "HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction" introduces a new approach that effectively combines Knowledge Graphs and vector-based retrieval to enhance the accuracy and relevance of information extraction. #DataScience #AI This is particularly helpful when dealing with financial documents such as earnings transcripts, as HybridRAG demonstrates significant improvements in generating precise, contextually relevant answers. By leveraging both structured and unstructured data, it outperforms existing methods, offering a powerful tool for advanced financial analysis. #Finance #MachineLearning #AI Check out paper here https://lnkd.in/eNJwyCyR

14

When assessing the performance of Large Language models (LLMs) we see the primary focus being technical benchmarks, such as graduate-level reasoning, coding proficiency, and multilingual math capabilities. These metrics are undeniably important, but only paint part of the picture. The recent performance comparison of models like Claude 3.5 Sonnet, Claude 3 Opus, GPT-4o, and Gemini 1.5 Pro underscores this point. While Claude 3.5 Sonnet excelled in graduate-level reasoning, coding, and multilingual math, it begs the question: How long until we start incorporating soft skills and behavioural traits into our model assessments? Soft Skills: The Human Touch in AI [1] Problem-Solving: Beyond solving complex equations, AI should demonstrate strong analytical and problem-solving skills, much like what we expect from top-tier talent. This involves tackling intricate financial problems and developing innovative solutions. [2] Communication: The ability to communicate technical concepts clearly and effectively to both technical and non-technical stakeholders is crucial. AI models need to bridge the gap between data science and business operations, ensuring everyone is on the same page. [3] Adaptability: As technologies evolve, so should our models. Adaptability in learning new technologies, adjusting to changing environments, and handling multiple tasks simultaneously is a trait we value in human employees and should equally prioritise in AI. [4] Teamwork: Collaboration is key. AI models should facilitate and enhance teamwork, contributing to a positive and productive work environment. They should be designed to integrate seamlessly with human teams, complementing their efforts. [5] Attention to Detail: Accuracy and precision in financial data and software development are non-negotiable. AI must exhibit meticulousness in its operations, ensuring high standards of quality and reliability. Behavioural Traits: Ethical and Proactive AI [1] Motivation and Initiative: Just as we look for candidates who demonstrate a proactive attitude towards learning and self-improvement, our models should show a capacity for continuous improvement and an 'interest' in advancing the fintech landscape. [2] Ethical Judgement: Understanding ethical considerations, such as data privacy, fairness, and regulatory compliance, is paramount. AI models must be designed with these principles in mind, ensuring they operate within ethical boundaries. Integrating these soft skills and behavioural traits into our assessment criteria will provide a more holistic view of an AI model's capabilities. It's not just about excelling in isolated technical tasks but also about how well these models can adapt, communicate, and integrate into broader business contexts. #AI #SoftSkills

13

Conditional Computation in LLMs (MoEs): Understanding the Concept of MoEs The Mixture of Experts (MoE) works on the idea of conditional computation, allowing the model to only run through some parts (an Expert) for a given input compared to a dense model where all parameters are used. It consists of a gating mechanism that selects the experts for the given input, enabling the model to scale without increasing the required computation. Popular LLM architectures like Mixtral and Grok use the same concept. What is MoEs? The Mixture of Experts (MoE) makes a simple modification to the usual architecture by replacing all or some of the Feed Forward layers with an MoE layer. The MoE layer consists of: 1. Router: A parameter gating mechanism that selects the experts used to process the given input by producing a probability distribution over experts. 2. Experts: Standalone NN modules with independent sets of parameters trained jointly with routers. It can use more complex architecture to create hierarchical MoE modules by implementing each expert as another MoE. - Implementation in Switch Transformers : Training was found to be 7x faster than training the base T5, keeping computation and training time constant. - Gshard and MeshTransformer : These frameworks propose efficient methods to train the MoE architectures using the parallelisation of models across GPUs/TPUs. Challenges with MoEs and Solutions 1. Inconsistent Batch Size for Experts: Can be overcome with the use of "soft" constraints in training loss. 2. Load Balancing Across Experts: The issue of repeatedly using the same few experts while training can be overcome by including better balancing mechanisms. 3. Model Instability Issues: Use of router-z Loss as an auxiliary loss to encourage smaller values within the router mechanism. 4. Model Overfitting: Requires different hyperparameter settings for effective fine-tuning. It was found that models benefit from high learning rates and smaller batch sizes. Recommended Reading/References 1. Hugging Face Blog on MoE - https://lnkd.in/g36gQJTx 2. Cameron Wolfe's Substack on Conditional Computation - https://lnkd.in/gtpjMHnJ

60

1 Comment

That says it all: “Sometimes, the most impactful research isn't about creating shiny new models. It's about saying the obvious thing out loud…. even if your work doesn't feel "novel" enough, your real-world experiences matter. Don't be afraid to state the obvious - it might just be what everyone's thinking, but no one's saying! ” 👉 It is very true, and proven in successful AI practices, that we can built excellent value using AI techniques on the same use case where others claim data quality issues, lack of resources, need for elaborate DL architectures, etc etc. 👉 In the end, experience, understanding of business processes and data, creative thinking, and vision are the key ingredients for success. #AI New York University

33

3 Comments

My collection of non-paywalled articles on robust portfolio construction. If you have no interest in consistently oriented bases for eigenanalysis, directional statistics, random matrix theory, stabilizing eigenvector pointing directions, the economic value of mean squared error, portfolio decision loss functions, shrinkage portfolio estimators, noisy covariance matrix estimators, ill-conditioned covariance matrix treatments, recursive clustering risk parity, hierarchical risk parity, portfolio optimization with tracking-error constraints, machine learning risk models, dictionary learning-based denoising, regularized Tyler's scatter estimators, James-Stein eigenvector shrinkage estimators, rotational invariant estimators, ways of using noisy observation matrices, additive or multiplicative rotational invariant perturbation,the replica method, asymptotic global law estimates, eigenvalue cleaning, short-sales and c-var constraints, the CTGAN algorithm, seriation methods for clustering-based capital allocation, quickest hub discovery, time-evolution of correlations, diversification measure distribution estimation, the method of hierarchical sensitivity parity, the commonality principle, hypersurface projections, asymmetric autoencoders, factor-based covariance matrix estimation, data augmentation for portfolio construction or deep learning in finance then just skip the first few papers in the list. Pull requests are welcome. #investing #portfoliomanagement #portfolio https://lnkd.in/eYxfU5UP

338

24 Comments