Asad Khan’s Post

Researchers at the University of Oxford developed a method to identify hallucinations in large language model outputs. Their approach estimates the likelihood of hallucinations by calculating the degree of uncertainty based on the distribution of generated meanings rather than word sequences. This method outperformed traditional entropy and P(True) methods, achieving an average AUROC of .790 across multiple datasets and models. Read our summary of the paper in #TheBatch: https://hubs.la/Q02HKD990

Surely great news to ensure safer and more effective AI deployment in various fields! In other more digestible words, Oxford scientists offered a new method proposed for detecting AI hallucinations (when AI systems generate false or misleading information). The researchers have developed a technique that enhances the reliability of AI outputs by identifying inconsistencies and discrepancies in generated content. This aims to improve the trustworthiness of AI systems, especially in critical applications where accuracy is essential. #hallucinations #AI #LLM #accuracy

🚨 Exciting Research Alert! 🚨 I'm thrilled to share a groundbreaking study that has just been published in Nature: "Detecting Hallucinations in Large Language Models Using Semantic Entropy" by Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. I believe this research can significantly enhance the reliability of AI applications, paving the way for safer and more trustworthy AI systems. I'm particularly excited about applying these insights to my own projects, as this method represents a major step forward in addressing the critical issue of hallucinations in LLMs. I'm eager to see how this innovation will transform the field and am keen to hear your thoughts on its potential implications! #AI #MachineLearning #LanguageModels #SemanticEntropy #Research #AItrustworthiness #TechInnovation #NaturePublication #LLM #AIResearch Let's discuss the future of AI reliability! 💬🔧

What makes LLM memorize things? Large language models (LLMs) occasionally demonstrate impressive recall from their training data, even without retrieval-augmented generation (RAG). This sparked my curiosity — how often does this happen, and what factors contribute to it? I've been reading academic research on this topic and will be sharing interesting insights and learnings along the way. This post marks the beginning of a multi-part series where I explore key findings and observations from the research community. Looking forward to engaging discussions as I share what I’ve discovered! https://lnkd.in/gXUnb89K

AppTek.ai | Yingbo (Ringo) Gao presenting “Prompting and Fine-Tuning of Small LLMs for Length-Controllable Telephone Call Summarization” paper by: David Thulke | Yingbo (Ringo) Gao | Rricha Jalota | Christian Dugast | Prof Hermann Ney who is also present at the conference. This paper explores the rapid development of a telephone call summarization system utilizing large language models (LLMs). The approach involves initial experiments with prompting existing LLMs to generate summaries of telephone conversations, followed by the creation of a tailored synthetic training dataset utilizing stronger frontier models ... https://lnkd.in/eRH45cCB

Understanding Emergent Capabilities in LLMs: Lessons from Biological Systems https://ift.tt/H5LSzNt How natural systems fundamental laws help explain AI’s unexpected abilities Image by Michaela. Pixabay.com Note: This article presents key findings from our recent research paper, “A non-ergodic framework for understanding emergent capabilities in Large Language Models” [link1] [link2]. While the paper offers a comprehensive mathematical framework and detailed experimental evidence, this post aims to make these insights accessible to a broader audience. However, how many kinds of sentence are there? Say assertion, question and command? There are countless kinds; countless different kinds of use of all the things we call “signs”, “words”, “sentences”. Furthermore, this diversity is not something fixed, given once for all; but new types of language, new language-games, as we may say, come into existence, and others become obsolete and get forgotten. We can get a rough picture of this from the changes in mathematics. Ludwing Wittgenstein, Philosophical investigations, first. publ. in 1953, 4th ed. 2009, Wiley-Blackwell, UK Motivation I remember the summer of 2006 vividly. As I sat at my desk, surrounded by stacks of papers on innovation management theory and the logic of invention, my research had hit a wall. The econometric frameworks for understanding how innovation emerges in organizations felt mechanical and out-of-date. I was missing something essential about how new possibilities arise from existing capabilities. That’s when I first found Stuart Kauffman’s book ‘Investigations’ ¹. Its ambitious subtitle—"The Nature of autonomous agents and the worlds they mutually create"—is both appealing and spooky. When I finally began reading, I found myself drawn into Kauffman’s exploration of how complex biological systems perpetually generate novelty. His central idea, the “adjacent possible,” described how each new innovation in biological systems opens up new possibilities, creating an ever-expanding space of potential futures. The idea wouldn’t let me go. Kauffman’s mathematical framework for understanding how biological systems explore and expand their possibility spaces through restricted combinations of existing elements rather than random search felt profound. It opened a path to think about innovation not as a linear process but as an organic expansion of possibilities. Fast forward to 2024. The rise of increasingly sophisticated large language models such as GPT 4o or Claude 3.5. Current research primarily emphasizes empirical observations and scaling laws—regularities found in the empirical analysis of LLM as well as in many real-world complex systems describing how system’s properties change with size. But prevailing approaches still struggle to explain the real large language model’s behavior. Fulfilling the promises of superintelligence and AGI will require a clear response to the question of how edge...

Here’s a blogpost that breaks down our work on Attribute Controlled Fine-tuning for Large Language Models . We introduced a new method that trains an auxiliary model to control a specific attribute (in this case toxicity). The approach regularizes the LLM's training by penalizing deviations from the desired (non-toxic) distribution, using the auxiliary model trained alongside the core LLM. This work, published at #EMNLP2024, was led by our intern Tao Meng (UCLA) together with our collaborators Ninareh Mehrabi, Palash Goyal, PhD, Anil Ramakrishna, PhD, Aram Galstyan, Richard Zemel, Kai-Wei Chang and Rahul Gupta. Blogpost: https://lnkd.in/eRzvKSKi Paper: https://lnkd.in/eHNwgWAt

💥💥💥 Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models Yuda Song, Hanlin Zhang, Carson Eisenach, Sham Kakade, Dean Foster, Udaya Ghai Abstract Self-improvement is a mechanism in Large Language Model (LLM) pre-training, post-training and test-time inference. We explore a framework where the model verifies its own outputs, filters or reweights data based on this verification, and distills the filtered data. Despite several empirical successes, a fundamental understanding is still lacking. In this work, we initiate a comprehensive, modular and controlled study on LLM self-improvement. We provide a mathematical formulation for self-improvement, which is largely governed by a quantity which we formalize as the generation-verification gap. Through experiments with various model families and tasks, we discover a scaling phenomenon of self-improvement -- a variant of the generation-verification gap scales monotonically with the model pre-training flops. We also examine when self-improvement is possible, an iterative self-improvement procedure, and ways to improve its performance. Our findings not only advance understanding of LLM self-improvement with practical implications, but also open numerous avenues for future research into its capabilities and boundaries. 👉 https://lnkd.in/dy4aGHdk #machinelearning

Mike Dillinger, PhD has done it again. Explained a complex notion in a few well-crafted paragraphs. He looks at the widespread reluctance towards adding the power of knowledge graphs to large language models (LLMs) through the lens of the unfounded opposition between allegedly "continuous knowledge" (for LLMs) and "discrete knowledge" (for knowledge graphs). Reality is (much) more nuanced. Quoting from the article: "It is clear that continuous vs discrete does not describe qualitatively different kinds of knowledge but instead different methods for representing it. But we still have the strong intuition that LLMs and knowledge graphs represent different sorts of things. And they do — but the difference is not between continuous and discrete representations. The much more fundamental and important difference is that LLMs represent observed behaviors like strings and pixels, while knowledge graphs represent grounded, not-directly-observable concepts and the relations between them."