Selda Kao

RAG using tabular data with function calling - my talk from @pydatalondon is now available on youtube: The talk is scattered with references to Monty Python's Quest for the Holy Grail (tabular data is the holy grail of RAG for Enterprises). https://lnkd.in/dAkSUMiE

13

Refreshing to see other approaches showing promise beyond just RAG (Retrieval-Augmented Generation) with this new paper from BlackRock and NVIDIA. "HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction" introduces a new approach that effectively combines Knowledge Graphs and vector-based retrieval to enhance the accuracy and relevance of information extraction. #DataScience #AI This is particularly helpful when dealing with financial documents such as earnings transcripts, as HybridRAG demonstrates significant improvements in generating precise, contextually relevant answers. By leveraging both structured and unstructured data, it outperforms existing methods, offering a powerful tool for advanced financial analysis. #Finance #MachineLearning #AI Check out paper here https://lnkd.in/eNJwyCyR

14

The next part of my Causal AI series is up on Towards Data Science, this time exploring how Linear Programming can be used to optimise treatment strategies: https://lnkd.in/eb2ErFgi

46

Exciting news! Gemma 2’s advancements in LLMs are impressive—approaching Meta Llama 3's performance is no small feat. The new features like sliding window attention and Grouped-Query Attention (GQA) sound promising for enhanced capabilities. Looking forward to seeing its impact across various applications! 🌟🚀 hashtag#Gemma2 hashtag#AI hashtag#MachineLearning hashtag#DeepLearning hashtag#Google

2

When assessing the performance of Large Language models (LLMs) we see the primary focus being technical benchmarks, such as graduate-level reasoning, coding proficiency, and multilingual math capabilities. These metrics are undeniably important, but only paint part of the picture. The recent performance comparison of models like Claude 3.5 Sonnet, Claude 3 Opus, GPT-4o, and Gemini 1.5 Pro underscores this point. While Claude 3.5 Sonnet excelled in graduate-level reasoning, coding, and multilingual math, it begs the question: How long until we start incorporating soft skills and behavioural traits into our model assessments? Soft Skills: The Human Touch in AI [1] Problem-Solving: Beyond solving complex equations, AI should demonstrate strong analytical and problem-solving skills, much like what we expect from top-tier talent. This involves tackling intricate financial problems and developing innovative solutions. [2] Communication: The ability to communicate technical concepts clearly and effectively to both technical and non-technical stakeholders is crucial. AI models need to bridge the gap between data science and business operations, ensuring everyone is on the same page. [3] Adaptability: As technologies evolve, so should our models. Adaptability in learning new technologies, adjusting to changing environments, and handling multiple tasks simultaneously is a trait we value in human employees and should equally prioritise in AI. [4] Teamwork: Collaboration is key. AI models should facilitate and enhance teamwork, contributing to a positive and productive work environment. They should be designed to integrate seamlessly with human teams, complementing their efforts. [5] Attention to Detail: Accuracy and precision in financial data and software development are non-negotiable. AI must exhibit meticulousness in its operations, ensuring high standards of quality and reliability. Behavioural Traits: Ethical and Proactive AI [1] Motivation and Initiative: Just as we look for candidates who demonstrate a proactive attitude towards learning and self-improvement, our models should show a capacity for continuous improvement and an 'interest' in advancing the fintech landscape. [2] Ethical Judgement: Understanding ethical considerations, such as data privacy, fairness, and regulatory compliance, is paramount. AI models must be designed with these principles in mind, ensuring they operate within ethical boundaries. Integrating these soft skills and behavioural traits into our assessment criteria will provide a more holistic view of an AI model's capabilities. It's not just about excelling in isolated technical tasks but also about how well these models can adapt, communicate, and integrate into broader business contexts. #AI #SoftSkills

13

Large Language Models (LLMs) are prone to generating factually incorrect information when responding to queries that involve real world statistical information. 𝐃𝐚𝐭𝐚𝐆𝐞𝐦𝐦𝐚 by Google are the world’s 𝐟𝐢𝐫𝐬𝐭 𝐨𝐩𝐞𝐧 𝐦𝐨𝐝𝐞𝐥𝐬 designed to help address the challenges of hallucination by integrating LLMs with 𝐃𝐚𝐭𝐚 𝐂𝐨𝐦𝐦𝐨𝐧𝐬, 𝒂 𝒗𝒂𝒔𝒕, 𝒐𝒑𝒆𝒏-𝒔𝒐𝒖𝒓𝒄𝒆 𝒓𝒆𝒑𝒐𝒔𝒊𝒕𝒐𝒓𝒚 𝒐𝒇 𝒑𝒖𝒃𝒍𝒊𝒄 𝒔𝒕𝒂𝒕𝒊𝒔𝒕𝒊𝒄𝒔 𝒇𝒓𝒐𝒎 𝒕𝒓𝒖𝒔𝒕𝒆𝒅 𝒐𝒓𝒈𝒂𝒏𝒊𝒛𝒂𝒕𝒊𝒐𝒏𝒔 𝒍𝒊𝒌𝒆 𝒕𝒉𝒆 𝑼𝒏𝒊𝒕𝒆𝒅 𝑵𝒂𝒕𝒊𝒐𝒏𝒔 (𝑼𝑵), 𝑪𝒆𝒏𝒕𝒆𝒓 𝒇𝒐𝒓 𝑫𝒊𝒔𝒆𝒂𝒔𝒆 𝑪𝒐𝒏𝒕𝒓𝒐𝒍 𝒂𝒏𝒅 𝑷𝒓𝒆𝒗𝒆𝒏𝒕𝒊𝒐𝒏 (𝑪𝑫𝑪) 𝒂𝒏𝒅 𝒈𝒍𝒐𝒃𝒂𝒍 𝒄𝒆𝒏𝒔𝒖𝒔 𝒃𝒖𝒓𝒆𝒂𝒖𝒔. It explores two primary methods: 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐈𝐧𝐭𝐞𝐫𝐥𝐞𝐚𝐯𝐞𝐝 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧 (𝐑𝐈𝐆), where the LLM is trained to produce natural language queries to retrieve data from Data Commons. 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐀𝐮𝐠𝐦𝐞𝐧𝐭𝐞𝐝 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧 (𝐑𝐀𝐆), where relevant data tables are fetched from Data Commons and used to augment the LLM's prompt. #AI #MachineLearning #DataScience #Innovation #DataGemma #AIAccuracy #GenerativeAI

18

1 Comment

In my opinion, the end state of LLMs is a swarm or hive mind intelligence. Here is a recent paper that explores this idea. Introducing MODEL SWARMS: a new approach to adapting large language models (LLMs) using swarm intelligence. Unlike traditional methods, MODEL SWARMS enables tuning-free adaptation, effective even in low-data scenarios with as few as 200 examples. By leveraging a pool of LLM experts working collaboratively, this approach outperforms 12 baselines by up to 21%, revealing new capabilities and enabling models to improve collectively. #AI #SwarmIntelligence #LLM Paper: https://lnkd.in/g_MBwFbc

21

💡 Great insights on fine-tuning LLMs using Sparse Mixture of Experts (MoE)! This method optimizes performance by activating only specific parts of the model, which can significantly enhance efficiency and cost-effectiveness. 📊💸 The study’s focus on training efficacy, runtime metrics, GPU utilization, and cost modeling is crucial for managing cloud expenses and improving resource allocation. 🔍🚀 It’s exciting to see advancements that help us get more out of our compute resources and budget accurately. A valuable read for anyone involved in AI research and development! 🙌 #AI #MachineLearning #LLM #MixtureOfExperts #TechInnovation #CostEfficiency #AIResearch

2

I have been spending a lot of time in the responsible AI space the last few years and wanted to share some tit bits that might be useful to my linked community. I read a lot of interesting things that my community posts so wanted to contribute as well. “Ever Wonder How AI ‘Talks’ Like Us? Here’s the Secret!” Did you know? Large Language Models (LLMs) like GPT are trained on hundreds of billions of words, but they don’t ‘understand’ language like we do. Instead, they predict the next word in a sequence based on patterns learned from vast datasets. What’s fascinating is that despite this predictive nature, LLMs can generate human-like content, hold complex conversations, and even assist in creative problem-solving. And these models can hallucinate, so it’s important for the human to be part of the loop. #AI #MachineLearning #LLM #TechInnovation #HumanInTheLoop If you want to learn more, the below article does a great job. https://lnkd.in/eNwihkhz

12

1 Comment

As a researcher and data scientist, managing data and testing models can be challenging. That's why I switched to ArcticDB, a tool that has transformed my workflow and boosted my productivity. ArcticDB lets me focus on my data and models without distractions. The near-zero learning curve has been a huge plus. If you’re looking to enhance your research productivity, try ArcticDB. If ArcticDB helped my productivity, it could help yours too! Read more about it here: https://lnkd.in/eB_PFKSb

45

4 Comments

𝗖𝗮𝗻 𝗔𝗜 𝗧𝗲𝗹𝗹 𝗦𝘁𝗼𝗿𝗶𝗲𝘀 𝗟𝗶𝗸𝗲 𝗛𝘂𝗺𝗮𝗻𝘀? 𝗘𝗻𝘁𝗲𝗿 𝗠𝗶𝘅𝘁𝘂𝗿𝗲-𝗼𝗳-𝗣𝗲𝗿𝘀𝗼𝗻𝗮𝘀 (𝗠𝗼𝗣) A new strategy called 𝗠𝗶𝘅𝘁𝘂𝗿𝗲-𝗼𝗳-𝗣𝗲𝗿𝘀𝗼𝗻𝗮𝘀 (MoP) is redefining how Large Language Models (LLMs) evaluate stories. By prompting LLMs to adopt personas focused on specific storytelling metrics—like empathy, authenticity, or narrative complexity—MoP improves alignment with human judgment by over 20%. Psychological Depth Scale (PDS): A New Lens for Stories Using the PDS framework, researchers assessed the psychological depth of LLM-generated short stories based on: • Authenticity: Genuine human-like emotions. • Empathy: Evoking connection with characters. • Engagement: Captivating readers. • Emotion Provocation: Stirring strong reactions. • Narrative Complexity: Intricate plots and characters. 𝗞𝗲𝘆 𝗙𝗶𝗻𝗱𝗶𝗻𝗴𝘀 • LLMs rival humans: Models like GPT-4 created stories with depth comparable to human-written ones. • Size isn’t everything: Smaller models (e.g., Llama-2-7B) performed surprisingly well. • Emotional AI: MoP showed that LLMs could reflect diverse, nuanced human perspectives. 𝗪𝗵𝘆 𝗜𝘁 𝗠𝗮𝘁𝘁𝗲𝗿𝘀 This research shifts the focus from technical quality to emotional and intellectual engagement, opening doors for AI in education, entertainment, and even therapy. Can storytelling AI become the next creative frontier? What do you think? Paper Link: https://lnkd.in/ewq23aFA Spotify Episode of this paper: https://lnkd.in/enqEe5TZ #AI #Storytelling #GenerativeAI #PsychologicalDepth #MixtureOfPersonas #LLM #NotebookLM #Innovation

4

Could we actually be running out of reality? As the marginal price of additional fresh data rises (and this will happpen), AI models will start eating their own output as input data. When models train on outputs that are themselves model-generated, they lose grounding in “reality”—original, human-generated content based on genuine experience or observation. As this happens, there’s a risk that AI becomes less reflective of the actual world and more reflective of a synthetic, self-perpetuating bubble. Errors and biases embedded in early generations of model output might get amplified, creating a distorted version of “reality” that compounds with each iteration. The balance between leveraging synthetic data and keeping models anchored in a continually updated, diverse corpus of real-world information will be crucial. Addressing “Model Autophagy Disorder” MAD will likely require both technical safeguards and social solutions to encourage a sustainable flow of genuine information.

5

🔥 New Series Alert: "Machine Learning - No Fluff, No Jargon, Just the Real Deal" 🔥 I’ve just kicked off the first article in my new Machine Learning series, and let me tell you, this isn’t your typical sugar-coated, jargon-filled nonsense. If you’re tired of wading through overly technical explanations without getting to the heart of what ML really is, this is the series for you. In the Introduction article, I lay the groundwork—what Machine Learning is. This piece is all about setting the stage, giving you the foundational knowledge you need to understand the big picture. But here’s the kicker—this is just the beginning. While this intro stays on the theoretical side, in the upcoming articles, I’ll be diving much deeper into the technical aspects. Expect to see hands-on code examples, real-world applications, and the nitty-gritty details that will take your understanding to the next level. https://lnkd.in/dy79PhQK If this resonates with you (or even if it pisses you off), don’t forget to hit that clap button, leave a comment, and follow me for more. The real action is yet to come, and you won’t want to miss it. If you found the article useful or know someone who would benefit from a clear-cut approach to ML, I’d really appreciate it if you could share this post. Let’s spread the knowledge and help more people get up to speed on Machine Learning. #MachineLearning #Tech #DataScience #NoBS #Learning #AI #Coding

9

🔎 No-Data Psychometric Testing works! Check out our recent work (with Nigel Guenole, PhD, Andrew Samo, and Tianjun Sun) on a new methodology called Pseudo-Factor Analysis (PFA). This methodology uses item embeddings from large language models (LLMs) to gauge psychometric properties before data collection. Our initial tests, comparing PFA results with traditional personality scales, have shown positive outcomes. PFA leverages cosine similarities between item text embeddings from LLMs, suggesting that these can stand in for correlations between item responses. We evaluated this method across several types of LLMs and embedding extraction methods to determine its efficacy in identifying measurement models for multidimensional personality measures. Our findings indicate that PFA can effectively identify factor structures and correlate well with empirical measurement model parameters, though factor congruence varies. Additionally, to support broader usage, we've developed a Shiny app that simplifies the adoption of these techniques for users unfamiliar with the underlying technology. Explore the app here: https://lnkd.in/eiYsMxy7 You can access the pre-print and related materials at https://osf.io/3mpzb/, but reach out if you are curious or have some ideas on expanding it! We look forward to further refining PFA and expanding its applications. #Psychometrics #DataScience #MachineLearning #ResearchInnovation

60

7 Comments

An useful read. “AutoML with AutoGluon: Transform Your ML Workflow with Just Four Lines of Code“

1

“Explain options trading in simple terms” while I’m on the tube with no internet 🛜 Here’s a very awkward demo on the London Underground of me using #opensource AI tooling (Ollama and Phi-3) to converse in natural language without an internet connection. Why is this important? As models get smaller, our devices get more powerful and all this compute happens locally (your devices - computers, phones, toasters?) we will begin to see new ways to work with models in new places. Here an example: A education application that could be used on your commute that tailors itself to you as you go. Currently that requires internet but with local on device models you could that anywhere, anytime and for free* Phi-3 is “tiny” and could fit on a phone. Its been done before. Apple and Google want to control the model on our phones but we shall see if this actually happens. An OpenAI phone? Maybe but I doubt it. Anyways just thought I’d try this and share it here for impact. We are moving towards on device AI compute. What a time to be alive.

18

5 Comments

Having led AI and Data Science research, development and strategy throughout my career, I cannot be more pleased to now see AI in mainstream enterprise and social applications. The attached AI adoption curve from McKinsey (figure at left below) shows how AI usage has shot up from 20% (the early AI visionaries) to 72% of companies using AI. GenAI, of course, is the fastest-growing area within AI. While GenAI models have been around for a while, it was mostly unreliable until RLHF turned the tables in 2022 to enable robust outputs from generative architectures aligned to human preferences thereby providing AI the ability to chat with humans and data! See how GenAI has shot up from 33% adoption to 65% adoption in just 1 year!  It gives a sense of accomplishment to the AI R&D community that the years of AI research optimizing neural net design, RL, and gradient-descent, finally has borne fruit.  Diving a bit further, one question is, are there specific sectors that are actively using AI vs others? Based on the McKinsey study (right figure below) the media, tech, and energy sectors have “shifted the most” between 2023 and 2024. Most interestingly, healthcare, life sciences, finance and insurance, consumer goods, and retail have a large portion of users that have not used GenAI actively. This is in part due to the regulatory aspects of finance and healthcare, and the legacy non-digital nature of retail businesses. On the bright side, this implies that the biggest opportunities ahead in the next few years lie in healthcare, finance, insurance, and B2B / B2C retail as these sectors digitize more and AI helps automate internal and client-facing functions in these sectors. Really exciting times ahead as these sectors transform!!!

35

2 Comments

In the paper "Grokked Transformers are Implicit Reasoners," a key takeaway is the potential superiority of adapting Large Language Models (LLMs) to specific domain data through the process of 'grokking.' which may outperform any traditional Retrieval-Augmented Generation (RAG) or prompting-based solutions. The paper highlights how a transformer's parametric memory is inherently linked to more complex reasoning capabilities that even todays SOTA models are lacking. https://lnkd.in/dWMq7w9X

10