Alessandro Delfino

New blog post out by the Bespoke AI team! Hallucination has become a blanket term. This blog post helps clarify and reframe LLM hallucinations in the context of more traditional NLP tasks. We’ve both been thinking this problem for a bit. We’ve been using Provenance-LLM as our primary hallucination detection validator, however we suspect that Minicheck performs better. @bespokelabsai's grounded factuality detector, Bespoke-Minicheck-7B, tops the LLM AggreFact leaderboard. This lightweight model outperforms larger foundation models, including GPT-4 and Mistral-Large 2 on grounded factuality detection. https://lnkd.in/g4tPCAKU

10

Evaluations of LLM applications is a big buzzword right now... but when do you really need new tools? Our latest blog post from Wenzhe Xue cuts through the noise to show with examples how built-in libraries such as pytest are often enough to get started. As complexity grows along the following dimensions, we're here to help! Metric based ➡ Human review Off the shelf eval or LLM as a judge ➡ Custom eval models trained on your data Offline ➡ Online, realtime https://lnkd.in/gGb3Sjah

22

Lourens Walters and I (but mostly Lourens) will present at the Data Science Festival how IIRIS is leveraging GenAI and LLMs to understand B2B audiences and support Informa's marketing teams #innovation #IIRIS #AI #LLMs #datascience

70

7 Comments

I have published two datasets this week. First one is consolidate, select and reformatting of a bunch of top Thai instruct dataset Second is Slim Orca partially(once state of the art dataset) translate to Thai using gpt-4o-mini https://lnkd.in/gq3pG_Ru https://lnkd.in/gMSWi_MT

8

📣 NEW ISSUE IS OUT NOW 📣 In this issue, the team have interviewed some truly pioneering startups within the deeptech industry including: 🚀CELUS 🚀Samphire Neuroscience 🚀PES Technologies 🚀Oxford Quantum Circuits (OQC) 🚀XPANCEO 🚀RobCo 🚀Oshen 🤖 The features of this issue focus on the topic of AI. From learning how to unlock the power of AI and low-code for startups to figuring out whether your startup may be AI washing 💪 Our Founder Focus this issue is the incredible, ⚡️Jordan Marie Brompton, Co-Founder of myenergi Check this out and more here https://lnkd.in/drNnRnX #StartupsMagazine #Startups #Entrepreneur #NewIssue #AI #DeepTech

64

5 Comments

🚀 Startups Magazine feature 🚀 In a recent interview with Startups Magazine, we shared our insights on the topic of AI and deeptech. The new issue is out today, find it here 👉 https://lnkd.in/drNnRnX (£) #QuantumComputing #QuantumStartup

11

We're featured in Startups Magazine, as part of their AI & Deeptech focus! It covers: 1) Why future ocean observing networks will need platforms that are simultaneously: 🏃‍♂️ Rapidly Deployable ⏳ Long Endurance 💷 Low Cost 2) The ebbs and flows: why the most difficult engineering challenges are the simplest sounding, and what combination of skills are needed to make a platform that's wind propelled and collects metocean & environmental data. 3) The Microtransat: A challenge for autonomous sailboats to cross the Atlantic ocean. Started in 2006, the fully autonomous category is yet to see a single successful completion. Key rules: 📏 Vessels must be less than 2.4m in length, 🤖 Fully autonomous (no messages sent once it crosses the start line), 🥅 After travelling 4000nm, must cross a finish line 25 miles in length. #Deeptech #Startups #AI #Robotics #Sailing

20

2 Comments

Expand your explainable-AI toolkit by following along Conor O'Sullivan's latest tutorial, where you'll learn how to apply the H-stat with the artemis package and interpret the pairwise, overall, and unnormalized metrics.

23

Expand your explainable-AI toolkit by following along Conor O'Sullivan's latest tutorial, where you'll learn how to apply the H-stat with the artemis package and interpret the pairwise, overall, and unnormalized metrics.

25

"...embeddings enable machine learning models to find similar objects." Read more of Gregory Hodum's article on embeddings https://lnkd.in/eFs6hhAP #ai #softwaredevelopment #embeddings #machinelearning

9

Our episode on the The Data Exchange Podcast with our fantastic host Ben Lorica 罗瑞卡 , and featuring my great colleague Hima Patel , on data preparation for LLM applications (RAG, fine-tuning, instruct-tuning, and pre-training) using the Data Prep Kit is out! Data Prep Kit is the open-source (Apache 2.0 license) framework and transforms for language and code, which we used in IBM Research to prepare the data for the training of #IBM 's #Granite models ( https://lnkd.in/eUvZVK2t ). Running at any-scale, from a laptop to a datacenter and handling from a few pages of text to hundreds of billions of documents, it is available in pure-#Python , #Ray , #Spark and with #KubeFlow pipelines. Feel free to check it out here: https://lnkd.in/ehGjpbzG #IBMResearch #AI #GenAI #LLM

42

2 Comments

Following the first of our Educational AMAs last week, we are excited to share the latest blog which explains some key terms that were discussed. Expect an introduction to: 🤖 AI Agents 👾 Foundational AI models like #LLMs 👨‍👩‍👦‍👦 Agent Collectives 🤓 An AI Agent #Web3 market analyst? All this and more below 👇 https://lnkd.in/gqREP6ab

6

The centrepiece of my current AI research. Synthetic Reasoning. This is my approach to emulating the human thought process in AI. This ISN’T an agent, as they think independently (hence agent, or agency). Synthetic Reasoning aligns closer to the current objectives of enterprises looking to automate complex, but linear tasks, in robust and predictable way. This approach allows for the offload of repetitive yet cognitively high tasks. Such as review and approval, refinement, information ranking and much more. Tasks that have multiple thoughts associated with them so fall outside of co-pilots but agents don’t adhere to an organisations approach closely enough. Let me know what you think. #ArtificialIntelligence, #SyntheticReasoning, #AIForEnterprise, #CognitiveArchitecture, #GenerativeAI, #AIInnovation, #MachineLearning, #FutureOfWork, #DigitalTransformation, #AIThoughtLeadership, #SystemThinking, #ResponsibleAI https://lnkd.in/ewUi3vAe

43

3 Comments

I got asked a question privately and I want to give it a decent answer “how does Synthetic Reasoning differ from the likes of 4o1 or other natively reasoning models?” Well Synthetic Reasoning is designed to replicate human thought by engineering a controlled series of discrete, manageable steps, or “individual thoughts.” It’s meant to emulate a mix of System 1 (fast, intuitive) and System 2 (slow, deliberate) thinking. By focusing on well crafted individual steps, it avoids cognitive overload, where too many divergent heuristics could overwhelm an LLM. 4o1 and other natively reasoning models have complete independence on how much and what thoughts they use to create an answer. This is great for generalising to a number of different and new tasks but not as powerful for emulating the exact process an organisation wants along with the transparency of how and why they model got to its conclusion. Both approaches have huge value and can be used in concert to great effect. But for control, transparency and accountability Synthetic Reasoning is hard to beat.

9

📄 NEW RESEARCH BLOG! In Simon Prince's latest work, Bayesian Neural Networks, he explores Bayesian learning for neural networks from the parameter perspective. This piece is part VI in a series where machine learning is considered from different viewpoints. 🧑‍💻 You can read the blog now at: https://lnkd.in/gnUhDXiF 🔎 Missed parts I-V? Find them here: https://lnkd.in/gFA2cEC9 #ML #machinelearning #AI #AIinCanada #research #researchtutorial #neuralnetworks #Bayesianmethods

32

Benchmarking Large Language Models: Challenges and Future Directions 📊 In our latest blog post, Giancarlo Mori explores the challenges of establishing benchmarks for large language models (LLMs) and measuring their performance. Key points: 1️⃣ Diversity of tasks and the dynamic nature of language make comprehensive benchmarking difficult 2️⃣ Current evaluation metrics have limitations in capturing nuanced aspects of language 3️⃣ Addressing bias and ensuring scalability are crucial for effective benchmarking Read the full article for insights on future directions and recommendations for improving LLM benchmarking! https://lnkd.in/dNjvZfVz #MVYL #EnterpriseAI #LLM #AI #Substack

1

1 Comment