Matthew Wilkinson, PhD

I am pleased to share that our paper on Jaynes machine is now available online via Open Access: https://lnkd.in/e64_Jgja John Hopfield and Geoffrey Hinton, who are receiving the Physics Nobel next month, looked to physics for inspiration to model neural networks. They developed the Hopfield network and the Boltzmann machine by using the Ising model and energy minimization as their guiding principles. I went in a completely different direction, looked to game theory, and identified potential games as the starting point. In our framework, the neurons and the connections compete in an error-minimization game to increase their effective utilities during training and reach an arbitrage equilibrium. The outcome is a lognormal distribution of weights and neuronal iota. Our paper shows that this is empirically observed in real-life deep neural nets such as the LLMs. The Hopfield model and the Boltzmann machine cannot predict the distribution of connection weights as it is beyond their capabilities. We show that the Hopfield network and the Boltzmann machine are special cases of the Jaynes machine. One can think of the former two as the "ideal gas law" version of neural networks, whereas ours is the "van der Waals equation". Just like the real-world gases are not ideal and behave more like van der Waals gases, the real-world deep neural networks such as VGGnet, BERT, and LLAMA are not Boltzmann machines, but are more like Jaynes machines.

132

14 Comments

Proprietary datasets and the combination of AI and experimental data are the key to build successful companies.

1

It's tonight! Please join the PyData Cambridge community to welcome Mayank P. (Astra Zeneca) with his talk "Image Registration: How to align images" and Ehecatl Antonio del Rio Chanona (Imperial College London) with "Data-driven optimization for expensive problems: a chemical reactor case study". This event is free and in-person only. Please signup on Meetup (link below) and reach out if you have any questions or wish to contribute in any form. We are always looking for speakers, attendees, and funding (company sponsorships, personal donations, etc.). Please also follow us and share this event! - MeetUp (event signup): https://lnkd.in/eT3FRzt6 - LinkedIn: PyData Cambridge - Twitter: https://lnkd.in/eCr9BNT5 - Location (Map): https://lnkd.in/evutQgnY #llm #ai #datascience #dataengineering #cambridgeuk #sponsorship

15

Expand your explainable-AI toolkit by following along Conor O'Sullivan's latest tutorial, where you'll learn how to apply the H-stat with the artemis package and interpret the pairwise, overall, and unnormalized metrics.

23

Expand your explainable-AI toolkit by following along Conor O'Sullivan's latest tutorial, where you'll learn how to apply the H-stat with the artemis package and interpret the pairwise, overall, and unnormalized metrics.

25

🧑‍💻 Don't forget to check out Simon Prince's latest research blog, Bayesian Neural Networks, where he explores Bayesian learning for neural networks from the parameter perspective. The piece is part VI in a series where machine learning is considered from different viewpoints. Read the blog here 🔗 https://lnkd.in/gnUhDXiF Missed Parts I - V? Find them below. Part I 🔗 https://lnkd.in/gxzpKMFb Part II 🔗 https://lnkd.in/g7haGzse Part III 🔗 https://lnkd.in/gbWWj6tZ Part IV 🔗 https://lnkd.in/gUstQyUv Part V 🔗 https://lnkd.in/gQcdE_ds Keep an eye out for the final installment of the series, coming soon. 👀 #ML #machinelearning #AI #AIinCanada #research #researchtutorial #neuralnetworks #Bayesianmethods #neuralnetworks

51

📄 NEW RESEARCH BLOG! In Simon Prince's latest work, Bayesian Neural Networks, he explores Bayesian learning for neural networks from the parameter perspective. This piece is part VI in a series where machine learning is considered from different viewpoints. 🧑‍💻 You can read the blog now at: https://lnkd.in/gnUhDXiF 🔎 Missed parts I-V? Find them here: https://lnkd.in/gFA2cEC9 #ML #machinelearning #AI #AIinCanada #research #researchtutorial #neuralnetworks #Bayesianmethods

33

https://lnkd.in/epNxY4fb I've been thinking about the way that large language models seem to be able to generalise to unseen problems lately and I've been thinking that the spectral bias of neural networks is probably a key component of that. Between this and the double descent problem, it's becoming increasingly clear that massively overparameterized neural networks are biased towards simple and smooth solutions to a problem. Often, the simplest representation of a domain is also the 'correct' one. Consider a dataset that consists exclusively of uniformly sized black dots against a white background. The most compact encoding that contains all the information for this dataset is just the x and y co-ordinates of the dot. These values also happen to be the real generator of the dataset. It’d be reasonable to expect a beta VAE to converge on this representation based on spectral bias and it would be the most generic encoding. Is a similar thing happening in large language models? When presented with conversations and papers about scientific matters where the next word depends on what is true, perhaps the simplest policy is to understand the topic and to model reality first - language second.

3

This clearly explains, how AGI cannot be achieved through LLMs, without understanding of enviornement there is no meaning behind the words inferenced by gpt. https://lnkd.in/dvFqU-BE

7

2 Comments

🚀 Want to employ the power of #PyTorch for your scientific models without a steep learning curve? 👀 Check out our latest blog, where Travis Oliphant discusses how PyTorch's integration with the Array API makes it easy to use with familiar libraries like NumPy and scikit-learn.🔥 https://lnkd.in/dDRkEY3v #Innovation

10

👨‍🔬 🚀 Thrilled to announce the new paper, “Softening Ontological Reasoning with Large Language Models” co-authored by our scientists! 🔗 https://lnkd.in/drX-vKyQ 🔍 Why It Matters: Prometheux data platform enables highly optimised big data queries to fragmented data sources and schemas, showing exactly how it obtains results. 💡This work will advance Prometheux integration with LLMs, further complementing the precision of structured data-processing with the flexibility of language models, empowering organisations to make real-time, fully auditable decisions driven by both structured and unstructured data. #ExplainableAI #knowledgegraphs #AI #neurosymbolic #LLMs

41

5 Comments

🚀 Catch up on the recent advancements in Vision Language Models in our latest blog post. We cover early contrastive approaches like CLIP to more advanced models like Flamingo and LLaVA. 🔗 Link: https://lnkd.in/dbXnb7mX

14

1 Comment

🔢 Exciting new blog from Olly Britton, a #QuansightLabs summer intern! He's added support for multidimensional numerical integration of array-valued functions to SciPy. This fills an important gap in SciPy's integration toolkit! Read his about it here: 👉 https://lnkd.in/dNmHTX3G #OpenSource #Python #SciPy

12

Keynote from Professor Neil Lawrence, Google DeepMind Professor of ML at University of Cambridge, as he presents a keynote: "The Atomic Human" at OpenUK Report Launch, drawing from his extensive research and new book. See the full schedule here https://lnkd.in/etBhY2W4 and watch the video recordings as they are shared. See Report https://lnkd.in/ewjeDY3j All content will be shared digitally after the event at https://lnkd.in/eCKkCQvt #openuk #opensource #theopenmanifesto #stateofopen #opensourceai #opendata #londondataweek #opensourcesoftware

6

Microsoft researchers developed Orca 2, using a technique that allows small language models to achieve performance nearly on par with larger models by following more cautious reasoning strategies. Read our summary of the paper in #TheBatch: https://hubs.ly/Q02BpKy90

429

4 Comments

"...embeddings enable machine learning models to find similar objects." Read more of Gregory Hodum's article on embeddings https://lnkd.in/eFs6hhAP #ai #softwaredevelopment #embeddings #machinelearning

9

Fine-tuning LLMs with Reinforcement Learning from Human Feedback This blog post explores how Reinforcement Learning from Human Feedback (RLHF) can be used to fine-tune Large Language Models (LLMs) to align with human preferences. It delves into the concept of Proximal Policy Optimization (PPO) and how it is used in this context. The key takeaway is that RLHF offers a powerful framework to train LLMs that can produce human-aligned text, while still maintaining creativity. This paves the way for more natural and nuanced language generation in the future of AI and natural language understanding. Read the article: https://lnkd.in/d9EGf579 #AI #machinelearning #goML

8