Michela Paganini

I am pleased to share that our paper on Jaynes machine is now available online via Open Access: https://lnkd.in/e64_Jgja John Hopfield and Geoffrey Hinton, who are receiving the Physics Nobel next month, looked to physics for inspiration to model neural networks. They developed the Hopfield network and the Boltzmann machine by using the Ising model and energy minimization as their guiding principles. I went in a completely different direction, looked to game theory, and identified potential games as the starting point. In our framework, the neurons and the connections compete in an error-minimization game to increase their effective utilities during training and reach an arbitrage equilibrium. The outcome is a lognormal distribution of weights and neuronal iota. Our paper shows that this is empirically observed in real-life deep neural nets such as the LLMs. The Hopfield model and the Boltzmann machine cannot predict the distribution of connection weights as it is beyond their capabilities. We show that the Hopfield network and the Boltzmann machine are special cases of the Jaynes machine. One can think of the former two as the "ideal gas law" version of neural networks, whereas ours is the "van der Waals equation". Just like the real-world gases are not ideal and behave more like van der Waals gases, the real-world deep neural networks such as VGGnet, BERT, and LLAMA are not Boltzmann machines, but are more like Jaynes machines.

132

14 Comments

Anthropic has unveiled groundbreaking insights into the inner workings of Large Language Models (LLMs) in a recent paper. This pioneering research employs "dictionary learning" to map and manipulate the internal states of these models, offering unprecedented transparency and control. This breakthrough not only enhances our understanding of neural networks but also opens new avenues for ensuring the safety and reliability of AI systems. Read more: https://lnkd.in/eAzYgnKj IdentifAI - Find Origin - #artificialintelligence

236

3 Comments

Gravity might be a universal decoherence mechanism, underlying quantum-to-classical transitions and the emergence of classical behaviors from a fundamentally quantum universe. https://lnkd.in/dijVRFzV

2

I'm now a peer-reviewer at ResearchHub. Looking forward to contributing to this #DeSci project. Thanks to Patrick Joyce Brian Armstrong Jeffrey Koury Arshia Malek and the rest of the folks at ResearchHub and ResearchHub Foundation for setting this community and opportunity up! More info through the link below. Check it out! #blockchain #science https://lnkd.in/ePEg5x3q

66

15 Comments

New GAI Textbook Announcement (The textbook is being used by Stanford CS372: AI for precision medicine and psychological disorders) The past two years have been a significant period in the field of AI, during which we have observed substantial strides in generative AI technologies. Having dedicated over 5,000 hours to research and analysis, I have compiled this book to share a collection of findings and hypotheses regarding Large Language Models (LLMs)—detailing their operational mechanisms, and strategies to mitigate biases and reduce instances of hallucination. The central premise is to enhance context awareness and define human objectives with greater precision. By doing so, LLMs can more effectively mirror human linguistic behaviors, utilizing context-specific linguistic features to fulfill intended goals. SocraSynth: Socratic Synthesis with Multiple Large Language Models. https://lnkd.in/giC4474p

25

How do people (and how can robots) engage in efficient Referring Expression Generation in realistic, large-scale environments? Mark Higger explores in his 2024 #CogSci paper, "GAIA: A Givenness Hierarchy Theoretic Model of Situated Referring Expression Generation"! Traditional models of Referring Expression Generation enable a speaker to select a set of properties to use to refer to an entity, in a way that disambiguates the target with respect to a set of distractors. But clearly we don't typically bother to rule out ALL distractors. In a classroom you can say "the blackboard" without worrying about all the other irrelevant blackboards you know about in the world. How do human speakers do this (and how could robots)? In this work, Mark answers this question by leveraging our the Givenness Hierarchy, identifying the insight that perhaps it is sufficient to only rule out entities with the same (or higher) /cognitive status/ as your target. If your target is activated, perhaps you only need to rule out other activated entities. If your target is familiar, you only need to rule out other familiar entities! Based on this intuition, Mark presents GAIA, a Givenness Hierarchy theoretic REG model. He then preliminarily validates it through a set of case studies. Look out in the future for full statistical validation. :) To learn more, read the paper! https://lnkd.in/g_fU89Fu And you can learn more about our lab's work on computational modeling of the Givenness Hierarchy at http://mirrorlab.mines.edu. #robotics #nlproc #hri #cogsci #nlg

13

Joint work with Aby Philip, Soorya Rethinasamy, and Vincent Russo now published in Quantum - the open journal for quantum science: https://lnkd.in/giwDBREV Title: Schrödinger as a Quantum Programmer: Estimating Entanglement via Steering Popular Summary: Entanglement is a unique feature of quantum mechanics, initially brought to light by Einstein, Podolsky, and Rosen. Determining whether a general quantum state is entangled or not, known as the separability problem, is of fundamental interest and relevant to various fields of physics: condensed matter, quantum gravity, quantum optics, and quantum information science. Motivated by the rapid development of quantum computers, we provide a variational quantum steering algorithm (VQSA) to address the separability problem. The quantum steering effect, discovered by Schrödinger, is central to the design of our algorithm, which is implementable on quantum computers that are available today. We provide benchmarks and simulations of our algorithm and establish a meaningful connection between steering, entanglement, quantum algorithms, and quantum computational complexity theory.

77

4 Comments

I am excited to share that our latest work, “Deep learning resilience inference for complex networked systems," is out in Nature Communications! In this study, we demonstrate how deep learning methods can effectively leverage the increasingly available observational data to extend the idea of resilience inference to real-world complex networked systems. We design a powerful deep learning framework that effectively integrates Transformer and Graph Neural Network architectures, facilitating the learning of Resilience Inference representations across various complex networked systems. #resilience #deeplearning #GraphNeuralNetwork #Transformer #AIforscience Read the article here: https://lnkd.in/egD8yR_k

89

2 Comments

Are you interested in generating images using quantum computing? Check out our new approach: LaSt-QGAN! https://lnkd.in/gMDh-6CP In this work: Latent Style-based Quantum GAN for high-quality Image Generation ✅ We propose a hybrid quantum-classical GAN approach, which we call Latent Style-based Quantum GAN (LaSt-QGAN) that generates large-size, high-quality images. As its name indicates, it embeds the data into a smaller latent space with a classical autoencoder, and trains the style-based quantum GAN in this latent space. ✅ We showcased its application on the standard MNIST, FashionMNIST and Earth observations images, achieving performance comparable to classical GANs, even overpassing in some cases, with similar resources. ✅ Despite concerns about statistical noise from limited measurements, our paper shows that thanks to a post-processing the classical decoder, this impact of noise is attenuated, allowing high-quality image generation even with fewer than 1000 shots. ✅ We addressed the barren plateau problem by showing that a polynomially deep generator circuit can be trained with a small angle initialization, providing a practical solution. In particular, we provide a scaling of the initialization range with respect to the number of qubits to mitigate BP. Overall, our work highlights LaSt-QGAN's potential for practical image generation through empirical experiments and theoretical analysis, paving the way for future applications on larger datasets. Congratulations to our PhD student Su Yeon Chang for her hard work and dedication to this paper and Supanut Thanasilp for his important contribution and support and to our collaborators Sofia Vallecorsa and Bertrand Le Saux #MachineLearning #QuantumComputing #GenerativeModeling #GenerativeAdversarialNetworks CERN Quantum Technology Initiative CERN Computing

63

Ever felt intimidated by the math of the Free Energy Principle? Ever wondered how one can use the Free Energy Principle for modelling human perception? See my talk Active Inference Institute, available on MLDawn: https://lnkd.in/eRv4yhAW #machinelearning #activeinference #deeplearning

31

1 Comment

AI Computational Model based on Human Behavior 🤔 🔬Researchers at MIT and the University of Washington presents a "novel method" that models the irrirational or suboptimal behavior of human or AI agents based on their computational constraints . ✔️Uses of this Framework : 1. To infer someone’s navigation goals from prior routes. 2. To predict players’ subsequent moves in chess matches. 3. To make an AI assistant much more useful. 📍Modeling Methods : 1. Boltzmann model 2. Markov Decision Process. 📌Testing - Test Cases 1. Inferring navigation goals from previous routes. 2. Guessing someone’s communicative intent from their verbal cues. 3. Predicting subsequent moves in human-human chess matches. 🚀Future Progress of this Model : 1. To model the planning process in other domains, such as reinforcement learning (a trial-and-error method commonly used in robotics). 2. For developing more effective AI collaborators. ❓What test cases would you add to test the computational model of human behaviour ? 🔗Paper: https://lnkd.in/gmnXPSYk #ai #artificialintelligence #humanbehaviour #airesearch

27

8 Comments

If you wonder how RNNs can harness noise to perform useful computations 🧠 and how one could analyze a phase transition in random RNNs through a topological field theory approach, check out my recent talk here 👇 https://lnkd.in/gbSPC8Rq

11

Three papers I've been thinking about lately. The first paper examines language modeling as compression, showing how large language models (LLMs) can achieve compression ratios far beyond traditional entropy-based methods by leveraging semantic context. LLMs, in essence, are compression engines. The second is a philosophy paper that rethinks understanding as the process of compressing information into compact, generalizable forms. Understanding, in this sense, is compression. The third brings these two ideas together, suggesting that LLMs rely on inductive reasoning to compress data further than classical methods. Understanding, again, is compression. While their titles do form the premises of a complete logical argument, I'm not fully ready to commit to any such bold conclusions here. So instead, I've simply placed these papers side by side as some, admittedly leading, food for thought. https://lnkd.in/eB8fzW8H https://lnkd.in/eKkf2nuv https://lnkd.in/eKkXu9js

3

5 Comments

A few weeks ago, KyungHyun Cho's blog post on anxiety and frustration at #NeurIPS (https://buff.ly/4fN4Hah) has provided a lot of insight into the recent history of the AI boom in academia and the industry, as well as the angst many PhD students might be facing in today's job market. I'd like to echo two points I find myself agreeing a lot with Kyunghyun and expand a little. 1. Indeed, a lot of AI jobs in the industry will no longer require a PhD, because with scale and production comes processes and reliability requirements that do not require paradigm-shifting changes all the time (which PhD programs train people to aspire to). However, this does not mean a PhD training will make no difference -- critical thinking, independence in problem solving, domain expertise, and vision about strategic technical directions are all valuable traits that takes a serious amount of training to obtain, and something one should be proud of and play to their own strength. 2. Having more roles and players in the AI market does not mean fewer opportunities for PhDs, because the market is still in an upward trend. It does mean, however, that opportunities might be lying in places you hadn't thought about or anticipated when you entered the PhD or were writing the research papers you are the most proud of. While many academic institutions are growing fast in recent years, there is likely still a fierce competition in more popular directions. In the meantime, there is actually a growing demand in the industry of AI talent, including the usual suspects, as well as many non-large-tech companies, including large companies not traditionally known for Internet/software products, AI-adjacent scale-up companies, and AI-native startup companies. Many of these can provide opportunities for exciting exploratory work with reasonable freedom, depending on the scale/focus of the business. Case in point, we are looking to expand the research/foundation models team at Orby AI and are looking for highly motivated researchers and ML/Research engineers. Please reach out if you're interested in learning more!

43

1 Comment

Our paper “Physical encoding improves OOD performance in deep learning materials property prediction” has been published today in Computational Materials Science. https://lnkd.in/ebuUiHTi Our comprehensive experiments showed that using physical (atomic) encoding rather than the widely used one-hot encoding for atoms/elements can significantly improve the OOD performance by increasing the models’ generalization performance, which is especially true for models trained with small datasets.

27

Jorge Murillo presented our paper titled "CDN-Shifter: Leveraging Spatial Workload Shifting to Decarbonize Content Delivery Networks" at the ACM Symposium on Cloud Computing (ACM SoCC) earlier this week. The paper presents a spatial workload scheduling algorithm called CDN-Shifter for greening CDNs. Read it here: https://lnkd.in/eSxCU-Qh Walid Hanafy David Irwin Ramesh Sitaraman #cloudcomputing #decarbornization

46

The idea of #AIDO (AI-driven Digital Organism) is put under close cross-examination and discussion in an intriguing article by Matteo Wong in The Atlantic today. As for any disruptive idea, there are both concurrence and skepticism of the AIDO vision, but I hereby want to share my optimism: no one would have imagined 10 years ago what we now see from #GPTs or #Geminis in terms of simulating human intelligence; and likewise, what we envision for AIDO to simulate biology at all levels is not a fantasy either thanks to even greater advancements in AI architecture, infrastructure, and data for the years to come. Coming along with #AIDO is a concrete and operationalizable engineering roadmap as outlined in https://lnkd.in/d8eZ7mJp, built on a multi-scale foundation model system which we will be implementing in our venture GenBio AI. I look forward to updating soon our next release of the AIDO models. Stay tuned! https://lnkd.in/dr8kxgVH

281

4 Comments

Find out the mathematics behind the many-worlds theory of quantum mechanics in my latest episode of the Quantum Foundations Podcast with Sam Kuypers! Following on from our more accessible discussion of Everettian quantum theory in last week's episode, this time we focus on key aspects of its mathematical implications using quantum information theory. We cover: measurement; observing single outcomes; irreversibility; interaction with the environment; decoherence; the preferred basis problem; and implications for what's happening inside a quantum computer. Hope you enjoy, video link in the comments!

204

9 Comments

My research paper, "Evolutions of Planetary Systems and the Role of Planetary Systems to Determine the Metallicity and Habitability of Exoplanets," has been accepted at PhysicsRN (Physics Research Network) under SSRN! Using machine learning techniques, astrophysical calculations, and a dataset of over 13,000 confirmed/unconfirmed exoplanets from NASA Exoplanet Archive and TESS, my study uncovers: ✅ An innovative method to differentiate between confirmed and unconfirmed exoplanets based on planetary attributes. ✅ A novel equation to calculate stellar metallicity levels. ✅ Predictive insights into stellar metallicity levels that determine the likelihood of habitability in exoplanets. This research provides fresh perspectives on what it takes for a planet to potentially harbor life. I invite you to read the full paper attached below:

7