Giovanni Ortolani

Unleashing the Future with Physics-Informed Neural Networks in Issue 8 of the Data & AI Magazine Imagine blending the precision of physics with the learning power of neural networks. This isn't sci-fi—it's happening now with Physics-Informed Neural Networks (PINNs). Francesco Gadaleta, Ph.D., founder of Amethix Technologies and Intrepid AI, shares his groundbreaking insights in the latest issue of Data & AI Magazine. With career-spanning roles like Chief Data Officer at Abe AI and a key member of Johnson & Johnson's Advanced Analytics Team, Francesco's expertise is as diverse as it is deep. From healthcare to finance, Francesco's work spans multiple industries, proving the limitless potential of AI. But how do physics and neural networks merge to create something revolutionary? Here's a snapshot: - Traditional physics models = precision but complex equations. - Neural networks = data-driven, adaptable, no need for explicit formulas. - PINNs = The best of both worlds, offering unparalleled accuracy and capability. Think of modelling aircraft dynamics or robotic operations—equations and data come together in ways never before possible. Are you curious to know how PINNs are transforming complex problem-solving and pioneering closed-form solutions? Dive deeper into this innovation by subscribing for FREE to Data & AI Magazine on the link below! https://lnkd.in/emYGNEND #DataandAIMagazine #PINNs

4

🚀 Excited to unveil our latest GitHub project on Constrained Deep Learning! 🧠✨ Dive into how we're integrating domain-specific constraints into neural networks for more reliable AI. From ensuring monotonicity to robustness, discover how we're contributing to the evolution of AI verification and incorporating AI into safety-critical systems. 🛡️💡 #DeepLearning #AI #AIVerification #SafetyCritical

7

2 Comments

"Recently the standard GNN has been combined with ideas from other ML models to develop exciting innovative applications. One such development is the integration of GNN with sequential models — Spatio-Temporal GNN that is able to capture both the temporal and spatial (hence the name) dependences of data, this alone could be applied to a number of challenges/problems in industry/research." #graphneuralnetworks #GNN #research #machinelearning #deeplearning #research #drugdiscovery

37

Can Quantum Computing help improving our ability to train large Neural Networks encoding language models (#LLMs)? This is my take on the subject after reemerging out of a very deep rabbit hole. https://lnkd.in/gXdAKKQw #MachineLearning #QuantumComputing #LLMs #AI #ArtificialIntelligence #InformationGeometry #FisherInformationMatrix #QuantumMachineLearning #GradientDescent

8

Proprietary datasets and the combination of AI and experimental data are the key to build successful companies.

1

Here's my notes and video for the keynote I gave at PyCon Italia ❤️ 🐍 🇮🇹 yesterday, "Closer to the Metal." The post itself is titled "don't worry about LLMs" and that's really what I was trying to get at with this talk. LLMs are a big topic and there is a lot of demand to build them quickly, but the space is very confusing and inflated with hype. How do we remove the hype and use LLMs to build things we need? We focus our attention on one thing at a time and use the practical engineering and machine learning skills we already have.

218

5 Comments

Mixture-of-Agents as an Event-Driven System 🤖☎️ This paper by Wang et al. shows you how to ensemble smaller LLMs to create a system that can outperform state-of-the-art larger models. We’ve implemented this paper in a fully async, event-driven workflow system thanks to Ravi Theja Desetty - treat each “small LLM” as an event-driven step that can process incoming events and respond to it, independently and in parallel. ✅ Take full advantage of processing an entire batch of requests  ✅ Cleaner, readable code LlamaPack: https://lnkd.in/gRuqQGk2 Learn more about workflows: https://lnkd.in/gqWwz7Xs

311

9 Comments

🔍 Fascinating insights from @He Chao on ByteDance approach to topology-aware scheduling for LLM workloads! #KubeConNA 🎯 Key Challenges: • Network bottlenecks over compute • Die-level affinity requirements • RDMA congestion at ToR level • Insufficient K8s topology management • Pod-level scheduling limitations 💡 Notable Solutions: • Modified K8s scheduler • Catalist Agent for resource management • Enhanced NUMA awareness • CCX-level workload alignment • GPU co-location strategies 🛠️ Perfect for: • ML Infrastructure Teams • Platform Engineers • Performance Engineers • Cloud Architects • SRE Teams ⚡️ Impact: • Improved inference performance • Better resource efficiency • Enhanced RDMA communication • Optimized cache management • Reduced network congestion 🎥 Watch the full presentation: https://lnkd.in/gNCaQ28F #MLOps #Kubernetes #CloudNative #AI #CNCF

2

Learn how you can convert a PyTorch model, trained on the web, to a mobile-optimized format. Madhur Zanwar describes the process using a few lines of code. He also discusses his journey and how one approach didn't yield satisfactory results while trying to run it in a React Native app. #machinelearning #pytorch #computervision #edgeml #reactnative #objectdetection #yolov5

14

Sir Demis Hassabis recommended this to me, which makes sense because it’s the most challenging book I’ve ever read… David Deutsch pulls together four strands of science: 🖥️ Computation 🧑‍🔬 Quantum physics 💭 Knowledge 🧬 Evolution And with them he argues for a fabric of reality, woven from these four threads, which goes a long way towards a unifying theory of the universe, life and everything. The four, apparently unrelated, areas of science join together elegantly to compose theories about: time travel, virtual reality, parallel universes, quantum physics, cryptography, time and the end of the universe. He deploys numerous tools throughout the book to explain these ideas: diaglogue, philosophy, examples, illustrations and diagrams. I’m not gonna pretend this was an easy book to read. It really wasn’t and a quite a few of the pages I had to read 3-4 times. But given the complexity of the subjects, David does a good job of making them understandable (as much as is possible) to someone with no scientific or physics background like me. David’s final chapters on the collapse of the universe felt entirely reasonable following the chapters before, in which he proposes an omnipotent and omniscient god-like thing which harnesses the infinite energy of a collapsing universe. I ended the book with the feeling of an expanded mind but a sense of relief. It was a workout for sure. Thx David and Demis.

10

The paper "𝐂𝐚𝐧 𝐋𝐚𝐫𝐠𝐞 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥𝐬 𝐔𝐧𝐥𝐨𝐜𝐤 𝐍𝐨𝐯𝐞𝐥 𝐒𝐜𝐢𝐞𝐧𝐭𝐢𝐟𝐢𝐜 𝐑𝐞𝐬𝐞𝐚𝐫𝐜𝐡 𝐈𝐝𝐞𝐚𝐬?" is making the rounds. On one hand, it supports a totally legitimate use of LLMs. LLMs for generating ideas. By mixing concepts plausibly, they're great at it. If you're scratching your head on something, it's always worth asking an LLM about it. At the least, it'll surface topics you just couldn't recall. On the other hand, some readers have egregiously extrapolated from this, suggesting LLMs can propose research ideas that are like to *work*. That's not at all what's shown here. All that is shown is that the LLM generates ideas that jive with a scientist's intuitions. There is a massive gulf between ideas that are plausible and ideas that work - that's the job of experimentation and it's what makes science hard. Very little can be said about lessening that gulf with this paper.

25

2 Comments

Exploring Redundancy in Transformers: Recent research on LLMs such as Llama has revealed groundbreaking insights into model optimization. The paper "What Matters in Transformers?" investigates whether all the layers within a transformer architecture contribute equally to model performance. Interestingly, the authors found that you can remove a substantial number of attention layers in these models without significantly affecting their overall performance. 1) Transformers and Layer Removal: The authors experimented with removing different layers within transformer architectures: attention layers, MLP (feed-forward) layers, and entire transformer blocks. Attention Layers: Removing up to half of the attention layers caused almost no noticeable degradation in model performance. 2) MLP Layers: Eliminating MLP layers resulted in a significant performance drop. 3) Entire Transformer Blocks: Removing complete transformer blocks also led to considerable performance loss. One of the most surprising outcomes of this study is that even after deleting 50% of the attention layers in the Llama 2 70B model, there was only a minor 2.4% decrease in model benchmarks, all while speeding up the model by 48%. This opens up new avenues for optimizing transformer models without sacrificing much accuracy. The authors used a cosine-based similarity score to decide which attention layers to remove. If the input and output of a layer were highly similar, they deemed the layer redundant and eliminated it. This strategic removal helped ensure minimal performance loss while boosting computational efficiency. Another notable feature of this method is its simplicity. Layers were removed in a one-shot manner, meaning the authors did not follow an iterative process of gradual removal and retraining. Additionally, there was no need to retrain the model post-removal, although retraining could potentially help recover some of the small performance losses. This approach could also be combined with existing model compression techniques, such as pruning and quantization, to compound efficiency gains. These findings are exciting for researchers looking to optimize LLMs and make them more scalable for real-world applications. However, the paper focuses mainly on academic benchmarks like HellaSwag and MMLU, which assess a model's knowledge and reasoning capabilities. It remains to be seen how well models with reduced attention layers will perform in more conversational and interactive settings, where maintaining high levels of fluency and coherence is crucial. The paper provides a simple but highly effective method to optimize large language models by identifying redundant components. It suggests that there might be significant computational redundancy in transformers, which can be leveraged to enhance speed and efficiency without compromising much on performance. As LLMs continue to scale, these findings may play a crucial role in developing more resource-efficient models.

1

1 Comment

Amazing conversation with Dr. Serrano in today's episode. I'd like to learn more from his book!

1

Exciting update! Google has just released Gemma 2, their latest open AI models. It's all about top performance and efficiency, making advanced AI more accessible. Built for developers and researchers, Gemma 2 integrates seamlessly into your workflows with broad compatibility across major AI frameworks like Hugging Face Transformers, JAX, PyTorch, and TensorFlow. Ready to see what Gemma 2 can do for you? #AI #Gemma2 #TechNews

16