Zhengzhong Tu

📢 Our new paper "Higher-Order GNNs Meet Efficiency: Sparse Sobolev Graph Neural Networks" has been accepted in IEEE Transactions on Signal and Information Processing over Networks! This work introduces sparse Sobolev graph neural networks (S2-GNN) to address a key challenge in scaling up GNNs. While traditional GNNs typically rely on immediate node neighbors, limiting their effectiveness in capturing complex graph relationships, S2-GNN employs higher-order, sparse filters to efficiently handle large graphs. Our key contributions: Spectral insights: We reveal a spectral similarity between higher-order Laplacians and their sparse approximations, which supports the design of high-performing yet efficient GNNs. Architecture: S2-GNN leverages cascades of sparse filters to enable accurate updates in node embeddings without excessive computational costs. Robustness: We provide stability analysis of S2-GNN, demonstrating robustness to graph perturbations. Extensive benchmarking: S2-GNN consistently outperforms or competes with leading GNN methods across diverse datasets and tasks. We hope this work contributes to advancing scalable, high-efficiency GNN architectures for real-world applications. 🔗 Preprint: https://lnkd.in/eAJJ9SAD 🔗 Code: https://lnkd.in/eWK7XZM2 Thanks to my co-authors Aref Einizade, Andjela Todorovic, Jhon Castro-Correa, Mohsen Badiey, Thierry Bouwmans, and Fragkiskos Malliaros. This work was supported by ANR, the French National Research Agency, Office of Naval Research, Hi! PARIS Center - AI for Science, Business & Society, Télécom Paris, Institut Polytechnique de Paris, University of Delaware, La Rochelle Université, CentraleSupélec, Université Paris-Saclay, and Inria. #GraphNeuralNetworks #SparseGNN

64

9 Comments

Predictions based on Machine Learning and Statistics often rely on maximal likelihood, which can trap LLMs in biases inherited from the majority opinions of training data. This problem has been substantially addressed through conditional statistics, allowing for mitigation of biases and preventing the cancellation of minority perspectives. https://lnkd.in/gMXKqwyB

7

Our new paper titled "Log-Scale Quantization in Distributed First-Order Methods: Gradient-based Learning from Distributed Data" is published in IEEE Transactions on Automation Science and Engineering. Thanks to my co-authors Muhammad Ibrahim Qureshi , M. Hossein Khalesi, Hamid R. Rabiee, and Usman Khan. You may read the paper at the following link: https://lnkd.in/diiQkeeT

40

1 Comment

In Shiwei's new paper accepted by #CMAME, we introduce an innovative deep learning model, the Temporal Graph Neural Network-based Simulator (TGNNS), for granular materials. By constraining physical information flow, TGNNS enhances particle dynamics representation and demonstrate exceptional robustness and efficiency. It is rigorously trained and validated with data from a hierarchical multiscale modeling approach that combines the Material Point Method (MPM) and Discrete Element Method (DEM), and operates 100 times faster than traditional numerical simulations using state-of-the-art GPU-based DEMPM. Additionally, TGNNS excels with diverse datasets, even under complex boundary conditions. This new approach advances our understanding of multiscale behaviors in granular materials and paves the way for enhanced physics-based modeling in digital twins. Check out the full paper: https://lnkd.in/dxN8dqNJ #GranularMaterials #DeepLearning #MultiscaleModeling #MPM #DEM #DigitalTwins #TGNNS #Particle

151

8 Comments

In this new paper we explore the use of classical programming techniques in implementing the quantum lattice Boltzmann method in the Intel Quantum SDK. https://lnkd.in/gpDSFe6b

14

LLMs are powerful but prone to 'hallucinations'—false yet plausible outputs. In our #NeurIPS2024 paper, we introduce a compute-efficient method for detecting hallucinations in single responses using hidden states, attention maps, and output probabilities. Our approach achieves >45x speedups over traditional methods while significantly boosting detection accuracy across diverse datasets. Link to the paper: https://lnkd.in/eBWNXRkx Joint work with Gaurang Sriramanan Siddhant Bharti Vinu Sankar Sadasivan Shoumik Saha Priyatham Kattakinda

310

2 Comments

Here is our new paper accepted in Neurocomputing journal. SITU: Stochastic input encoding and weight update thresholding for efficient memristive neural network in-situ training The Analog-to-Digital Converter (ADC) sensing and conductance update are the most power-demanding processes in the in-situ training of memristive neural networks. In this work, we propose a new thresholded weight update method in conjunction with stochastic input encoding to reduce the ADC sensing requirement and weight updates. This leads to better power and area efficiency for simple neural network models developed in situ on memristive crossbars. This is an interesting research work by Xuening Dong and collaboration between University of Toronto, James Cook University , and York University. https://lnkd.in/gY4-GgtC

58

1 Comment

𝐌𝐮𝐬𝐭-𝐫𝐞𝐚𝐝, 𝐟𝐫𝐞𝐞 𝐛𝐨𝐨𝐤 𝐭𝐡𝐚𝐭 𝐩𝐫𝐨𝐯𝐢𝐝𝐞𝐬 𝐚 𝐩𝐚𝐭𝐡𝐰𝐚𝐲 𝐟𝐫𝐨𝐦 𝐆𝐀𝐈 𝐭𝐨 𝐀𝐆𝐈! ​ Yann LeCun has expressed skepticism about the potential of LLMs to achieve Artificial General Intelligence (AGI), citing limitations in memory, planning, and grounding in real-world understanding. However, we argue that 𝗟𝗟𝗠 𝗖𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝘃𝗲 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 (𝗟𝗖𝗜)—particularly with multiple multimodal LLMs working together—offers a promising architecture to address these challenges and progress toward AGI. ​ Two key architectural innovations underlie LCI: ​ 𝐒𝐞𝐩𝐚𝐫𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐔𝐧𝐜𝐨𝐧𝐬𝐜𝐢𝐨𝐮𝐬 𝐏𝐫𝐨𝐜𝐞𝐬𝐬𝐢𝐧𝐠 𝐚𝐧𝐝 𝐂𝐨𝐧𝐬𝐜𝐢𝐨𝐮𝐬 𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠 𝐚𝐧𝐝 𝐏𝐥𝐚𝐧𝐧𝐢𝐧𝐠: Inspired by the human mind’s dual-layer structure, LCI divides its architecture into foundational and adaptive layers. The foundation layer, akin to unconscious processing, leverages extensive datasets to build robust pattern recognition, encoding essential responses much like human instincts. The adaptive layer, mirroring conscious thought, enables rapid adaptation and contextual reasoning. This dual-layer approach allows LLMs to perform complex tasks with foundational knowledge while adjusting to new information on the fly, much as a child learns new concepts within a developed cognitive framework. ​ 𝐃𝐢𝐬𝐭𝐢𝐧𝐜𝐭 𝐑𝐨𝐥𝐞𝐬 𝐟𝐨𝐫 𝐄𝐚𝐜𝐡 𝐋𝐋𝐌-𝐀𝐠𝐞𝐧𝐭: Each LLM-agent is assigned a specific role, such as executive (knowledge processing), legislative (behavior guardrails), and judicial (adapting to cultural norms). ​ Additionally, our validation of the LCI framework has been thoroughly tested through real-world deployment in applications across various domains, including healthcare, sales planning, investment, and debiasing news. These empirical deployments highlight the practical viability and adaptability of the LCI framework, showcasing its ability to enhance reasoning and decision-making through collaborative intelligence. ​ The eleven aphorisms presented in this free book illustrate the philosophical foundation of the LCI framework, backed by empirical studies. Through LCI, we envision a clear pathway to AGI, rooted in collaborative intelligence and flexible, context-aware adaptation. https://lnkd.in/gn74DkNj (free) https://lnkd.in/gszSvfqP ($6.98 printing fee)

9

Last week at ACM MobiCom 2024, my student, Hossein Khalili, presented our latest research on safeguarding vision models against adversarial attacks in autonomous systems. The method, called LightPure, provides a lightweight yet powerful solution for real-time cyber-physical systems vulnerable to adversarial machine-learning attacks. This work was done in collaboration with Ali Payani at CISCO. More information about the paper can be found here: https://lnkd.in/gf3Aq5WK. Details of the code and implementation can be accessed here: https://lnkd.in/gbHhjpDX

128

1 Comment

The optimal design of a structural system poses significant challenges, such as the high dimensionality of random variable space and the cost of computational simulations repeated during optimization. In this new paper, co-authored by Dr. Jungho Kim at UC Berkeley (Kim Jungho; a former Ph.D. student of my research group at Seoul National University) and Prof. Ziqi Wang (Ziqi Wang) at UC Berkeley, we proposed to adaptively train a heteroscedastic Gaussian process based surrogate model described in a low dimensional active subspace for accurate and efficient optimization of structures. The proposed method was successfully demonstrated and tested by three examples: (1) nonlinear mathematical functions of up to 100 random variables, (2) a space truss structure described by 57 random variables, and (3) a steel lattice transmission tower subjected to seismic excitations described by 100 random variables. Congratulations, Dr. Kim!

130

2 Comments

😀 A newly published paper from 😊 IECE Transactions on Intelligent Systematics, 2024, Volume 1, Issue 2 😃 https://lnkd.in/g-c-AxqJ 😁 Modeling Brain Functional Networks Using Graph Neural Networks: A Review and Clinical Application 😄 Abstract The integration of graph neural networks (GNNs) with brain functional network analysis is an emerging field that combines neuroscience and machine learning to enhance our understanding of complex brain dynamics. We first briefly introduce the fundamentals of brain functional networks, followed by an overview of Graph Neural Network principles and architectures. The review then focuses on the applications of these networks and address current challenges in the field, such as the need for interpretable models and effective integration of multi-modal neuroimaging data. We also highlight the potential of GNNs in clinical perimenopausal areas such as perimenopausal depression research, demonstrating the broad applicability of this approach. The review concludes by outlining future research directions, including the development of more sophisticated architectures for large-scale, heterogeneous brain graphs, and the exploration of causal inference in brain networks. By synthesizing recent advances and identifying key research directions, this review aims to summarize the focal points of brain functional network analysis and GNNs, explore the potential of their integration, and provide a reference for advancing this interdisciplinary field.

17

2 Comments

New GAI Textbook Announcement (The textbook is being used by Stanford CS372: AI for precision medicine and psychological disorders) The past two years have been a significant period in the field of AI, during which we have observed substantial strides in generative AI technologies. Having dedicated over 5,000 hours to research and analysis, I have compiled this book to share a collection of findings and hypotheses regarding Large Language Models (LLMs)—detailing their operational mechanisms, and strategies to mitigate biases and reduce instances of hallucination. The central premise is to enhance context awareness and define human objectives with greater precision. By doing so, LLMs can more effectively mirror human linguistic behaviors, utilizing context-specific linguistic features to fulfill intended goals. SocraSynth: Socratic Synthesis with Multiple Large Language Models. https://lnkd.in/giC4474p

25

Join us for an enlightening workshop titled "New Approaches in Robotic Grasping: From Deep Learning, Reinforcement Learning to Graph Neural Network" at the upcoming ICCKE 2024 conference. In this workshop, we present the culmination of six years of intensive research conducted at the Human and Robot Interaction Laboratory, University of Tehran, about Robotics Grasping. I will present this workshop alongside two of my Ph.D. students, Hamed Ghasemi and Hamed Hosseini Ghasemi. Together, we will unravel the complexities of robot grasping from a deep learning perspective, shedding light on cutting-edge techniques, challenges, and future prospects. Date: 19th Nov, 13:00 Online https://lnkd.in/dqcGb8KE https://lnkd.in/dzXKb-ES Abstract: The workshop titled "New Approaches in Robotic Grasping: From Deep Learning, Reinforcement Learning to Graph Neural Networks" delves into the cutting-edge advancements in robotic grasp detection, a critical component for the automation of industrial tasks. Over the past six years, the Human-Robot Interaction Laboratory has focused on developing innovative solutions for grasp detection, utilizing a combination of geometric approaches and intelligent methods. This workshop will showcase the practical applications of these approaches through the development of a 3-DOF Delta robot and an advanced gripper. Attendees will gain insights into the integration of deep learning models for accurate and adaptive grasp detection, the application of reinforcement learning to enhance the decision-making capabilities of robotic systems, and the utilization of graph theory for optimizing grasp strategies and robot movements. In the workshop, we will also discuss how Transformers are revolutionizing robotic grasping and manipulation. These models, like GPT-4V and Gemini, enable complex robotic tasks with minimal training, using just input prompts. We'll review the latest developments in this area, including models like RT-1 and RT-2. By exploring these new approaches, participants will be equipped with the knowledge and tools necessary to advance the field of robotic grasping, contributing to the evolution of more autonomous and efficient robotic systems in industrial settings. Workshop Outlines: 1. Introduction to Robotic Grasping 2. Geometric Methods and Integration with Intelligent Methods 3. Reinforcement Learning (RL) Based Methods for Data Insufficiency 4. Graph Analysis and Modeling for Compatible Problems 5. Popular Simulators Overview 6. Practical Setup and Results Presentation 7. Applications of Grasping Over the Developed Gripper and Delta Robot 8. Latest Researches in Robotic Manipulation #robot #robotics #grasp #deep_learning #RL #reinforcement_learning

320

3 Comments

Thrilled to collaborate with Prof. Gang Xu on accelerating the computation of isogeometric analysis (IGA) for topology-consistent models - a rewarding partnership that began during his visit to Delft in 2016. In this recent paper accepted by Journal of Computational Physics, we present IGA-Graph-Net, an innovative isogeometric analysis framework that leverages Graph Neural Networks and introduces a custom loss function to address Dirichlet boundary conditions, achieving significantly improved accuracy over traditional CNN-based methods for isogeometric analysis on complex geometries. By integrating ResNetV2 and PointTransformer, the framework effectively handles B-spline data and complex boundary conditions, demonstrating enhanced performance on partial differential equation datasets. "IGA-Graph-Net: Isogeometric analysis-reuse method based on graph neural networks for topology-consistent models", Journal of Computational Physics, 2024 (https://lnkd.in/erS5q6ZF) "IGA-Reuse-NET: A deep-learning-based isogeometric analysis-reuse approach with topology-consistent parameterization", Computer Aided Geometric Design (https://lnkd.in/e_WiJzFt) "Isogeometric computation reuse method for complex objects with topology-consistent volumetric parameterization", Computer-Aided Design (https://lnkd.in/eZKDsip4) #isogeometricanalysis #machinelearning

66

1 Comment

Exciting Advances in Vision Language Models! I’m thrilled to share insights from a recent paper titled "Token-Level Detective Reward Model for Large Vision Language Models" by Deqing Fu, Tong Xiao, Rui Wang, Wang Zhu, Pengchuan Zhang, Guan Pang, Robin Jia, and Lawrence Chen. This innovative research addresses key challenges in reward models for multimodal large language models. Key Highlights: - Traditional reward models assign binary feedback to entire texts, limiting their effectiveness in providing nuanced feedback. - In the context of multimodal language models that process both images and text, existing models risk developing biases, leading to a disconnect from image content. - The proposed Token-Level Detective Reward Model (TLDR) offers fine-grained annotations for each text token, enhancing model accuracy and grounding. Methodology: - The authors introduce a perturbation-based approach to generate synthetic hard negatives, allowing for the creation of token-level labels. - TLDR models demonstrate their utility in helping off-the-shelf models self-correct and in evaluating hallucinations in generated content. Impact: - TLDR models can accelerate human annotation processes by 3x, broadening the acquisition of high-quality vision-language data. This work showcases the potential of refined feedback mechanisms in improving the performance and reliability of multimodal models. Congratulations to the team for pushing the boundaries of AI research! #AI #MachineLearning #VisionLanguageModels #Research #Innovation #Meta Aakanksha Tiwari 🪙

18

🌟 Exciting News from the Research! 🌟 I am thrilled to share our latest publication, "Leveraging Multi-Criteria Integer Programming Optimization for Effective Team Formation," now available in the Early Access section of IEEE Xplore - IEEE Transactions on Learning Technologies. https://lnkd.in/gCh_9yWw This paper, a collaborative effort with co-first authors including myself, introduces a novel methodology using multi-criteria integer programming (MCIP) to navigate the complexities of team formation. This innovative approach not only considers multiple criteria simultaneously but also enhances team diversity and minimizes potential conflicts, setting a new standard in team assembly processes. The core innovation lies in our two-stage optimization process: the initial segmentation using a weighted heterogeneous multivariate K-means algorithm followed by a robust surrogate optimization technique. A huge thanks to my co-authors Pallavi Singh, Kevin H. Nguyen, B.S.B.E., Dr. Trung (Tim) Le, and Dr. Wilfrido Moreno, whose insights and dedication have been instrumental in this work. #TeamFormation #Optimization #IntegerProgramming #IEEE #ResearchInnovation

31

Summer 2024 has been 🔥🔥🔥. After 3 ICML and 1 UAI paper accepted and presented in July, and a Runner-Up Best Paper in MIDL, 4x ECCV papers got accepted last week: VISA: Reasoning Video Object Segmentation via Large Language Model, with Haochen Wang SIGMA: Sinkhorn-Guided Masked Video Modeling, with Michael Dorkenwald Mohammadreza Salehi Yuki Asano Cees Snoek GeneralAD: Anomaly Detection Across Domains by Attending to Distorted Features, with Mohammadreza Salehi Yuki Asano Cees Snoek Click Prompt Learning with Optimal Transport for Interactive Segmentation, with Jie Liu Jan-Jakob Sonke Code will soon follow. Amsterdam equals AI 😁

276

4 Comments

Our recent study on how to best deploy LLMs on edge devices with limited resources. We try to find answers to some questions, such as: Would compressed large-scale LLMs work better than small-scale ones without compression? Would RAG work better than fine-tuning? What is the best compression technique? Does downstream task matter? in collaboration with Jinjun Xiong Check out our 50-page paper at https://lnkd.in/gd3yHwT7 #machinelearning #llm #largelanguagemodel #ai

123

I'm happy to share the news that our paper "Context-Based Meta-Reinforcement Learning with Bayesian Nonparametric Models" is now published by IEEE Transactions on Pattern Analysis and Machine Intelligence 🚀. In this work, we introduce a novel deep clustering framework (DPMM-VAE) based on an infinite mixture of Gaussians, which combines the Dirichlet process mixture model and the variational autoencoder, to simultaneously learn task representations and cluster the tasks in a self-adaptive way. Integrating DPMM-VAE into MELTS enables it to adaptively discover the multi-modal structure of the nonparametric task distribution, which previous methods using isotropic Gaussian random variables cannot model. Dive into the details with our open-access paper: https://lnkd.in/drVG6qGr. For a deeper understanding of DPMM-VAE, check out our associated paper on arXiv: https://lnkd.in/dHFwrsx8. Let's revolutionize meta-reinforcement learning together! 💡 #MetaRL #RL #Robotics

27