Ashish Kapoor

Our recent paper (with Eliot Nychka) introduces an alternative method for designing celestial navigation systems. Traditional designs often rely on variants of Kalman filtering to manage uncertainty probabilistically. In contrast, we propose an approach grounded in H2/Hinf theory and linear parameter-varying (LPV) systems theory, focusing on the development of LPV estimators that handle norm-bounded uncertainties, with specific applications to navigation in Cislunar space. The paper is available here: https://lnkd.in/gtZP97-P. The key takeaways are: 1. The navigation system's performance and stability are rigorously guaranteed by design. 2. Uncertainty is represented as norm-bounded rather than probabilistic. 3. The design incorporates the full nonlinear dynamics (in LPV form) without any approximations. 4. Sensing requirements and the observer design, ensuring user-specified accuracy, are jointly determined within an integrated convex optimization framework. We gratefully acknowledge the AFOSR DDIP program and Erik Blasch for funding this research (FA9550-22-1-0539). We also extend our thanks to the Sandra and William R. Wheeler '70 Graduate Fellowship in Aerospace Engineering. #AFOSR, #sda, #cislunar, #TAMU, #tamuengineering Texas A&M University Department of Aerospace Engineering

123

3 Comments

In this paper, Linyu Liu etal’s study the problem of uncertainty estimation and calibration for LLMs. We first formulate the uncertainty estimation problem for LLMs and then propose a supervised approach that takes advantage of the labeled datasets and estimates the uncertainty of the LLMs’ responses. Based on the formulation, they illustrate the difference between the uncertainty estimation for #LLMs and that for standard #ML models and explain why the hidden neurons of the LLMs may contain uncertainty information. They designed an approach which demonstrates the benefits of utilising hidden activations to enhance uncertainty estimation across various tasks and shows robust transferability in out-of-distribution settings. They distinguish the uncertainty estimation task from the uncertainty calibration task and show that a better uncertainty estimation mode leads to a better calibration performance. The method is easy to implement and adaptable to different levels of model accessibility including black box, grey box, and white box. The key is to develop an LLM which provides the user with a level of understanding of the likelihood that the results are good enough to make decisions, or not. https://lnkd.in/eYYHRZxz #Uncertainty #Estimation #LLMs #AI

9

Before moving further with more of TensorRT implementation, let's take a pause and learn a little about CUDA flag architectures for computation... I'm excited to share my latest Medium article diving deep into CUDA flag architectures! This piece builds upon my previous guides on TensorRT and CUDA: 1. "How to Install TensorRT 8.6 with CUDA 12.2 on Linux Mint 21.3" 2. "Building a TensorRT Engine from ONNX Model on Linux Mint" In this new article, I explore: • What CUDA flag architectures are and why they matter • Key differences between architectures like Volta and Ampere • How to optimize your code for specific GPU generations • Practical examples of matrix multiplication across different architectures Whether you've followed my TensorRT installation guide or learned to build TensorRT engines from ONNX models, this new piece will help you understand the underlying GPU architectures that power these tools. Ready to take your GPU programming skills to the next level? Check out my latest article here: [https://lnkd.in/ggnSBKSS] What's your experience with CUDA and different GPU architectures? Share your thoughts in the comments! #CUDA #GPUProgramming #TensorRT #DeepLearning #ParallelComputing

12

LLMs and AI: The article discusses how Large Language Models (LLMs) have transformed natural language processing (NLP) and AI, particularly their reasoning abilities. Reasoning Capabilities: It explains reasoning in LLMs as making logical deductions from given information, highlighting advancements in models like GPT-3 and GPT-4. Top LLMs in 2024: The text lists several LLMs known for their reasoning skills, including GPT-4, Eurus, Dolphin Llama 13B, Gemini, and Claude 3, noting their applications and ongoing research. Unlocking Reasoning: Techniques to enhance LLM reasoning, such as “Chain of Thought” prompting and integrating symbolic reasoning with LLMs, are explored. Also, the article shares several notebooks thoroughly tested in Google Colab to demonstrate the capabilities of the following LLMs: FALCON2, GEMINI, MISTRAL, GPT-4o, CLAUDE3, and LLAMA3. The article emphasizes that while LLMs show promise in reasoning tasks, they could be better and require further research. It also mentions open-source LLMs contributing to AI advancements.

1

ICLR'24:Lie Group Decompositions for Equivariant Neural Networks https://lnkd.in/dkf2wisc Abstract: Invariance and equivariance to geometrical transformations have proven to be very useful inductive biases when training (convolutional) neural network models, especially in the low-data regime.Much work has focused on the case where the symmetry group employed is compact or abelian, or both.Recent work has explored enlarging the class of transformations used to the case of Lie groups, principally through the use of their Lie algebra, as well as the group exponential and logarithm maps.The applicability of such methods to larger transformation groups is limited by the fact that depending on the group of interest G, the exponential map may not be surjective.Further limitations are encountered when G is neither compact nor abelian.Using the structure and geometry of Lie groups and their homogeneous spaces, we present a framework by which it is possible to work with such groups primarily focusing on the Lie groups G=GL+(n,R) and G =SL(n,R), as well as their representation as affine transformations Rn⋊G.Invariant integration as well as a global parametrization is realized by decomposing the "larger" groups into subgroups and submanifolds which can be handled individually.Under this framework, we show how convolution kernels can be parametrized to build models equivariant with respect to affine transformations.We evaluate the robustness and out-of-distribution generalisation capability of our model on the standard affine-invariant benchmark classification task, where we outperform all previous equivariant models as well as all Capsule Network proposals.

3

Human On the Loop Demand Aware Robot Scheduling: A Mixed Reality Based User Study, IEEE International Conference on Robot and Human Interactive Communication (Ro-MAN) 2024 #robotics #mixedreality #multiarmbandit #reinforcementlearning https://lnkd.in/gM7mchke Ajay Kumar Sandula Rajatsurya Madhusudhana Debasish Ghose Scheduling tasks and subtasks for multiple heterogeneous robots poses a significant challenge, particularly in scenarios where human supervision is essential. The complexity is further compounded when considering human factors in the scheduling process, especially in demand-aware task scheduling environments (tasks are generated based on external demand). The primary goal of this research is to understand human decision-making and how it affects their scheduling process, and we undertook two studies. In the first study, we employed a mixed reality based user study to explore how human perception of the scheduling environment influences task scheduling and facilitates personalized resource allocation. Our findings indicate that human task schedulers exhibit enhanced performance when assisted by autonomous agents, compared to scenarios with limited autonomy in robotic systems. To explore the impact of robot planning on human decision-making and its effects on task scheduling and collision-aware scheduling capabilities, We conducted the second study. This study employed a mixed reality-based warehouse environment, where two users controlled different robots with shared objectives. The findings suggest that human operators exhibited improved collision-aware scheduling without compromising their demand-aware scheduling capabilities when visual aids such as collision cones were incorporated.

22

Speaking at the NeurIPS conference in December 2024, Ilya Sutskever, former Chief Scientist of OpenAI, declared the era of "pre-training" AI on massive datasets nearing its end. He envisions a future dominated by agent AI, systems that learn and adapt continuously through interaction with their environment, much like biological organisms. This shift marks a move from passive data consumption to active learning, with AI agents actively exploring and shaping their understanding of the world. Key points: Sutskever predicts current pre-training methods will end due to finite training data, comparing it to the "fossil fuels" of AI Future AI systems will be genuinely "agentic" and capable of reasoning, making them more unpredictable than current models. He draws parallels between AI scaling and evolutionary biology, suggesting the field might discover new approaches beyond current pre-training methods. Driving this transition are several key factors. Firstly, the limitations of pre-training are becoming increasingly apparent. Finite datasets can only take models so far, and access to real-world data is becoming more challenging due to privacy concerns. Secondly, parallelization, inspired by the parallel processing capabilities of the human brain, will allow AI to process information from multiple sources concurrently, enabling more dynamic and sophisticated learning. This is crucial for handling the complexities of real-world interactions and achieving a deeper understanding of the environment. Thirdly, synthetic data will play a pivotal role. As real-world data becomes scarce, synthetic data offers a viable alternative for training AI models on vast, diverse datasets without compromising privacy. This shift towards agent AI has the potential to unlock superintelligence, where AI systems surpass human capabilities in a wide range of domains. Sutskever predicts these future AI systems will be less predictable and more capable of reasoning and independent thought, potentially exhibiting traits associated with consciousness. This opens up exciting possibilities and raises concerns about control and alignment with human values. In essence, Sutskever, as presented at NeurIPS, paints a future where AI transcends its current limitations, evolving from passive learners to active agents that shape their destiny. This new generation of AI, empowered by parallelization and fueled by synthetic data, promises to be more adaptable, independent, and capable of genuine interaction with the world. However, realizing this vision will require careful consideration of the ethical and societal implications of increasingly autonomous and intelligent machines.

1

Are your language models 𝘀𝘁𝘂𝗰𝗸 𝗶𝗻 𝗿𝗶𝗴𝗶𝗱 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀? Here is a new lib that came out over Christmas 🎁 🔧 𝗦𝗺𝗼𝗹𝗮𝗴𝗲𝗻𝘁𝘀 simplifies agentic workflows for LLMs → Code, search, real-world interactions   📈 Using 𝗲𝘅𝗲𝗰𝘂𝘁𝗮𝗯𝗹𝗲 𝗰𝗼𝗱𝗲 𝗮𝗰𝘁𝗶𝗼𝗻𝘀 instead of JSON  → boosts efficiency and flexibility     🤗 it supports various LLMs  → Obviously the HF ecosystem 🔍 Dynamic decision-making  → allows for nuanced user queries. 𝗪𝗵𝗮𝘁 𝗜 𝘁𝗵𝗶𝗻𝗸 1. Can we use it to create datasets? > I haven't seen datasets been generate using Agentic workflows > So maybe I should dig into? > What do you think here? 2. Don't jump to agents if you don't feel you need it. > Exhaust simple approaches first > Only then try fancy 3. Instead, why not building Tools > You may be an expert into something > Or you have a SaaS, work on some cool open-source lib > Create a tool version to integrate agentic workflows Kudos to Aymeric Roucher, Merve Noyan, Thomas Wolf and the HF team for another great lib Blog: https://lnkd.in/eKHAJR-k Code: https://lnkd.in/ekiuTFJb

11

Our #icra2024 paper is now available from #ieee dl. We proposed a new Inverse Reinforcement Learning based method for hand trajectory prediction. Mukund Mitra Gyanig K. Partha Pratim Chakrabarti

18

2 Comments

🚀 Yet another exciting share: "Setting Up NVIDIA Jetson Nano B01 on Linux Mint: A Comprehensive Guide" Are you a Linux Mint user looking to dive into AI and robotics with the NVIDIA Jetson Nano B01? I've just published a detailed guide tailored specifically for you! In this article, you'll learn: • How to install essential tools like Balena Etcher and PuTTY on Linux Mint • Step-by-step instructions for first boot configuration • Tips for headless setup using PuTTY Whether you're a seasoned developer or just starting out, this guide will help you get your Jetson Nano up and running smoothly on Linux Mint. Check out the full article here: https://lnkd.in/gFETDf5A #JetsonNano #LinuxMint #AI #NVIDIA #EmbeddedSystems Let's connect and discuss your experiences with Jetson Nano or Linux Mint. What projects are you working on?

7

🔍 Excited to share my latest technical article on implementing real-time people detection using NVIDIA's PeopleNet model on Jetson devices! In this comprehensive guide, I cover: - Basic model setup using Jetson native libraries - Performance optimization techniques - Advanced detection processing - Implementation best practices This article complements my previous guide on Raspberry Pi Camera V2 integration with Jetson Nano, creating a complete series for computer vision enthusiasts. 🔑 Key highlights: - Achieve ~10 FPS on Jetson Nano - 91.82% detection accuracy with FP16 precision - Real-time processing capabilities - Optimized CUDA acceleration Check out the full article on Medium to start building your own people detection system! #ComputerVision #NVIDIA #DeepLearning #JetsonNano #ArtificialIntelligence https://lnkd.in/gdivsGWw

10

2 Comments

SigLLM = anomaly detection using LLM https://lnkd.in/gvYGFHWh

2

Check out this cool addition to ArXiv, the open source scientific paper repository, which adds comments. Called alphaxiv, you can either change the URL of any arxiv.org paper to alphaxiv.org and see the comments or load a Chrome plug in that automatically adds them. Comes from Stanford student @ahmadrehaan. I hope this is incorporated in regular ArXiv soon!

58

3 Comments

Lie Group Decompositions for Equivariant Neural Networks https://lnkd.in/edWHiHPe Mircea Mironenco, Patrick Forré Abstract: Invariance and equivariance to geometrical transformations have proven to be very useful inductive biases when training (convolutional) neural network models, especially in the low-data regime. Much work has focused on the case where the symmetry group employed is compact or abelian, or both. Recent work has explored enlarging the class of transformations used to the case of Lie groups, principally through the use of their Lie algebra, as well as the group exponential and logarithm maps. The applicability of such methods to larger transformation groups is limited by the fact that depending on the group of interest �, the exponential map may not be surjective. Further limitations are encountered when � is neither compact nor abelian. Using the structure and geometry of Lie groups and their homogeneous spaces, we present a framework by which it is possible to work with such groups primarily focusing on the Lie groups �=GL+(�,�) and �=SL(�,�) , as well as their representation as affine transformations ��⋊�. Invariant integration as well as a global parametrization is realized by decomposing the "larger" groups into subgroups and submanifolds which can be handled individually. Under this framework, we show how convolution kernels can be parametrized to build models equivariant with respect to affine transformations. We evaluate the robustness and out-of-distribution generalisation capability of our model on the standard affine-invariant benchmark classification task, where we outperform all previous equivariant models as well as all Capsule Network proposals.

10

Data Augmentation in High Dimensional Low Sample Size Setting Using a Geometry-Based Variational Autoencoder by Stéphanie Allassonniere (Univ Paris Diderot) Video: https://lnkd.in/euMhbNmk Slides: https://lnkd.in/eSH3Yeei European CaLISTA (https://lnkd.in/eCBuY4zB) Workshop "Geometry-Informed Machine Learning" organized by Thales and Mines Paris - PSL https://lnkd.in/eGAtMgJW Abstract: In this presentation, we propose a new method to perform data augmentation in a reliable way in the High Dimensional Low Sample Size (HDLSS) setting using a geometry-based variational autoencoder (VAE). Our approach combines the proposal of 1) a new VAE model, the latent space of which is modeled as a Riemannian manifold and which combines both Riemannian metric learning and normalizing flows and 2) a new generation scheme which produces more meaningful samples especially in the context of small data sets. The method is tested through a wide experimental study where its robustness to data sets, classifiers and training samples size is stressed. It is also validated on a medical imaging classification task on the challenging ADNI database where a small number of 3D brain magnetic resonance images (MRIs) are considered and augmented using the proposed VAE framework. In each case, the proposed method allows for a significant and reliable gain in the classification metrics. For instance, balanced accuracy jumps from 66.3% to 74.3% for a state-of-the-art convolutional neural network classifier trained with 50 MRIs of cognitively normal (CN) and 50 Alzheimer disease (AD) patients and from 77.7% to 86.3% when trained with 243 CN and 210 AD while improving greatly sensitivity and specificity metrics.

5