Sai Vemprala

🎇 On National Day, I went for a leisurely drive in San Francisco and ended up "stress-testing" a Waymo self-driving car on the road. 🚗 As an autonomous driving practitioner, who wouldn't be curious about the real-time robustness of the cutting-edge Waymo One driving system? While cruising downtown, I accidentally noticed a Waymo car tailgating me. While this isn't unusual for SF citizens, a wild, "evil" idea suddenly hit me: Why not directly "adversarially attack" the world's autonomous driving status quo? I executed an unexpected maneuver—suddenly reversing—to see how it would react. 🌟 The response was stellar! At the moment I reversed, Waymo One honked instantly—quicker than any human could—activated its hazard lights, and backed away to maintain a safe distance. This reckless move on my part served as an edge case to test the algorithm's robustness under extreme conditions and, potentially, could be a challenging training sample to enhance Waymo's future autonomous systems. 💫 Thrilled to personally experience how current cutting-edge autonomous algorithms handle rare driving behaviors—and how stable and safe Level 4 autonomy is in dealing with diverse scenarios. However, it also prompted deep reflection as an AI researcher in this field: 🤔 In an industry with little room for error, how can we ultimately avoid or minimize issues that AI fails to handle? 💡 I believe two research directions are particularly promising for achieving Level 5 level autonomy in future mobility systems: - 1️⃣ Development and deployment of vehicle-to-everything (V2X) cooperative systems (including V2V, V2I, V2P, etc). Our initial studies (e.g., V2X-ViT, ECCV'2022 arxiv.org/abs/2203.10638) show that in scenarios with severe occlusions or noise, such cooperative systems can significantly enhance the robustness of perception systems, thereby eventually improving traffic safety. - 2️⃣ Adversarial scenarios generation (including Sim-to-Real, generative modeling). Research done by my colleagues at UCLA (V2XP-ASG, ICRA'2023 https://lnkd.in/gu5nKVHD) shows that adversarial learning techniques can effectively simulate adversarial scenarios, greatly improving model robustness in complex "corner case" situations. Of course, it's often infeasible to collect such collision scenarios. 👨‍🏫 As a new Assistant Professor of CS at Texas A&M University, I will lead a group focusing on these exciting research directions, which can be a proactive approach to reducing accidents and improving safety for all. 🔥 I look forward to future collaborations with governments, academics, and companies to research and develop data and algorithms that can help enhance the safety of vulnerable road users, especially seniors and children. We envision a people-centered intelligent transportation system in the future. Interested in these topics? Let's connect and discuss further! #AutonomousVehicles #AI #MachineLearning #SmartCities #Transporation #Humanity #Mobility

144

3 Comments

Code available here 👇 to run on Google Colab (no installs needed). https://lnkd.in/ewiqzZNe Use #Python to do Lattice Boltzmann #simulation Check out this blog and code by Philip: https://lnkd.in/eDx55-KG

120

5 Comments

📰 Just read the latest edition of The Compiler, and it's got some intriguing tech developments worth discussing: 1. X is challenging California's AI deepfake labeling law 🏛️ This raises important questions about content moderation at scale. How are you preparing your systems for potential regulatory changes? 2. Skydio secured $170M for autonomous drone tech 🚁 Edge AI and computer vision are advancing rapidly. Are your ML infrastructure teams ready for efficient model deployment across distributed fleets? 3. ABL Space Systems shifts focus to missile defense 🚀 A reminder of the value of adaptable system design in aerospace. How modular is your current infrastructure? Could it be easily repurposed for new markets? These stories highlight the need for flexible, future-proof architectures in our rapidly evolving tech landscape. What's your take? Read the full newsletter here: https://lnkd.in/gc5tMuWv #TechNews #Engineering #AI #Aerospace

4

1 Comment

Our recent paper (with Eliot Nychka) introduces an alternative method for designing celestial navigation systems. Traditional designs often rely on variants of Kalman filtering to manage uncertainty probabilistically. In contrast, we propose an approach grounded in H2/Hinf theory and linear parameter-varying (LPV) systems theory, focusing on the development of LPV estimators that handle norm-bounded uncertainties, with specific applications to navigation in Cislunar space. The paper is available here: https://lnkd.in/gtZP97-P. The key takeaways are: 1. The navigation system's performance and stability are rigorously guaranteed by design. 2. Uncertainty is represented as norm-bounded rather than probabilistic. 3. The design incorporates the full nonlinear dynamics (in LPV form) without any approximations. 4. Sensing requirements and the observer design, ensuring user-specified accuracy, are jointly determined within an integrated convex optimization framework. We gratefully acknowledge the AFOSR DDIP program and Erik Blasch for funding this research (FA9550-22-1-0539). We also extend our thanks to the Sandra and William R. Wheeler '70 Graduate Fellowship in Aerospace Engineering. #AFOSR, #sda, #cislunar, #TAMU, #tamuengineering Texas A&M University Department of Aerospace Engineering

123

3 Comments

As Waymo enters an era of scaling they are making a big show of publishing safety data. But not all is as they want it to seem. Scaling, especially to serve new cities in addition to adding vehicles to the fleet, is where the risk of edge cases to safety is going to be the most challenging. Higher operational speeds (e.g., for airport service) additionally introduce a higher fatality risk to any crash they do have. The safety statements being made by their public and government relations apparatus are misleading. Maybe this will work out, and maybe it won't. But this data is in many ways just a distracting sideshow from the core safety question of how fatalities will turn out. Waymo hopes that will turn out well (and so do I), but it is irresponsible to practice what amounts to transparency theater and make misleading safety statements in the pursuit of fast scale-out. Details here in this week's substack posting:

67

10 Comments

Yet another innovative paper on State Space Models https://lnkd.in/eevz_GiE achieves arbitrarily low memory decay (meaning arbitrarily long memory) by filtering the input sequence with a precomputed (not learned) spectral filter. Recurrent models, such as SSMs, have a general form of x_t = A x_t-1 = A^2 x_t-2 = … = A^t x_0. Because of the power term A^t, they model nonlinearity well. However, this comes with a cost, ||A|| has to be < 1. Otherwise, the model is “explosive” and will grow out of control rather quickly. For any A satisfying ||A|| < 1, A^t goes to 0 very quickly, meaning the effect of x_0 on x_t diminishes rapidly, which is known as “memory decay”. This limitation is why SSMs may not outperform transformers in the needle-in-the-haystack test. That is until this paper. And the trick? View the signal in its waveform. If something occurs repeatedly without decay, it must be a wave. By using a spectral filter that transforms signals from the time domain to the frequency domain, the paper proves that Spectral SSMs can have a memory longer than any given L. And the best part? The spectral filter is precomputed, not trained. Like any other SSMs, a hybrid architecture leads to better performance. Intuitively, this is because some signals can be efficiently expressed in the frequency domain, while some other signals can be efficiently expressed in the time domain, and having both helps. #artificialintelligence #machinelearning #deeplearning

73

3 Comments

My paper on the Cruise robotaxi pedestrian dragging mishap has been updated and is now available in its final form. I'll be presenting this paper at Safecomp in September, but you can see the preprint right now. Thanks to the SafeComp reviewers for their review comments! If you haven't had a chance to look at this paper before, the short takeaway is that there is a lot buried in the external reports that is not obvious from a casual reading. My paper is the result of carefully assembling the pieces to tell a more direct story of the events that unfolded. Title: Anatomy of a Robotaxi Crash: Lessons from the Cruise Pedestrian Dragging Mishap (Safecomp 2024 practical experience report) Abstract: An October 2023 crash between a GM Cruise robotaxi and a pedestrian in San Francisco resulted not only in a severe injury, but also dramatic upheaval at that company that will likely have lasting effects throughout the industry. Is-sues stem not just from the loss events themselves, but also from how Cruise mishandled dealing with their robotaxi dragging a pedestrian under the vehicle after the initial post-crash stop. External investigation reports provide raw material describing the incident and critique the company's response from a regulatory point of view, but exclude safety engineering recommendations from scope. We highlight specific facts and relationships among events by tying together different pieces of the external report material. We then explore safety lessons that might be learned related to: recognizing and responding to nearby mishaps, building an accurate world model of a post-collision scenario, the in-adequacy of a so-called "minimal risk condition" strategy in complex situations, poor organizational discipline in responding to a mishap, overly aggressive post-collision automation choices that made a bad situation worse, and a reluctance to admit to a mishap causing much worse organizational harm down-stream. https://lnkd.in/e5h84-up

105

9 Comments

Is architectural coupling killing the Software Defined Vehicle? We've seen plenty of high-profile struggles and failures in this area. I'll bet some of the problems come from technology changes that have enabled a proliferation of high-coupling/low-cohesion architectures. This applies to both technical systems and the companies that develop them. The car business needs to figure out an architectural approach that works better going forward. Read more in my free substack article. https://lnkd.in/ekqfJxXF

163

39 Comments

Wake up - new scaling laws paper just dropped! "Scaling Laws for Precision" Multiple interesting findings: * Training larger models in lower precision can be compute optimal - i.e. for a given compute budget you want to train larger models in lower precision rather than smaller models in higher precision in order to achieve the optimal loss * Too much pretraining data can actually be harmful as you quantize such models to lower precision for inference. Llama models popularized the data saturation regime (training on many more tokens / param than recommended by Chinchilla scaling laws (20 tok/param)) as their goal was not to train a compute optimal model but instead to optimize for inference. These models are more suceptible to degradations due to aggressive quantization. * When number of params is held fixed, compute optimal precision increases approximately logarithmically with data, i.e. for larger models you probably don't want to train in low precision even if your GPUs/training stack supports it! We're hitting the quantization limits that have historically brought us a lot of speed improvements (going from fp32 -> bf16 -> fp8). Lots more details in the paper: https://lnkd.in/dC3_EVrs --- 🔔 If you're looking to hire best ML/tech talent check out my platform here: https://lnkd.in/duJZ3D9T post a job here: https://lnkd.in/df-Ukm8E

169

4 Comments

I took a few spare minutes to map out the developer's experience within the MathWorks ecosystem. My approach was to analyze the artifacts produced, tools utilized on various platforms, and the MathWorks products required. It's a complex task... what am I missing here? VersionBay

24

3 Comments

Before moving further with more of TensorRT implementation, let's take a pause and learn a little about CUDA flag architectures for computation... I'm excited to share my latest Medium article diving deep into CUDA flag architectures! This piece builds upon my previous guides on TensorRT and CUDA: 1. "How to Install TensorRT 8.6 with CUDA 12.2 on Linux Mint 21.3" 2. "Building a TensorRT Engine from ONNX Model on Linux Mint" In this new article, I explore: • What CUDA flag architectures are and why they matter • Key differences between architectures like Volta and Ampere • How to optimize your code for specific GPU generations • Practical examples of matrix multiplication across different architectures Whether you've followed my TensorRT installation guide or learned to build TensorRT engines from ONNX models, this new piece will help you understand the underlying GPU architectures that power these tools. Ready to take your GPU programming skills to the next level? Check out my latest article here: [https://lnkd.in/ggnSBKSS] What's your experience with CUDA and different GPU architectures? Share your thoughts in the comments! #CUDA #GPUProgramming #TensorRT #DeepLearning #ParallelComputing

12

𝐄𝐦𝐩𝐡𝐚𝐬𝐢𝐳𝐢𝐧𝐠 𝐒𝐢𝐦𝐩𝐥𝐢𝐜𝐢𝐭𝐲. I've said, half joking, "with every model deployed to production, someone learns to value simplicity more." A simple model: 🔹predicts faster. 🔹loads faster. 🔹has less surface area to evaluate. 🔹creates fewer bugs. 🔹is easier to debug. 🔹is easier to communicate and store. 🔹is easier to interpret. 🔹can be less sensitive to feature drift. After reading a textbook demonstrating the fanciest of machine learning techniques, it's tempting to use them wherever they seem to fit. Do not. You should choose the model that is acceptably performant, and is otherwise optimized for engineering needs. The business may not be especially sensitivity to model performance beyond a point, and it is certainly sensitive to model failures. To really feel this, I'd check out chapters 6-9 of 𝐃𝐞𝐬𝐢𝐠𝐧𝐢𝐧𝐠 𝐌𝐚𝐜𝐡𝐢𝐧𝐞 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐒𝐲𝐬𝐭𝐞𝐦𝐬 by Chip Huyen (https://lnkd.in/giu2Dw-b). It spells out everything involving in managing an effective ML system. Now ask yourself.. 𝑯𝒐𝒘 𝒎𝒖𝒄𝒉 𝒆𝒂𝒔𝒊𝒆𝒓 𝒘𝒊𝒍𝒍 𝒂𝒍𝒍 𝒕𝒉𝒊𝒔 𝒘𝒐𝒓𝒌 𝒃𝒆 𝒘𝒊𝒕𝒉 𝒂 𝒔𝒊𝒎𝒑𝒍𝒆𝒓 𝒎𝒐𝒅𝒆𝒍?

90

6 Comments

🔗 Simulate PyTorch Models with Simulink! 🚀 Engineers, unlock new possibilities by simulating PyTorch and other Python-based models directly in Simulink using co-execution blocks. This integration streamlines your workflow, enabling seamless model testing and validation. 🔍 What's Inside: >> Co-execution of Python models in Simulink << >> Enhanced simulation and validation processes << 👉 Dive into the details: https://spr.ly/6041o1cOX #DeepLearning #Simulink #Python #ModelSimulation #Engineering #MATLAB #MAthWorks #PyTorch

9

🚀 Excited to share my latest project: A ROS2 package for nonlinear model predictive control for drones running PX4 Autopilot! Over the last few weeks, I’ve been working on a ROS2 C++ package that implements a nonlinear model predictive control (NMPC) pipeline for trajectory tracking for PX4-based aerial vehicles. Key Features: •⁠ ⁠NMPC using ACADOS for solving the optimal control problem. •⁠ ⁠Integration with PX4 through the PX4-ROS2 Interface Library by Auterion. •⁠ ⁠External state estimation integration to fuse data from sources like motion capture, visual-inertial, or lidar-inertial odometry with PX4's internal estimation. 🔗 Check out the GitHub repository: https://lnkd.in/eawrvjt3 If you’re exploring UAV control, I’d love for you to try it out! Contributions, feedback, and ideas for improvement are welcome! 🚁 #ROS2 #PX4 #ACADOS #UAV #Autonomous #Control

227

9 Comments

New paper published: Benchmarking the Parallel 1D Heat Equation Solver in Chapel, Charm++, C++, HPX, Go, Julia, Python, Rust, Swift, and Java https://lnkd.in/gCVufi_M

38

8 Comments

Excited to share our latest work, where we demonstrate the first application of SAEs in a real-world use case that outperforms traditional methods!

11

2 Comments

 A pre-print of my summer intern's recent paper, entitled “Domain decomposition-based coupling of operator inference reduced order models via the Schwarz alternating method”, is available on ArXiv here: https://lnkd.in/g8qZ94CA. The paper presents and evaluates an alternating Schwarz-based approach for coupling together subdomain-local reduced order models constructed via non-intrusive Operator Inference (OpInf) with each other and with subdomain-local full order models (FOMs).

53

Berkeley team have developed an enhanced method, called LoRA+, to efficiently fine-tune large language models using Low Rank Adaptation (LoRA). LoRA reduces the number of trainable parameters by introducing low-rank matrices A and B to adapt pretrained weights. However, the standard LoRA setup, which applies the same learning rate to both A and B, leads to suboptimal feature learning for very wide models.     The authors show mathematically that the optimal learning rates for A and B should increase differently with model width. LoRA+ addresses this by using different learning rates for the two matrices, with the learning rate of matrix B being a fixed multiple λ of the learning rate of matrix A. Guidelines provide recommendations for choosing λ based on model architecture and task.   Experiments validated that LoRA+ improves both fine-tuning speed and performance compared to standard LoRA for GPT-2, RoBERTa, and LLaMA models on various benchmarks. Benefits were greater for tasks requiring efficient feature learning to align the model. Integrating LoRA+ is straightforward, requiring only setting the learning rate ratio hyperparameter λ, with a recommended default of λ=16.    This work provides guidance on hyperparameter settings for LoRA, a standard for efficiently specializing large language models. By showing how to "scale up" LoRA for today's massive models, LoRA+ is a useful tool for practitioners wanting to leverage powerful foundation models while controlling costs.    Arxiv: https://lnkd.in/ep6MHyWh

1

Diffusion Transformers just got orders of magnitude more efficient, 25-106x smaller latency when generating 1024x1024 - 4096x4096 images! Few methods they used to achieve this: 1. More aggressive compression in the autoencoder (latent space went from 4x smaller to 32x smaller than the image space) 2. Linear attention instead of "vanilla" attention 3. Decoder-only transformer as encoder (they used Gemma instead of T5 - encoder-decoder transformer) 4. New solver: Flow-DPM-Solver to reduce number of sampling steps excerpt: "Sana-0.6B is very competitive with modern giant diffusion model (e.g. Flux-12B), being 20 times smaller and 100+ times faster in measured throughput. Moreover, Sana-0.6B can be deployed on a 16GB laptop GPU, taking less than 1 second to generate a 1024×1024 resolution image" paper: https://lnkd.in/dkaA__p6

257

8 Comments