Hongbo Zhang

Xian Ke, this is incredible! The speed at which you and Vishal Punjabi leveraged crewAI, Streamlit, and various LLM models to create Finbot, a powerful company analysis tool for investors, was mind-blowing. ICYMI, Finbot is here: https://lnkd.in/eSQD69qA I couldn't help but chuckle when I saw the Finbot name - it reminded me of the Wall Street Bets crowd. Maybe you can get them to yolo their life savings based on Finbot's analysis? (Kidding, of course. Or am I? 💎🙌) But in all seriousness, your enthusiasm and passion for AI and LLMs is contagious. It's engineers like you who are pushing the boundaries of what's possible with AI and paving the way for a brighter future.🏆 I'd love to extend an invitation to any other software engineers from the bootcamp to join our Devs Do AI Discord server. It's a great place for engineers without an ML / AI background to connect with like-minded devs, share knowledge, and collaborate on LLM projects. DM me for the invite link. 🤝

13

🌟 Boost Meta's Llama 3.3 70B inference throughput by 3x with Tensor RT-LLM optimizations including speculative decoding, in-flight batching, KV caching, custom FP8 quantization, and more.🦙 Technical deep dive ➡️

8

You want to do Machine Learning on large scale Marketing solutions ? Check below 🚀

3

Sharing my thoughts on How Waymo (formerly known as the Google Self-Driving Car Project) is Pioneering Autonomous Driving with AI. Waymo has become a trailblazer in self-driving technology, and at the core of it all is AI. Leveraging advanced machine learning and deep neural networks, Waymo’s AI-powered system continuously improves through real-world data, making autonomous vehicles smarter, safer, and more reliable. Perception: AI allows Waymo’s cars to "see" and understand their surroundings by processing data from sensors like cameras, LIDAR, and radar. This helps detect pedestrians, other vehicles, and objects in real time. Prediction: The AI system predicts the behavior of everyone around the vehicle—whether it’s cars changing lanes or pedestrians crossing the street—ensuring smooth and safe driving decisions. Planning & Control: AI makes split-second decisions about accelerating, braking, and steering based on the analysis of complex traffic patterns, enabling Waymo vehicles to safely navigate roads. With millions of autonomous miles driven, Waymo’s AI is constantly learning and improving, helping create the future of safer and smarter transportation. The road ahead is AI-driven! www.rxsmeta.com #AI #AutonomousVehicles #Waymo #SelfDrivingCars #MachineLearning #SmartMobility #TransportationTech #FutureOfTransport #AIinAction #Innovation

2

A recent breakthrough titled "Matrix Multiplication-Free LLMs" demonstrates a huge advancement in the area of Large Language Models (LLMs) by reducing computational costs. The authors have eliminated MatMul operations from LLMs, claiming to 10 times reduction in memory usage and a 25.6% increase in training speed, all while maintaining strong performance at billion-parameter scales. Paper link: https://lnkd.in/ggph8qXc #AI #machinelearning #deeplearning #LLMs

15

The @IEEE Computer Society San Diego Chapter is proud o host Professor Nitin J Sanket of robotics engineering department of Worcester Polytechnic Institute for the 11th talk of the 2024 Invited Seminar Series. Professor Sanket will talk about TINY #ROBOT #AUTONOMY, #NAVIGATION, and #INTERACTION. Speaker: Dr. Nitin J Sanket of robotics engineering department of Worcester Polytechnic Institute. Title: FROM MYSTERY TO MASTERY: TINY ROBOT AUTONOMY USING THE MANIFESTATION OF THE UNKNOWN Date: September 25th - 05:30 PM to 06:30 PM PST Venue: Virtual Registration: https://lnkd.in/gWhqGGRA Abstract: The human fascination to mimic ultra-efficient living beings like insects and birds has led to the rise of small #autonomous #robots. Smaller robots are safer, more agile and are task distributable as swarms. One might wonder, why do we not have small robots deployed in the wild today? Smaller robots are constrained by a severe dearth of computation and sensor quality. To further exacerbate the situation, today's mainstream approach for #autonomy on small robots relies on building a 3D map of the scene that is used to plan paths for executing a control algorithm. Such a methodology has severely bounded the potential of small autonomous robots due to the strict distinction between #perception, #planning, and #control. Instead, we re-imagine each agent by drawing inspiration from insects at the bottom of the size and computation spectrum. Specifically, each of our agents comprises a series of hierarchical competencies built on bio-inspired sensorimotor #AI loops by utilizing the action-perception synergy. Here, the agent controls its movement and physical interaction to make up for its lack of computation and sensing. Such an approach imposes additional constraints on the data gathered to solve the problem using Active and Interactive Perception. To unify the class of motion problems, we present a method to exploit the unknown, i.e., the #uncertainty of the predictions to obtain additional informational cues in a new theory called Novel Perception that utilizes the statistics of motion fields to tackle various classes of problems from navigation and interaction. This method has the potential to be the go-to mathematical formulation for tackling the class of motion-field-based problems in robotics and made it into the cover of Science Robotics journal.

17

2 Comments

☎☎☎ WeLore for LLM Compression & PEFT ☎ ☎ ☎ Sharing our recent new work rich with interesting insights: From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Paper Link: https://lnkd.in/eSDUiEdc Code: https://lnkd.in/eViVgCjV Model Checkpoints: https://lnkd.in/euU5GFDw Blogpost: https://lnkd.in/efwGzn4t TL;DR; 1. We establish a consequential relationship between low rank gradient dynamics during pretraining and the emergence of low-rank structures in weight matrices. 2. Low-rank structures in weights emerge non-uniformly across different layers and types of matrices in LLMs - and that is rooted in the inherent optimization difficulty of LLMs. 3. This understanding leads to WeLore, a simple yet effective algorithm that simultaneously addresses LLM weight compression and parameter-efficient fine-tuning (PEFT) in one unified approach. And it works amazingly well in practice! Shiwei Liu Zhenyu (Allen) Zhang Atlas Wang Yuandong Tian #LuYin Jiawei Zhao

48

2 Comments

Last year we released a collection of open-source quantum algorithm software called Qiskit addons. These software packages are designed to enable quantum algorithm developers to explore utility-scale workloads such as those demonstrated by IBM Quantum research teams. Sample-based quantum diagonalization ( 🔗 https://lnkd.in/g2YigRkc): Classically postprocess noisy quantum samples to yield more accurate eigenvalue estimations. Approximate quantum compilation with tensor networks ( 🔗 https://lnkd.in/gxxsuXYH): Compile the initial portion of a quantum circuit into a nearly equivalent approximation of that circuit, but with much fewer layers. Operator backpropagation ( 🔗 https://lnkd.in/ggwcp52H): Reduce circuit depth by trimming operations from its end at the cost of more operator measurements. Multi-product formulas ( 🔗 https://lnkd.in/ggyCy82M): Improve accuracy of simulating quantum dynamics at the cost of increased circuit executions. Check out the Github repos to see the source code and links to the documentation and research papers. Let us know what you think!

105

Excited to share this new breakthrough in ML optimization from my colleagues at Apple: AdEMAMix, an advanced Adam-based optimizer that takes the use of momentum in Stochastic Gradient Descent to the next level. Training large neural networks in fields like computer vision and NLP often requires optimizing complex, non-convex loss functions. Traditional optimizers like Adam and AdamW excel here, but AdEMAMix goes further by using two Exponential Moving Averages: one fast-moving to track recent gradients and another slower-moving to capture early training dynamics. This novel approach improves convergence speed, boosts performance, and significantly slows down model forgetting during training, which enhances model stability and ensures more robust generalization across diverse tasks. Sharing the link in the comments below. #AI #MachineLearning #ComputerVision #LLM #NLP

3,001

64 Comments

Excited to share that our recent work "Static Graph Approximations of Dynamic Contact Networks for Epidemic Forecasting" is published at Scientific Reports. Thanks to my amazing co-authors Razieh Shirzadkhani, Abby L. and Reihaneh Rabbany. In this work, we propose two novel methods, EdgeMST and DegMST, to approximate the structure of dynamic contact networks with a static graph for epidemic forecasting. This is important for forecasting future trends of ongoing epidemics where future contact networks are not observed. Paper Link: https://lnkd.in/ei4bx58P Github Link: https://lnkd.in/ecgZW_xK

66

💡 Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? Christopher highlights some crucial findings from a recent study that caught my attention. While LLMs are powerful, integrating new knowledge through fine-tuning presents some very real challenges. The balance between learning efficiency and avoiding hallucinations is more critical than ever. How do we manage this trade-off at scale in practical AI deployment? 🤔 For #AI professionals, understanding the risks of overtraining and hallucination is key to building reliable, future-proof models. Curious to hear others' thoughts on strategies you've found effective in tackling this challenge!

1

I will be attending IJCAI 2024 in Jeju, South Korea, this week! I’ll be presenting our paper, “Temporal Inductive Logic Reasoning over Hypergraphs,” a collaborative effort with our partners at Georgia Tech, led by Yuan Yang. In this work, we introduce Temporal Inductive Logic Reasoning (TILR), an inductive logic programming (ILP) method designed for temporal hypergraphs. To facilitate hypergraph reasoning, we developed the multi-start random B-walk, a novel graph traversal technique for hypergraphs. By integrating this with a path-consistency algorithm, TILR effectively learns logic rules by generalizing from both temporal and relational data. I look forward to reconnecting with my collaborators and meeting new colleagues. Let's catch up and share our latest research insights! Looking forward to connecting with fellow researchers and exploring the latest advancements in AI! #IJCAI2024 #Cisco #CiscoResearch #CiscoOutshift #GraphReasoning #AI

26

2 Comments

Read our latest published research paper about SuperCoder

27

🌟 2024: The Year of Generative AI Deployment! 🌟   If 2022 was the year the world discovered the potential of generative AI (GenAI) and 2023 marked breakthroughs in large-scale training, then 2024 is set to revolutionize how organizations deploy these powerful systems. Companies are now actively testing GenAI in production, unlocking unprecedented benefits. However, this shift isn't without its challenges—managing complexity and ensuring rapid response times are crucial for a seamless, human-like experience.   🚀 Introducing NVIDIA NIM™   NVIDIA has launched NIM™ inference microservices, designed to streamline GenAI deployment and significantly reduce time to market. Learn more about NIMs here https://lnkd.in/dyFkBfbi.   But what if the specific model you need isn't available as a NIM yet? You can still be extremely performant, and we’ll show you how.   📚 Unlocking Potential with Optimized Inferencing   In our latest blog , we explore how to leverage NVIDIA's cutting-edge inferencing technologies—Triton Inference Server and TensorRT-LLM—to enhance large language model (LLM) inferencing in production. We showcase the DictaLM-2.0 model with a custom tokenizer (which has recently led the Hugging Face Open Leaderboard for Hebrew LLMs) and demonstrate how to create optimized engines for both FP16 and quantized INT4 versions, all served through Triton Server. This setup allows for efficient interaction with the LLM, delivering streaming outputs with minimal latency.   🔗 Dive Deeper!   Check out our blog and explore the accompanying Jupyter notebook that details each step of the process. We’d love for you to try it out and share your feedback! Blog - https://lnkd.in/d68QYepb Unofficial Jupyter notebook - https://lnkd.in/digcgqQh   Big shoutout to the incredible team behind this initiative and the guys over at Dicta for the amazing collaboration: Shaltiel Shmidman Amit Bleiweiss Zvi Lapp   #GenerativeAI #MachineLearning #NVIDIA #TritonInferenceServer #TensorRT #LLM

18

1 Comment

Just did a keynote in PyTorch conference, on LLM deployment. Thanks for the great work of PyTorch team, our partners, and the interests from the audience! Please try out our deployment ecosystem of torchchat, torch.compile, torch.export, AOT Inductor, torchao and ExecuTorch, and checkout all the amazing talks, posters and demos in the conference!

50

3 Comments

Starting off my academic year with a relatively small project, for CS335 at California State University, Fullerton. With my team (Vladyslav Korenevskyi and Vishakha Sanger) we developed an algorithm visualizer using Python, focusing on 4 implementations of Bubble, Merge, Quick and Radix sort, along with a Linear Searching Algorithm. We utilized the 'tkinter' library for the UI, as well as 'matplotlib' to visualize the algorithms into side-by-side graphs. We also used 'time' to measure time and 'multiprocessing' to make more efficient use of the CPUs, creating parallel processes. The GitHub repository: https://lnkd.in/epGxNYS6

2

I've mentioned this to #investors outside of sports and entertainment. The adoption curve for technology in sports is a hockey stick. Years of pilots in fast paced, high stress environments, but if the application is scoped out properly with the right forces behind it can lead to BIG wins in terms of adoption. I don't know much about AiFi Inc. but this looks to be an awesome example on the adoption for checkout free applications. Well done. For #roboticsINsports check out Hop Robotics

18

1 Comment