Xiyuan Zhang

For teams using PyTorch…

5

Kudos to #Composable_kernel team and please check out the AMD engineer Haocong Wang presents the ROCm Composable Kernel library which allows to write high-performance kernels for machine learning with HIP C++. #Composable_kernel #ROCm

39

1 Comment

Have you ever wondered how live camera feeds achieve seamless synchronization? As a Research Assistant at the Indian Institute of Technology, Kanpur, I tackled this challenge using GStreamer. This robust multimedia framework also serves as a backend for NVIDIA Deepstream. From filters and codecs to encoders and decoders, GStreamer proved indispensable in creating robust real-time video pipelines. In my latest Medium blog (part 3), I dive deeper into GStreamer’s functionalities and how it simplifies complex multimedia processing. If you're curious about leveraging GStreamer for your projects, head to my Medium post for all the technical details. 🔑 Key Takeaways: Understanding how filters (encoders, decoders, muxers, demuxers) process data Leveraging queues and tees for asynchronous processing and stream branching Managing data with buffers and understanding GStreamer's event states This post will help you master the essentials of GStreamer and optimize your media workflows! 🎬💡 Check out the full post for more insights #GStreamer #C++ #MediaProcessing #Coding #Tech #SoftwareDevelopment #OpenSource

20

Last year we released a collection of open-source quantum algorithm software called Qiskit addons. These software packages are designed to enable quantum algorithm developers to explore utility-scale workloads such as those demonstrated by IBM Quantum research teams. Sample-based quantum diagonalization ( 🔗 https://lnkd.in/g2YigRkc): Classically postprocess noisy quantum samples to yield more accurate eigenvalue estimations. Approximate quantum compilation with tensor networks ( 🔗 https://lnkd.in/gxxsuXYH): Compile the initial portion of a quantum circuit into a nearly equivalent approximation of that circuit, but with much fewer layers. Operator backpropagation ( 🔗 https://lnkd.in/ggwcp52H): Reduce circuit depth by trimming operations from its end at the cost of more operator measurements. Multi-product formulas ( 🔗 https://lnkd.in/ggyCy82M): Improve accuracy of simulating quantum dynamics at the cost of increased circuit executions. Check out the Github repos to see the source code and links to the documentation and research papers. Let us know what you think!

105

Open-source AI for evaluating language models Researchers from KAIST AI, LG AI Research, CMU, MIT, Allen Institute for AI, and UIC have introduced Prometheus 2, a groundbreaking open-source evaluator model that assesses language model outputs with accuracy rivaling expensive proprietary solutions like GPT-4. A few key highlights: Prometheus 2 combines models trained on direct assessment and pairwise ranking, enabling it to excel at both common evaluation formats. It leverages the new Preference Collection dataset with 1,000 evaluation criteria to boost performance. Prometheus 2 achieved the highest correlation with human judgments and proprietary models across 8 benchmarks, in some cases reaching 0.90 Pearson correlation and 85%+ accuracy. This helps narrow the gap between open-source and closed-source AI evaluation without sacrificing transparency and control As an AI and Data Strategist, I like that this will make high-quality AI evaluation more accessible and scalable for researchers and practitioners. Some potential strategic impacts: Accelerating AI/ML system development by enabling rapid, reliable evaluation of generated language outputs at lower cost Establishing clearer standards and benchmarks for assessing language model performance as open alternatives mature #ai #machinelearning #nlp #opensource #research #aistrategy Paper - https://lnkd.in/gsnGDwJd

1

Hi everyone, I just published a blog where I uncover common KV Cache optimization techniques for LLMs in their inference stage, requiring no re-training or fine-tuning. Check it out and share any thoughts, feedback or questions! I'd really appreciate it. KV Cache Secrets: Boost LLM Inference Efficiency https://shorturl.at/PiUMC

12

Here's the technical breakdown of the AI Penalty Challenge authored by Alok Pattani (check out this video if you haven't seen the demo yet: https://lnkd.in/gsD3xjJ2) Key Takeaways: ☁️ Firestore for triggering slow-mo video capture on Android phones & uploading to Google Cloud via Firebase Storage. 🧠 Cloud AutoML for training custom object detectors (ball & goal tracking) to calculate speed & accuracy. 🗣️ Gemini Pro 1.5 on Vertex AI for personalized coaching feedback based on multiple data points. 🎨 Imagen on Vertex AI for instant, prompt-driven image editing for cool player pics. More technical insights from the project to come... #AIChallenge #MachineLearning #SportsTech #GoogleCloud #footballNOTsoccer

89

5 Comments

A recent breakthrough titled "Matrix Multiplication-Free LLMs" demonstrates a huge advancement in the area of Large Language Models (LLMs) by reducing computational costs. The authors have eliminated MatMul operations from LLMs, claiming to 10 times reduction in memory usage and a 25.6% increase in training speed, all while maintaining strong performance at billion-parameter scales. Paper link: https://lnkd.in/ggph8qXc #AI #machinelearning #deeplearning #LLMs

15

Discover how Perplexity AI built AI architecture to scale its Llama 3 #LLM deployment to serve millions of inference requests. Key takeaways: ✅ 435M queries/month ✅ 20+ AI models, including Llama 3 (8B, 70B, 405B) ✅ 3x cost reduction

7

I'm happy to share that our paper, "Distributed-Memory Parallel Algorithms for Sparse Matrix and Sparse Tall-and-Skinny Matrix Multiplication," has been accepted for presentation at SC24! (https://lnkd.in/gp6txyc2) Our algorithm achieves 5x performance improvement over 2D and 3D SUMMA (https://lnkd.in/gzUhf2gV). Here are related papers we've previously published: Scalable Node Embedding Algorithms Using Distributed Sparse Matrix Operations (https://lnkd.in/g4ybbybG) Distributed Sparse Random Projection Trees for Constructing K-Nearest Neighbor Graphs (https://lnkd.in/gvuyhmw2)

62

3 Comments

I thought it was very interesting to hear Andrej Karpathy talk at GPU MODE IRL 2024 about llm.c and the conversion of PyTorch code into C/CUDA, as well as the potential implications of this work for future LLM progress. The idea of converting high-level deep learning frameworks like PyTorch into low-level, highly efficient C/CUDA code could drastically improve model performance and execution speed, especially in specialized environments. It will be exciting to see how software engineering (SWE) practices evolve over the coming years as this becomes more widely adopted. One of the most intriguing aspects is the potential for a model to generate a specific kernel for a use case at someone's company in code that has no dependencies and could be fully optimized for that specific task. This capability could not only make models more efficient by reducing overhead but also streamline the workflow for developers, allowing them to focus on higher-level design and problem-solving. Essentially, it introduces a whole new paradigm for how we think about integrating AI into SWE roles. Instead of relying on pre-built libraries or packages, engineers might one day leverage AI models to dynamically create specialized, hyper-optimized code that meets their unique needs, reducing both the time and complexity of development. This shift could also lead to a greater level of automation, where AI models help developers by taking over tasks like performance optimization or low-level code generation, potentially reshaping the way we approach software development as a whole. You can listen to the keynote here: https://lnkd.in/eqJYS8Gb #AI #MachineLearning #DeepLearning #PyTorch #CUDA #SoftwareEngineering #C #LLM

2

Completed another; Amazon's product recommendation systems hands-on experience, incorporating exploratory data analysis, model building, and performance optimization through hyperparameter tuning across four different models. Valuable insights were gained by leveraging userID, productID, ReviewerRating, and TimeStamp to enhance sales conversions using AI-powered Recommendation Systems. The integration of Matrix Factorization with user-user Similarity-based Collaborative Filtering has proven to be a game-changer, ensuring superior precision and accuracy. This hybrid approach effectively addresses challenges related to information overload, enhancing the overall user experience and boosting sales. Recommending an implementation strategy which aligns with the client's objectives, reinforcing their competitive edge in the market. #Amazon #DataScience #RecommendationSystem #Modeling #MatrixFactorization #RankBased #SimilarityBased #ArtifiicalIntelligence #Dataset #MachineLearning #HandsOnProject

6

Sharing my new article on Intel GPU parallelism! 🌟 It features a detailed breakdown of matrix addition using SYCL and PyTorch, highlighting how efficient scheduling and workgroups maximize performance on Intel hardware. 💻✨ #Intel #GPU #PyTorch #ParallelComputing

19

🚀 NVIDIA NIM on Bedrock Marketplace 🚀 We’re thrilled to announce that NVIDIA NIM are now available on Amazon Bedrock Marketplace, SageMaker JumpStart, and AWS Marketplace. With this expansion, developers can now deploy high-performance, NVIDIA-optimized inference solutions for generative AI models across key AWS services, delivering: ✅ Faster AI inference ✅ Lower latency ✅ Cost efficiency From Meta’s Llama 3 to NVIDIA Nemotron and more, over 100 prebuilt NIM microservices are now at your fingertips. Tailored for scalability and secure deployment, NIM brings unmatched performance to models running on EC2, EKS, SageMaker, and Amazon Bedrock. This collaboration between NVIDIA and AWS accelerates innovation across industries, from healthcare to manufacturing, ensuring enterprises can meet the growing demand for generative AI applications. 💡 Ready to unlock the power of NVIDIA NIM on AWS? Explore the catalog, request a developer license, or start a 90-day NVIDIA AI Enterprise trial to experience the difference firsthand. Another great project by this wonderful team! Abhishek Sawarkar Jiahong Liu Kshitiz Gupta Timothy Ma Saurabh Trikande Charlie Huang Rohil Bhargava Let’s power the next wave of AI together! #GenerativeAI #NVIDIA #AWS #Bedrock #AIInference #AWSreInvent2024

44

1 Comment

I remember when this all started. Faris - invited me to catch up at our favorite spot around Blok M. I had no idea he’d just wrapped up a procedure at Pondok Indah Hospital! (He really knows how to add drama to our catch-ups.) We’d been on separate paths and projects since "the winding down project", but there he was, reminding me of our dream to ride this AI wave together again. It was a plan we’d hatched with Sahal Zain & Gurnoor Dhillon when the world of AI was still mostly theoretical and jargon-heavy. Somehow, the conversation continued with Gurnoor Dhillon while he was waiting for his meeting with someone who would eventually become one of CalvinBall’s key early adopters. It felt like everything was coming together, like a series of seemingly random dots connecting to form a bigger picture. Seeing how far we’ve come as a team at CalvinBall Technologies is truly exciting. What started as theoretical discussions has now evolved into practical Gen AI solutions that are solving everyday problems and hold endless possibilities to shape the industry. 👨‍🚀 👩‍🚀 Here’s to pushing boundaries and achieving new milestones together! Gurnoor Dhillon, Sandeep Ramesh, Faris -, Sahal Zain, Eva Zuliya, Ratu Nadiah Khairunnisa 🚀 #GenAI #AIInnovation #TechJourney #ArtificialIntelligence #TechTransformation #CalvinballTech #Calvinballers

50

2 Comments