AI Technology & Industry Review
By combining State-Space Models (SSMs) for efficient long-range dependency modeling with dense local attention for coherence, and using training strategies like diffusion forcing and frame local attention, researchers from Adobe Research successfully overcome the long-standing challenge of long-term memory in video generation.
In a new paper The Perfect Blend: Redefining RLHF with Mixture of Judges, a research team from Meta GenAI and FAIR developed Constrained Generative Policy Optimization (CGPO), which offers a more structured approach to RLHF, advancing the performance of general-purpose LLMs.
Apple introduces Depth Pro, a state-of-the-art foundation model designed for zero-shot metric monocular depth estimation. This model can generate high-resolution depth maps with exceptional clarity and fine detail, producing a 2.25-megapixel depth map in just 0.3 seconds on a standard GPU.
A joint research team from Meta and the University of Illinois Urbana-Champaign introduces CrossEval, a benchmark designed to assess both individual and cross capabilities. Their findings demonstrate that LLMs often adhere to the “Law of the Weakest Link”—where performance on complex tasks is limited by the weakest capability.
In a new paper Zero-shot Cross-lingual Voice Transfer for TTS, a Google research team presents a new VT module that seamlessly integrates into a multilingual TTS system, enabling voice transfer across languages.
In a new paper FineZip : Pushing the Limits of Large Language Models for Practical Lossless Text Compression, a research team from UC Berkeley and NYU introduces FineZip, a novel LLM-based compression system designed to significantly reduce compression time.
On September 24, ByteDance’s technology arm, Volcano Engine, introduced two state-of-the-art video generation models, PixelDance and Seaweed, which significantly enhanceContinue Reading
A Microsoft Research Asia research team introduces MarS, a financial market simulation engine powered by a Large Market Model, which addresses the unique demands of modeling the market impact of orders while enabling highly realistic, controllable simulations.
A research team presents SciAgents which aims to automate the process of scientific discovery by revealing hidden interdisciplinary relationships that traditional research methods often overlook. SciAgents operates on a scale, precision, and exploratory power that far surpasses human-driven approaches.
A Sandford U’s research team introduces an experimental framework aimed at evaluating LLMs’ ability to generate research ideas. This study, the first of its kind, compares the ideation capabilities of over 100 expert NLP researchers against an LLM-based ideation system.
A Salesforce AI Research team presents the xLAM series, a collection of large action models designed to enhance the performance of open-source LLMs for autonomous AI agents. This work aims to accelerate innovation in the field and make high-performance models for agent tasks more accessible.
A research team introduces TinyAgent, a framework designed to train and deploy small, task-specific language models capable of performing function calls for agentic systems at the edge, which outperforms larger models such as GPT-4-Turbo in this specific function-calling ability.
A Microsoft research team introduces the Fully Pipelined Distributed Transformer, which leverages the multiple memory hierarchies available in modern GPU clusters, enhancing hardware efficiency and cost-effectiveness while achieving exceptionally high Model FLOPs Utilization (MFU).
In a new paper Diffusion Models Are Real-Time Game Engines, a Google research team presents GameNGen, the first game engine powered entirely by a neural model that enables real-time interaction with complex environments over extended sequences, maintaining high-quality output.
A research team from Samsung makes a first attempt to facilitate LLM deployment on edge devices using integer-only quantization. The proposed MobileQuant, is a post-training quantization technique that reduces both inference latency and energy consumption while preserving accuracy comparable to those achieved with 16-bit activations.
A research team presents GPUDrive, a GPU-accelerated multi-agent simulator built on the Madrona Game Engine, which is capable of generating over a million experience steps per second, making it a game-changer for applying sample-inefficient yet powerful reinforcement learning algorithms to multi-agent planner design.
In a new paper LLM Pruning and Distillation in Practice: The Minitron Approach, an NVIDIA research team presents the Minitron compression strategy, which effectively produces a robust 4B model from Llama 3.1 8B and a cutting-edge Mistral-NeMo-Minitron-8B model derived from Mistral NeMo 12B.
In a new paper Sapiens: Foundation for Human Vision Models, a Meta research team introduces Sapiens, a suite of models designed to address four core human-centric vision tasks: 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction.
In a new paper Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2, a Google DeepMind research team introduces Gemma Scope, a comprehensive suite of JumpReLU SAEs.
A Snowflake research team presents Arctic-TILT, a model that is specifically engineered for large-scale, cost-effective deployment while also being adaptable to various domains. It achieves state-of-the-art performance on benchmarks for both business and long documents.
An Apple research team introduces the foundation language models developed to power Apple Intelligence features. These models include a ∼3 billion parameter model optimized for efficient on-device performance and a larger server-based model designed for Private Cloud Compute.
In a new paper Achieving Human Level Competitive Robot Table Tennis, a Google DeepMind research team introduces the first robot agent that attains amateur human-level performance in competitive table tennis.
In a new paper Wolf: Captioning Everything with a World Summarization Framework, a research team introduces a novel approach known as the WOrLd summarization Framework (Wolf). This automated captioning framework significantly advances video captioning—both in terms of quality (improved by 55.6%) and similarity (improved by 77.4%)—compared to GPT-4V.
In a new paper 500xCompressor: Generalized Prompt Compression for Large Language Models, a Cambridge U team proposes the 500xCompressor, a method designed to condense extensive natural language contexts into a minimum of just one special token, achieving compression ratios ranging from 6x to 480x.
In a new paper The Llama 3 Herd of Models, a Meta AI research team presents Llama 3, a new set of foundation models for language, delivering competitive performance comparing to state-of-the-art language models such as GPT-4 on a plethora of tasks.
In a new paper PianoMime: Learning a Generalist, Dexterous Piano Player from Internet Demonstrations, a research team introduces PianoMime, a framework for training a robot to play the piano using internet-sourced demonstrations.
In a new paper OpenDevin: An Open Platform for AI Software Developers as Generalist Agents, a research team introduces OpenDevin, an Open Platform for AI Software Developers as Generalist Agents. This community-driven platform supports the development of AI agents that interact with software systems.
A DeepMind research team release PaliGemma, a robust and versatile vision language model with 3 billion parameters. PaliGemma excels in transfer learning across various vision and language tasks, achieving state-of-the-art performance in a multitude of open-world applications.
A research team from Saskatchewan University and Google introduces an innovative unsupervised method for automatic video highlight detection, eliminating the requirements for manual annotations while achieving superior performance compared to previous methods.
A Stanford University research team proposes Hypothetical Minds, builds on recent advancements in LLM-based agents designed for multi-agent environments, aiming to enhance adaptability in competitive, cooperative, and mixed-motive scenarios with concealed information.
A DeepMind research team introduces PEER, a innovative layer design leverages the product key technique for sparse retrieval from an extensive pool of tiny experts (over a million), which unlocks the potential for further scaling transformer models while maintaining computational efficiency.
A research team from Microsoft and University of Surrey introduces MInference (Milliontokens Inference), which employs a sparse calculation approach designed to expedite the pre-filling of long-sequence processing. It can reduce inference latency by up to 10 times on an A100 GPU while preserving accuracy.
In a new paper FACTS About Building Retrieval Augmented Generation-based Chatbots, an NVIDIA research team introduces the FACTS framework, designed to create robust, secure, and enterprise-grade RAG-based chatbots.
A Meta AI research team introduces Meta Large Language Model Compiler, a suite of robust, openly available, pre-trained models is specifically designed for code optimization tasks, aiming to provide a scalable, cost-effective foundation for further research and development in compiler optimization.
In a new paper How to Boost Any Loss Function, a Google research team provides a constructive, formal answer, demonstrating that any loss function can be optimized with boosting.
In a new paper RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Fold, a research team provides insights into how synthetic data affects performance, suggesting that a specific schema can achieve consistent gains over using only positive data, achieving performance by 8× in synthetic data volume.
A research team from University of Illinois Urbana-Champaign introduces HPTSA, a multi-agent system that significantly advances cybersecurity exploits, achieving up to 4.5 times better performance on a benchmark of 15 real-world vulnerabilities compared to previous efforts.
In a new paper Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning, a research team from the University of Oxford and Google DeepMind introduces methods to achieve cultural accumulation in Reinforcement Learning (RL) agents. This research opens new pathways for modeling human culture through artificial systems.
In a new paper SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals, a research team introduces SleepFM, the first attempt at developing a multi-modal contrastive learning (CL) approach for PSG analysis, outperforming baselines in tasks like demographic attribute prediction and sleep stage classification.
In a new paper Proofread: Fixes All Errors with One Tap, a Google research team introduces Proofread, an innovative Gboard feature powered by a server-side LLM. This feature allows for seamless sentence and paragraph corrections with a single tap. Launched on Pixel 8 devices, it benefits thousands of users daily.
“Global Vision, Ideas in Collision, Leading Cutting-Edge Innovations” – The 6th annual BAAI Conference successfully concluded on June 15. Over 200 AI scholars and industry leaders gathered to discuss the trajectories and applications of advanced AI technologies.