AI at Meta

🎥 Today we’re excited to premiere Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do ➡️ https://go.fb.me/00mlgt Movie Gen Research Paper ➡️ https://go.fb.me/zfa8wf 🛠️ Movie Gen models and capabilities • Movie Gen Video: A 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. • Movie Gen Audio: A 13B parameter transformer model can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. • Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes. • Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

Llama is giving Sevilla FC an edge in how they scout the next wave of soccer stars. Together with IBM they created Scout Advisor — a generative AI-driven scouting tool designed and built on watsonx, with Llama 3.1. How Sevilla FC is discovering future soccer stars with Llama ➡️ https://go.fb.me/3kbjv4 Scout Advisor helps them analyze and leverage unstructured data from over 300,000 scouting reports with crucial insights that go beyond statistics like goal counts and passing accuracy to signals for attitude, tenacity, leadership and more.

Introducing Aria Gen 2, the next generation of glasses from Meta's Project Aria that we hope will enable researchers across industry and academia to unlock new work in machine perception, egocentric & contextual AI, robotics and more. More details on what Aria Gen 2 can do ➡️ https://go.fb.me/oa6tnx Get updates on device availability ➡️ https://go.fb.me/4h1h8r Highlights for researchers • State-of-the-art sensor suite featuring an RGB camera, 6DOF SLAM cameras, eye tracking cameras, spatial microphones, IMUs, barometer, magnetometer, and GNSS + a new PPG sensor (heart rate) and contact microphone.  • Ultra low-power and on-device machine perception for SLAM, eye tracking, hand tracking, and speech recognition using Meta’s custom silicon. • Capable of six to eight hours of continuous use. • Open-ear force-canceling speakers, enabling user-in-the-loop system prototyping. We’re excited to see how researchers at Meta and beyond will leverage Aria Gen 2 to pave the way for future innovations that will shape the next computing platform.

Meta PARTNR is a large-scale human and robot collaboration benchmark for planning and reasoning in embodied multi-agent tasks. Work like this informs our work as scientists and engineers pushing this important field of study forward. Research paper ➡️ https://go.fb.me/8cwppu Dataset and code ➡️ https://go.fb.me/rld05r

Audiobox Aesthetics is a new model from Meta FAIR for unified automatic quality assessment for speech, music and sound. The model makes predictions that analyze content enjoyment, content usefulness, production complexity and production quality. Model weights and code ➡️ https://go.fb.me/uxx1pg Research paper ➡️ https://go.fb.me/cgt01g Try the demo on Hugging Face ➡️ https://go.fb.me/fq6brx How we're advancing machine intelligence through human-centered research ➡️ https://go.fb.me/uxx1pg Extensive experiments showed that Audiobox Aesthetics outperformed prior works with higher correlation to human judgement, proving its effectiveness as an automatic metric for quality evaluation.

In support of UNESCO's work, we’re inviting collaborators to join us in democratizing language technology and building more inclusive AI systems with the Language Technology Partner Program. This work is part our our long-term commitment to supporting under-served languages. More details on this program and how you can apply ➡️ https://go.fb.me/rs48dc

A closer look at the evolution of the robotics research at Meta FAIR that led to our newest open source releases. We're excited about how work like this will support new possibilities for the future of human-robot collaboration. More details on Meta PARTNR, including links to code, datasets and the research paper ➡️ https://go.fb.me/4cc23h

April 29, see you at the first-ever LlamaCon. 🦙 https://go.fb.me/LlamaCon & we'll see you again later this year at Meta Connect, September 17-18.

Meta FAIR researchers, in collaboration with BCBL - Basque Center on Cognition, Brain and Language are sharing two breakthroughs that show how AI can help advance our understanding of human intelligence. 1. Research that successfully decodes the production of sentences from non-invasive brain recordings ➡️ https://go.fb.me/w81a3s 2. A study toward understanding the neural mechanisms that coordinate language production in the human brain ➡️ https://go.fb.me/a9drx7 More details on all of this work ➡️ https://go.fb.me/4drvq1 We're excited to share these breakthroughs with the neuroscience community and hope that it will help to inspire and unlock new research in this important space.

Excited to share updates about a new collaboration that showcases the potential of open source AI models to unlock new creative possibilities! This partnership with artist Ruben Fro, Fisheye Immersive, and the Bibliothèque Nationale de France, using our SegmentAnything2 model, has resulted in a breathtaking work called "Deep Diving". We've also updated SAM to make it more easily accessible to artists and developers everywhere. SAM 2.1 is now available on Amazon SageMaker JumpStart, making it easier to deploy and integrate into your applications and workflows. Check out our update here: https://lnkd.in/g83tJYqz