Introducing Glider - the smallest model to beat GPT-4o-mini on eval tasks 🚀👀 - Open source, open weights, open code - Explainable evaluations by nature - Trained on 183 criteria and 685 domains And that’s our 12th day of Christmas at Patronus AI 😉🌲 Download Glider on Hugging Face: https://lnkd.in/eud69M8w Try out Glider on Patronus for free: https://app.patronus.ai Read the ArXiv paper: https://lnkd.in/eSnAmZ9g Read our blog: https://lnkd.in/ej3a4fME Glider demo on Hugging Face Spaces: https://lnkd.in/eRyn3WY8 Read the VentureBeat coverage by Michael Nuñez: https://lnkd.in/eZ8xrg-2
Patronus AI’s Post
More Relevant Posts
-
vLLM 0.6.1 just got released, and it's coming another major boost to its multimodal capabilities with Pixtral support! 🖼 vLLM support for Pixtral, Mistral AI's first multimodal model, was just merged, hours after the model was first released. This effort was led by Patrick von Platen from the Mistral AI team in collaboration with Roger Wang, Cyrus Leung, Simon Mo and other vLLM contributors. This means you can try out this model on your own infrastructure, simply by updating vLLM to 0.6.1! 📈 Mixtral's first multimodal model Pixtral is expected to outperform most other open source models including Qwen2-7b and LlaVA-OV on a variety of benchmarks. Some of the benchmark results, such as Mathvista, are even better than GPT-4o-mini! 🚀 Try it now by * Install vLLM: `pip install -U vllm==0.6.1` * Take a run with an example script: https://lnkd.in/g8hmS-jp * PR: https://lnkd.in/gxY7Wk3B If you're interested in learning more, the core members of the vLLM team will be at Ray Summit this year. Come by! https://lnkd.in/gRfYQ-ST (Photo credit goes to swyx on twitter)
-
-
Enterprise Data Science Specialist @Mastech Digital | NLP | NER | Deep Learning | Gen AI | MLops
🚀 The release of vLLM 0.6.1 with support for Pixtral is a major milestone for the AI community! 🖼 The seamless integration of Pixtral, Mistral AI’s first multimodal model, into vLLM so soon after its release demonstrates just how rapidly innovation is happening in the world of AI. Huge kudos to Patrick von Platen, Roger Wang, Cyrus Leung, Simon Mo, and the entire vLLM team for pulling this off. With vLLM 0.6.1, users can now deploy Pixtral on their own infrastructure, bringing cutting-edge multimodal capabilities directly to their projects. The potential of Pixtral is incredibly exciting. It’s expected to surpass leading models like Qwen2-7b and LlaVA-OV across various benchmarks, making it a game-changer for multimodal tasks. Even more impressive, the performance on benchmarks like Mathvista, where Pixtral is reported to outperform models such as GPT-4o-mini, suggests this model is a serious contender in the open-source space. With this release, developers and researchers can now experiment with Pixtral's multimodal capabilities, including vision and text tasks, by simply updating to vLLM 0.6.1. The installation process is straightforward, and with the example script provided, you can get started almost immediately. If you're attending Ray Summit, don’t miss the opportunity to meet the core members of the vLLM team to dive deeper into what’s next for this groundbreaking project. This is a great step forward for anyone involved in multimodal AI, machine learning, and AI research! Thanks again to everyone involved for pushing the boundaries of what’s possible in AI! #vLLM #Pixtral #MultimodalAI #MistralAI #DeepLearning #MachineLearning #AIResearch #Qwen2 #LlaVA #GPT4o #OpenSourceAI #RaySummit #AIInnovation #DataScience #MLOps
vLLM 0.6.1 just got released, and it's coming another major boost to its multimodal capabilities with Pixtral support! 🖼 vLLM support for Pixtral, Mistral AI's first multimodal model, was just merged, hours after the model was first released. This effort was led by Patrick von Platen from the Mistral AI team in collaboration with Roger Wang, Cyrus Leung, Simon Mo and other vLLM contributors. This means you can try out this model on your own infrastructure, simply by updating vLLM to 0.6.1! 📈 Mixtral's first multimodal model Pixtral is expected to outperform most other open source models including Qwen2-7b and LlaVA-OV on a variety of benchmarks. Some of the benchmark results, such as Mathvista, are even better than GPT-4o-mini! 🚀 Try it now by * Install vLLM: `pip install -U vllm==0.6.1` * Take a run with an example script: https://lnkd.in/g8hmS-jp * PR: https://lnkd.in/gxY7Wk3B If you're interested in learning more, the core members of the vLLM team will be at Ray Summit this year. Come by! https://lnkd.in/gRfYQ-ST (Photo credit goes to swyx on twitter)
-
-
NVIDIA Senior Research Manager & Lead of Embodied AI (GEAR Lab). Stanford Ph.D. Building Humanoid Robots and Physical AI. OpenAI's first intern. Sharing insights on the bleeding edge of AI.
The power of GPT-4o in the palm of our hands. It's a monumental day - for the first time in history, open weights catch up with the latest frontier models. This figure charts the historic run of open models against the closed ones. The former starts humble, but rise with a much higher gradient and a much more diverse ecosystem. Research like multimodal LM and robot foundation models wouldn't have been possible without white box access to a strong base LM. Llama-3.1 release includes more than just weights: - "Open Source AI is the Path Forward" - a manifesto that clearly lays out Zuckerberg's vision. It explains the commercial strategies, ecosystem positioning, and even geopolitical concerns, basically answering FAQs for OSS LLM in general: https://lnkd.in/gJt_nYZp - A 71-page long paper that treats all LLM researchers to a buffet of training details and analysis. They even discuss large-scale cluster failures and remedies - issues that only emerge with 16K H100s! It's literally early Christmas for my team ;) GPT-4o and Claude-3.5 are great, but I would easily rank Llama-3.1 the No. 1 highlight in the 2024 LLM landscape.
-
-
💥To my network studying #platforms, #openinnovation and #strategy 💥 The race between closed-source vs. open-weight models will shape technology adoptions, platform adoptions, and business #ecosystem for years to come. There are plenty of research questions relevant for business, policy makers, tech specialists. For example:1) how will the race shape knowledge trajectories in the industries, 2) which ecosystems and actors (e.g. incumbents, agents at the environemt fringe with specialized knowledge, etc) will benefit most from the race, 3) how will policy makers react/interact against such trends? I hope to read sparkling research 🤓 ✨️. Tagging some who may be interested: Hakan Özalp Carmelo Cennamo Christopher Tucci Gaétan de Rassenfosse Prof. Annabelle Gawer Tobias Kretschmer Milan Miric Anil Doshi.
NVIDIA Senior Research Manager & Lead of Embodied AI (GEAR Lab). Stanford Ph.D. Building Humanoid Robots and Physical AI. OpenAI's first intern. Sharing insights on the bleeding edge of AI.
The power of GPT-4o in the palm of our hands. It's a monumental day - for the first time in history, open weights catch up with the latest frontier models. This figure charts the historic run of open models against the closed ones. The former starts humble, but rise with a much higher gradient and a much more diverse ecosystem. Research like multimodal LM and robot foundation models wouldn't have been possible without white box access to a strong base LM. Llama-3.1 release includes more than just weights: - "Open Source AI is the Path Forward" - a manifesto that clearly lays out Zuckerberg's vision. It explains the commercial strategies, ecosystem positioning, and even geopolitical concerns, basically answering FAQs for OSS LLM in general: https://lnkd.in/gJt_nYZp - A 71-page long paper that treats all LLM researchers to a buffet of training details and analysis. They even discuss large-scale cluster failures and remedies - issues that only emerge with 16K H100s! It's literally early Christmas for my team ;) GPT-4o and Claude-3.5 are great, but I would easily rank Llama-3.1 the No. 1 highlight in the 2024 LLM landscape.
-
-
Senior Software Engineer specialized in Machine Learning, Blockchain, Cloud & Full Stack development.
Here we are ⭐️, open-source has reached the level of closed-source models. This is a massive day for open-source AI and for all the independent reserchers and engineers that would like to understand what’s really taking place at the technical level. This is more than just open-weights 🔥 🚀
NVIDIA Senior Research Manager & Lead of Embodied AI (GEAR Lab). Stanford Ph.D. Building Humanoid Robots and Physical AI. OpenAI's first intern. Sharing insights on the bleeding edge of AI.
The power of GPT-4o in the palm of our hands. It's a monumental day - for the first time in history, open weights catch up with the latest frontier models. This figure charts the historic run of open models against the closed ones. The former starts humble, but rise with a much higher gradient and a much more diverse ecosystem. Research like multimodal LM and robot foundation models wouldn't have been possible without white box access to a strong base LM. Llama-3.1 release includes more than just weights: - "Open Source AI is the Path Forward" - a manifesto that clearly lays out Zuckerberg's vision. It explains the commercial strategies, ecosystem positioning, and even geopolitical concerns, basically answering FAQs for OSS LLM in general: https://lnkd.in/gJt_nYZp - A 71-page long paper that treats all LLM researchers to a buffet of training details and analysis. They even discuss large-scale cluster failures and remedies - issues that only emerge with 16K H100s! It's literally early Christmas for my team ;) GPT-4o and Claude-3.5 are great, but I would easily rank Llama-3.1 the No. 1 highlight in the 2024 LLM landscape.
-
-
XR Responsible Innovation Advocate. Executive Leader. Creative Storyteller. Metaverse Researcher.
Strong thoughts from Jim Fan. While GPT-4o mini is an attempt to provide device level support in a closed model - credit to Zuck, Yann LeCun, and Meta AI for taking the open source approach. Democratized open-sourced AI is more responsible by nature because we can audit weights and practices directly
NVIDIA Senior Research Manager & Lead of Embodied AI (GEAR Lab). Stanford Ph.D. Building Humanoid Robots and Physical AI. OpenAI's first intern. Sharing insights on the bleeding edge of AI.
The power of GPT-4o in the palm of our hands. It's a monumental day - for the first time in history, open weights catch up with the latest frontier models. This figure charts the historic run of open models against the closed ones. The former starts humble, but rise with a much higher gradient and a much more diverse ecosystem. Research like multimodal LM and robot foundation models wouldn't have been possible without white box access to a strong base LM. Llama-3.1 release includes more than just weights: - "Open Source AI is the Path Forward" - a manifesto that clearly lays out Zuckerberg's vision. It explains the commercial strategies, ecosystem positioning, and even geopolitical concerns, basically answering FAQs for OSS LLM in general: https://lnkd.in/gJt_nYZp - A 71-page long paper that treats all LLM researchers to a buffet of training details and analysis. They even discuss large-scale cluster failures and remedies - issues that only emerge with 16K H100s! It's literally early Christmas for my team ;) GPT-4o and Claude-3.5 are great, but I would easily rank Llama-3.1 the No. 1 highlight in the 2024 LLM landscape.
-
-
Manager Customer Success | Technology Leader | Global Team | Digital Transformation | Cloud
In collaboration with #Meta, #Microsoft is announcing Llama 3.1 405B available today through #Azure AI’s Models-as-a-Service as a serverless API https://lnkd.in/gryDpaaf Great read on the benefits of open source: Open Source #AI is the Path Forward" - a manifesto that clearly lays out Zuckerberg's vision. It explains the commercial strategies, ecosystem positioning, and even geopolitical concerns, basically answering FAQs for OSS LLM in general: https://lnkd.in/gJt_nYZp
NVIDIA Senior Research Manager & Lead of Embodied AI (GEAR Lab). Stanford Ph.D. Building Humanoid Robots and Physical AI. OpenAI's first intern. Sharing insights on the bleeding edge of AI.
The power of GPT-4o in the palm of our hands. It's a monumental day - for the first time in history, open weights catch up with the latest frontier models. This figure charts the historic run of open models against the closed ones. The former starts humble, but rise with a much higher gradient and a much more diverse ecosystem. Research like multimodal LM and robot foundation models wouldn't have been possible without white box access to a strong base LM. Llama-3.1 release includes more than just weights: - "Open Source AI is the Path Forward" - a manifesto that clearly lays out Zuckerberg's vision. It explains the commercial strategies, ecosystem positioning, and even geopolitical concerns, basically answering FAQs for OSS LLM in general: https://lnkd.in/gJt_nYZp - A 71-page long paper that treats all LLM researchers to a buffet of training details and analysis. They even discuss large-scale cluster failures and remedies - issues that only emerge with 16K H100s! It's literally early Christmas for my team ;) GPT-4o and Claude-3.5 are great, but I would easily rank Llama-3.1 the No. 1 highlight in the 2024 LLM landscape.
-
-
If you doubt that open source LLM will catch up with commercial ones, take one look at the chart below. Hard not to gloat, but I called this a year ago: https://lnkd.in/dd5nwjNM
NVIDIA Senior Research Manager & Lead of Embodied AI (GEAR Lab). Stanford Ph.D. Building Humanoid Robots and Physical AI. OpenAI's first intern. Sharing insights on the bleeding edge of AI.
The power of GPT-4o in the palm of our hands. It's a monumental day - for the first time in history, open weights catch up with the latest frontier models. This figure charts the historic run of open models against the closed ones. The former starts humble, but rise with a much higher gradient and a much more diverse ecosystem. Research like multimodal LM and robot foundation models wouldn't have been possible without white box access to a strong base LM. Llama-3.1 release includes more than just weights: - "Open Source AI is the Path Forward" - a manifesto that clearly lays out Zuckerberg's vision. It explains the commercial strategies, ecosystem positioning, and even geopolitical concerns, basically answering FAQs for OSS LLM in general: https://lnkd.in/gJt_nYZp - A 71-page long paper that treats all LLM researchers to a buffet of training details and analysis. They even discuss large-scale cluster failures and remedies - issues that only emerge with 16K H100s! It's literally early Christmas for my team ;) GPT-4o and Claude-3.5 are great, but I would easily rank Llama-3.1 the No. 1 highlight in the 2024 LLM landscape.
-
-
Using Tech, Data & AI to leverage business and impact people for a fairer and more evolved world. "Happiness is Only Real When Shared"
A year ago, I saw Julien Chaumond 's post https://lnkd.in/dVp4VKxP? predicting the rapid advancement of open-source models. Today, looking to Jim Fan 's insights, maybe this prediction is materializing. Open-weight models have caught up with leading closed-source models for the first time. The Llama-3.1 release, in particular, marks a crucial point, not only closing the performance gap but also demonstrating the strength and diversity of the open-source ecosystem. I'm impressed with the report on LLaMA 3.1. It's remarkable. It's great to see everything open and thoroughly explained. For more details, you can read the full report here: https://lnkd.in/dxf6SjSQ Llama-3.1's capabilities underscore the transformative potential of open-source initiatives, offering robust performance and innovative features that rival proprietary models. This progress reaffirms the role of open-source in driving the future of AI, fostering collaboration and transparency in the machine learning community.
NVIDIA Senior Research Manager & Lead of Embodied AI (GEAR Lab). Stanford Ph.D. Building Humanoid Robots and Physical AI. OpenAI's first intern. Sharing insights on the bleeding edge of AI.
The power of GPT-4o in the palm of our hands. It's a monumental day - for the first time in history, open weights catch up with the latest frontier models. This figure charts the historic run of open models against the closed ones. The former starts humble, but rise with a much higher gradient and a much more diverse ecosystem. Research like multimodal LM and robot foundation models wouldn't have been possible without white box access to a strong base LM. Llama-3.1 release includes more than just weights: - "Open Source AI is the Path Forward" - a manifesto that clearly lays out Zuckerberg's vision. It explains the commercial strategies, ecosystem positioning, and even geopolitical concerns, basically answering FAQs for OSS LLM in general: https://lnkd.in/gJt_nYZp - A 71-page long paper that treats all LLM researchers to a buffet of training details and analysis. They even discuss large-scale cluster failures and remedies - issues that only emerge with 16K H100s! It's literally early Christmas for my team ;) GPT-4o and Claude-3.5 are great, but I would easily rank Llama-3.1 the No. 1 highlight in the 2024 LLM landscape.
-
-
Open-source LLMs are catching up fast, and it's coming from the most unlikely source in my opinion: Meta and their Llama 3.1 models Since Meta's goal is to remain #1 in content generation and ad-tech business, they have much less to do with selling LLM-as-a-service. They are primarily focused on keeping their users (social media app users) and customers (companies selling ads on across their Threads/Instagram/WhatApp platform) engaged with user activity (e.g. creating memes and keeping "influencers" relevant). Helping users create content better and more fun, with hopes of going "viral" using AI-augmented content is what they are after. Open-sourcing has dual goals: 1) Reduce user "outflow" to outside photo/video/text generation app - if users can create photo / music / video in Instagram without leaving = more screen time. 2) Future proof their business - they don't want to turn into Google, whose search-engine based ad-tech is starting to go the way of Blackberry.
NVIDIA Senior Research Manager & Lead of Embodied AI (GEAR Lab). Stanford Ph.D. Building Humanoid Robots and Physical AI. OpenAI's first intern. Sharing insights on the bleeding edge of AI.
The power of GPT-4o in the palm of our hands. It's a monumental day - for the first time in history, open weights catch up with the latest frontier models. This figure charts the historic run of open models against the closed ones. The former starts humble, but rise with a much higher gradient and a much more diverse ecosystem. Research like multimodal LM and robot foundation models wouldn't have been possible without white box access to a strong base LM. Llama-3.1 release includes more than just weights: - "Open Source AI is the Path Forward" - a manifesto that clearly lays out Zuckerberg's vision. It explains the commercial strategies, ecosystem positioning, and even geopolitical concerns, basically answering FAQs for OSS LLM in general: https://lnkd.in/gJt_nYZp - A 71-page long paper that treats all LLM researchers to a buffet of training details and analysis. They even discuss large-scale cluster failures and remedies - issues that only emerge with 16K H100s! It's literally early Christmas for my team ;) GPT-4o and Claude-3.5 are great, but I would easily rank Llama-3.1 the No. 1 highlight in the 2024 LLM landscape.
-
