Get more done with Gemini: Try 1.5 Pro and more intelligent features

May 14, 2024

5 min read

We’re bringing Gemini 1.5 Pro to Gemini Advanced subscribers in over 35 languages, along with a 1 million token context window, a new conversational experience and tools that let Gemini take action on your behalf.

Sissie Hsiao

Vice President and General Manager, Gemini experiences and Google Assistant

Gemini is designed to be your personal AI assistant — one that’s conversational, intuitive and helpful. Whether you use it in the app or through the web experience, Gemini can help you tackle complex tasks, and it can take action on your behalf.

Now, we’re making several updates — including a longer context window, new data analysis capabilities, connections to additional Google apps and more customizable options — so you can collaborate with the most intelligent and personalized Gemini yet.

Analyze documents with the world’s longest context window

Today we’re bringing Gemini 1.5 Pro, our cutting-edge model, to Gemini Advanced. Over 1 million people have signed up to try Gemini Advanced in just three months. And now Gemini 1.5 Pro brings you our latest technical advances, including a greatly expanded context window starting at 1 million tokens — the longest of any widely available consumer chatbot in the world. A context window this long means Gemini Advanced can make sense of multiple large documents, up to 1,500-pages total, or summarize 100 emails. Soon it will be able to handle an hour of video content or codebases with more than 30,000 lines.

An animation comparing Gemini Advanced’s 1 million token context window to those of Claude 3 (200K), GPT-4 (128K) and Gemini app (32K).

To take advantage of this long context window, we’re adding the ability to upload files via Google Drive or directly from your device, right into Gemini Advanced. Now you can quickly get answers and insights about dense documents, like figuring out the details of the pet policy in your rental agreement or comparing key arguments of multiple long research papers. And soon, Gemini Advanced will act as a data analyst, uncovering insights and building custom visualizations and charts on the fly from uploaded data files like spreadsheets.

Gemini keeps your files private to you, and they’re not used to train our models.

An illustration of a user uploading many different Google Sheets into Gemini, alongside the prompt: “Visualize the growth rate across all of my projects, in a single chart.”

Gemini is natively multimodal, and 1.5 Pro brings big improvements to image understanding. For example, you can snap a photo of a dish at your favorite restaurant and ask for a recipe, or take a picture of a math problem and get step-by-step instructions on how to solve it — all from a single image.

Gemini 1.5 Pro will be available to Gemini Advanced subscribers in more than 150 countries and over 35 languages.

Have more natural conversations with Gemini Live

We’re also adding new ways to interact with Gemini more naturally, whether you’re texting or talking. With Gemini in Google Messages, you can now chat with Gemini in the same app you message your friends.

And in the coming months, we’re rolling out Live for Gemini Advanced subscribers, a new mobile conversational experience that uses our state-of-the-art speech technology to make speaking with Gemini more intuitive. With Gemini Live, you can talk to Gemini and choose from a variety of natural-sounding voices it can respond with. You can even speak at your own pace or interrupt mid-response with clarifying questions, just like you would in any conversation.

Animation of a mobile phone showing the Gemini spark dropping from the top of the screen where it reads “Gemini Live” to the bottom, where it turns into glowing blue and purple gradient overlaid with a “Hold” button and red “Stop” button. On the left of the image text reads “Gemini Live.”

Let’s say you’re getting ready for a job interview: Just go Live and ask Gemini to help you prepare. Gemini can rehearse with you, and even suggest skills to highlight when talking to your potential employer. Later this year you’ll be able to use your camera when you go Live, opening up conversations about what you see around you.

Take the hassle out of making complex plans, like trip itineraries

Sometimes you spend more time researching and organizing a trip than enjoying the trip itself. Gemini Advanced’s new planning experience will go beyond showing a list of suggested activities to create a custom itinerary just for you.

Imagine you ask Gemini: “My family and I are going to Miami for Labor Day. My son loves art and my husband really wants fresh seafood. Can you pull my flight and hotel info from Gmail and help me plan the weekend?”

This prompt requires Gemini to do so much more than present publicly available information like other chatbots. Gemini takes into account your flight timing, meal preferences and information about local museums, while also understanding where each stop is located and how long it will take to travel between each activity. It grabs your flight information from Gmail, taps Google Maps for restaurant and museum recommendations near your hotel, and uses Search to recommend other activities, like a walking tour of the Design District or beach time, to fill out the rest of your day. It synthesizes all that information for you and creates a personal, customized itinerary that satisfies all your requests. And if you make changes or add more details, the itinerary will update automatically.

A prompt details a complex request for help planning a family vacation to Miami, including entertainment and food preferences. A stylized animation shows how Gemini identifies and connects the various prompts to help create a trip itinerary and locate flight information from Gmail. Animation disclaimer reads: Results from products and features are for illustrative purposes. Research prototype shown. Check responses for accuracy. Subscription may be required. Country and language availability varies.

This dynamic new planning experience is coming to Gemini Advanced in the coming months.

Personalize Gemini with Gems

For an even more personal experience, Gemini Advanced subscribers will soon be able to create Gems — customized versions of Gemini. You can create any Gem you dream up: a gym buddy, sous chef, coding partner or creative writing guide. They’re easy to set up, too. Simply describe what you want your Gem to do and how you want it to respond — like “you're my running coach, give me a daily running plan and be positive, upbeat and motivating.” Gemini will take those instructions and, with one click, enhance them to create a Gem that meets your specific needs.

In the Gems user interface, a user creates a new Gem, instructing it to act as a storyteller who specializes in crafting short stories with mysterious twists. They name this Gem “Cliffhanger Curator,” give it detailed instructions, and add it to a list of other Gems they’ve created, including Yoga Bestie, Sous Chef, Calculus Tutor, and Peer Reviewer. Animation disclaimer reads: Results from products and features are for illustrative purposes. Research prototype shown. Check responses for accuracy. Subscription may be required. Country and language availability varies.

Connect with more Google apps

Last year we brought Extensions directly into Gemini, allowing you to get more things done with the Google apps and services you already use. We’ve continued to bring Google apps to Gemini, like the YouTube Music Extension rolling out now. With Gemini, you can search for your favorite music even if you don't know the song title by mentioning a favorite verse or a featured artist.

Soon, we’ll connect even more Google tools with Gemini, including Google Calendar, Tasks and Keep. So you’ll be able to do things like snap a picture of your child’s school syllabus and ask Gemini to create a calendar entry for each assignment, or take a photo of a new recipe and add it to your Keep as a shopping list.

With these updates, many of which are coming to our business customers too, you’ll get the most personal and helpful experience with Gemini yet — all from a single prompt.

Look out for these new features soon, and start chatting with Gemini today.