Gemini 2.0 is here and promises to be able to do (almost) anything

Google has used the last days of the year to launch its most
anticipated artificial intelligence model: Gemini 2.0. This is its
next-generation AI model, which promises to be a big step forward in terms of
intelligence and capabilities.

If the previous model focused on multimodality, version 2.0 is
based on AI agents, which are able to
act more autonomously and solve complex problems with less human
intervention. With this, Google is at the forefront of the race for the most
advanced AI models on the market. Here are all the details!

On the occasion of the launch of Gemini 2.0, Sundar Pichai, CEO
of Google and Alphabet, shared the following: “Information is at the heart of
human progress. That’s why for more than 26 years we’ve been focused on our
mission to organize the world’s information and make it accessible and
useful. And that’s why we continue to push the boundaries of AI to organize
that information at every input and make it accessible through any output so
that it can be truly useful to you. (…) Today, millions of developers are
developing with Gemini, which helps us reinvent all of our products
(including the 7 that have 2 billion users) and create new ones. In the last
year, we have invested in developing more agile models, i.e. able to better
understand the world around you, anticipate and act on your behalf, under
your supervision. Today we are excited to launch our next era of models
designed for this new era of agents: we are introducing Gemini 2.0, our most
capable model yet. With new advances in multimodality
(such as native audio and image output) and use of native tools, it will
allow us to create new AI agents that bring us closer to our vision of a
universal assistant (…)”.

In this video you can see a summary of the new capabilities of the
model:

The first model released by the company is Gemini 2.0 Flash, the
smaller and less powerful model, although even better than the current Pro
model. According to Demis Hassabis, CEO of Google
DeepMind, this model is more versatile and capable than
previous models and can generate multilingual images and audio
natively: “Flash even outperforms 1.5 Pro in key
benchmarks, with twice the speed and also comes with new capabilities. In
addition to supporting multimodal inputs such as images, video, and audio,
Flash 2.0 now supports multimodal outputs, such as natively generated images
mixed with text and multilingual audio synthesized from text (TTS). It is
also natively integrated with tools such as Google Search or code execution,
as well as third-party user-defined functions.”

This model is already available as an experimental model via the
Gemini API, with multimodal input and text output, native text-to-speech
conversion, and image generation.

It will be widely available in January, along with more model
sizes.

The biggest new feature of Gemini 2.0 lies in the AI agents. It
now includes native UI action capabilities, along with
other enhancements such as multimodal reasoning,
understanding long contexts, tracking and planning complex instructions,
calling composite functions, using native tools, and improving
latency.

These AI agents will have a major influence over the next few
years, and Google is exploring this field with several prototypes that can
help people perform tasks like never before.

It is still in its early stages of development, but one example is
the updated Project Astra, a prototype that explores the future capabilities
of a universal AI assistant.

We also find Project Mariner, which explores the future of
human-agent interaction, starting with the browser. Or Jules, an AI-powered
code agent that helps developers in their tasks, integrated directly into a
GitHub workflow.

Astra Project

A few months ago, Google launched this project, which they
presented as an evolution of virtual assistants, and which
can analyze our environment for numerous actions, such as
finding lost objects or describing situations.

With the arrival of Gemini 2.0, Project Astra has also been
improved:

• Improved dialogues: you now have the
  ability to converse in several languages, as well as a better understanding
  of accents or less common words.
• New use of tools: you can now
  use the search engine, Google Lens or Maps.
• Improved memory: you now have
  up to 10 minutes of memory during the session and can remember other
  conversations you have had with him in the past thanks to his
  personalization.
• Improved latency: thanks to new
  streaming features and native audio understanding, the AI agent can understand
  language with a latency similar to that of a human
  conversation.

Mariner Project

As briefly mentioned above, Project Mariner is a research
prototype built with Gemini 2.0 that explores the future
of human-agent interaction.

It is able to understand and reason, through browser screen
information, about pixels, text, code, images, or forms, and then use this
information through a Chrome extension that completes the tasks for
you.

It’s still at an early stage, but the results are looking very
promising.

Here comes the challenge of building it
securely and responsibly, so it can only type, scroll, or
click on the active browser tab and ask the user for a final confirmation
before performing certain sensitive actions.

With all these advances, Google and DeepMind have also emphasized
their commitment to security and accountability when
developing AI agents. As such, they are taking an
explorative and incremental approach to product development, testing multiple
prototypes, insisting on security integration and training, working with
trusted testers and external experts, and conducting thorough risk and
security and assurance assessments.

Without a doubt, Gemini 2.0 and the new prototypes open a great
door to a new generation of smarter and more autonomous AI models, and one
that we look forward to exploring and discovering. We will be sharing demos
using this new version very soon.