Model Drop

Gemini 2.0 Flash

December 11, 2024 2 min read

Gemini 2.0 Flash, which Google DeepMind released as a public experimental model on December 11, 2024 (the stable API version followed on February 5, 2025), is a fast multimodal workhorse model with native tool use.

Model: Gemini 2.0 Flash Experimental (gemini-2.0-flash-exp)
Maker: Google DeepMind
Ship date: December 11, 2024 (experimental; GA on February 5, 2025)
Pricing: Free during the experimental preview; $0.10 / $0.40 per M tokens at GA
Context: 1M tokens
Where you can use it:
- Google AI Studio and Vertex AI (Gemini API)
- Gemini app, as a chat-tuned version
When to reach for it:
- Fast multimodal work on images, video, audio
- Agents that call Search or run code
- Skip it for production apps until GA
How it compares:
- Out-voted GPT-4o mini and Claude 3.5 Haiku
- Trails Claude 3.5 Haiku on overall smarts
What makes it different:
- Native image and speech output (early access)
- Live API streams real-time audio and video
Early reactions: Simon Willison called it an outstanding multimodal model and its live streaming mode "wildly impressive."
Read more: Introducing Gemini 2.0: our new AI model for the agentic era

The benchmarks

Google compared the experimental model only against its own Gemini 1.5 Flash 002 and Gemini 1.5 Pro 002, with no outside rivals; Natural2Code and MRCR are internal Google evals.

Google DeepMind's numbers

MMLU-Pro (harder general knowledge): 76.4%. +9.1 over Gemini 1.5 Flash (67.3%), +0.6 over Gemini 1.5 Pro (75.8%).
Natural2Code (held-out code generation): 92.9%. +13.1 over Gemini 1.5 Flash (79.8%), +7.5 over Gemini 1.5 Pro (85.4%).
LiveCodeBench (recent coding problems): 35.1%. +5.1 over Gemini 1.5 Flash (30.0%), +0.8 over Gemini 1.5 Pro (34.3%).
MATH (competition math problems): 89.7%. +11.8 over Gemini 1.5 Flash (77.9%), +3.2 over Gemini 1.5 Pro (86.5%).
GPQA Diamond (graduate-level science): 62.1%. +11.1 over Gemini 1.5 Flash (51.0%), +3.0 over Gemini 1.5 Pro (59.1%).
MMMU (college-level multimodal reasoning): 70.7%. +8.4 over Gemini 1.5 Flash (62.3%), +4.8 over Gemini 1.5 Pro (65.9%).
MRCR (1M) (long-context recall): 69.2%. −2.7 under Gemini 1.5 Flash (71.9%), −13.4 under Gemini 1.5 Pro (82.6%).

Independent numbers

Artificial Analysis Intelligence Index (current v4.3.2, AA estimate): 8.2. +1.5 over GPT-4o mini (6.7), −0.7 under Claude 3.5 Haiku (8.9).
LMArena text Elo (human preference votes): 1355. +82 over GPT-4o mini (1273), +117 over Claude 3.5 Haiku (1238).