Model Drop

Gemini 2.5 Pro

March 25, 2025 2 min read

Gemini 2.5 Pro, which Google DeepMind released as an experimental model on March 25, 2025, is Google's first 2.5 thinking model, built to reason through a problem before it answers.

Model: Gemini 2.5 Pro Experimental (gemini-2.5-pro-exp-03-25)
Maker: Google DeepMind
Ship date: March 25, 2025
Pricing: Free experimental tier at launch; $1.25 / $10 per M tokens from April 4, 2025 (prompts up to 200K)
Context: 1M tokens (2M promised)
Where you can use it:
- Google AI Studio via the Gemini API
- Gemini app for Gemini Advanced subscribers
When to reach for it:
- Hard math, science and logic questions
- Coding across a whole repo in one prompt
- Skip it for scaled production (rate limits)
How it compares:
- Out-reasons o3-mini, Grok 3 and DeepSeek R1
- Claude 3.7 Sonnet still leads agentic coding
What makes it different:
- Thinking is always on by default
- Long context plus audio and video input
Early reactions: Simon Willison called it a very strong new model whose Python coding felt comparable to Claude 3.7 Sonnet.
Read more: Gemini 2.5: Our most intelligent AI model

The benchmarks

All Gemini scores are single-attempt pass@1 at default settings (thinking always on); rival scores are self-reported or from public leaderboards (Claude 3.7 Sonnet mostly with extended thinking, Grok 3 Beta with extended reasoning), SWE-bench used Google's custom agent setup, o3-mini and DeepSeek R1 took Humanity's Last Exam text-only, and the Aider Polyglot row uses Gemini's diff-format score (74.0% in whole-file format) to match the rivals' diff scores.

Google DeepMind's numbers

Humanity's Last Exam (no tools) (expert-level questions): 18.8%. +4.8 over o3-mini (high) (14.0%), +12.4 over GPT-4.5 (6.4%), +9.9 over Claude 3.7 Sonnet (8.9%), +10.2 over DeepSeek R1 (8.6%).
GPQA Diamond (graduate-level science): 84.0%. +4.3 over o3-mini (high) (79.7%), +5.8 over Claude 3.7 Sonnet (78.2%), +3.8 over Grok 3 (80.2%), +12.5 over DeepSeek R1 (71.5%).
AIME 2025 (competition math): 86.7%. +0.2 over o3-mini (high) (86.5%), +37.2 over Claude 3.7 Sonnet (49.5%), +9.4 over Grok 3 (77.3%), +16.7 over DeepSeek R1 (70.0%).
LiveCodeBench (code generation, v5): 70.4%. −3.7 under o3-mini (high) (74.1%), −0.2 under Grok 3 (70.6%), +6.1 over DeepSeek R1 (64.3%).
Aider Polyglot (code editing, diff): 68.6%. +8.2 over o3-mini (high) (60.4%), +23.7 over GPT-4.5 (44.9%), +3.7 over Claude 3.7 Sonnet (64.9%), +11.7 over DeepSeek R1 (56.9%).
SWE-bench Verified (real GitHub fixes): 63.8%. +14.5 over o3-mini (high) (49.3%), +25.8 over GPT-4.5 (38.0%), −6.5 under Claude 3.7 Sonnet (70.3%), +14.6 over DeepSeek R1 (49.2%).
SimpleQA (factual recall): 52.9%. +39.1 over o3-mini (high) (13.8%), −9.6 under GPT-4.5 (62.5%), +9.3 over Grok 3 (43.6%), +22.8 over DeepSeek R1 (30.1%).
MMMU (multimodal reasoning): 81.7%. +7.3 over GPT-4.5 (74.4%), +6.7 over Claude 3.7 Sonnet (75.0%), +5.7 over Grok 3 (76.0%).

Independent numbers

Artificial Analysis Intelligence Index (current v4.3.2, AA estimate): 15.0. +4.0 over o3-mini (high) (11.0), −2.7 under Claude 3.7 Sonnet (thinking) (17.7), +4.6 over Grok 3 (10.4), +3.6 over DeepSeek R1 (11.4).
LMArena text Elo (human preference votes): 1443. +39 over Grok 3 (1404), +45 over GPT-4.5 (1398), +139 over Claude 3.7 Sonnet (thinking) (1304).