Model Drop

Grok 4

July 9, 2025 2 min read

Grok 4, which xAI released on July 9, 2025, is a reasoning model trained with reinforcement learning at pretraining scale and built to call its own tools, like web search, X search and a code interpreter.

Model: Grok 4 (grok-4-0709)
Maker: xAI
Ship date: July 9, 2025
Pricing: $3 / $15 per M tokens
Context: 256K tokens
Where you can use it:
- Grok for SuperGrok and Premium+ users
- The xAI API
When to reach for it:
- Hard math, science and research questions
- Skip it when you need fast, cheap replies
How it compares:
- Edges o3 and Gemini 2.5 Pro on reasoning
- Same price as Claude Sonnet 4
What makes it different:
- Reinforcement learning at pretraining scale
- Trained to pick its own searches and tools
Early reactions: Reviewers liked the scores, but the MechaHitler fallout and answers that searched Musk's own posts took over the week.
Read more: Grok 4 and the Grok 4 Model Card (added August 20)

The benchmarks

xAI's charts compare Grok 4 against o3, Gemini 2.5 Pro and Claude Opus 4, mostly without tools; the tool rows let Grok 4 use Python (plus web search on Humanity's Last Exam, full set as of April 3, 2025), and the Grok 4 Heavy multi-agent numbers are left out here.

xAI's numbers

Humanity's Last Exam (no tools): 25.4%. +4.4 over o3 (21%), +3.8 over Gemini 2.5 Pro (21.6%).
Humanity's Last Exam (with tools): 38.6%. +13.7 over o3 (24.9%).
GPQA Diamond (grad-level science): 87.5%. +4.2 over o3 (83.3%), +1.1 over Gemini 2.5 Pro (86.4%), +7.9 over Claude Opus 4 (79.6%).
LiveCodeBench (coding problems, Jan to May): 79%. +7.0 over o3 (72%), +4.8 over Gemini 2.5 Pro (74.2%).
AIME 2025 (math contest, no tools): 91.7%. +2.8 over o3 (88.9%), +3.7 over Gemini 2.5 Pro (88%), +16.2 over Claude Opus 4 (75.5%).
HMMT 2025 (math contest, no tools): 90%. +12.5 over o3 (77.5%), +7.5 over Gemini 2.5 Pro (82.5%), +31.7 over Claude Opus 4 (58.3%).
USAMO 2025 (olympiad math proofs): 37.5%. +15.8 over o3 (21.7%), +3.0 over Gemini 2.5 Pro (34.5%), −11.9 under Gemini Deep Think (49.4%).

Independent numbers

Artificial Analysis Intelligence Index (current v4.3.2, AA estimate): 22.5. +2.3 over o3 (20.2), +1.9 over Claude Opus 4 (20.6), +6.4 over Gemini 2.5 Pro (16.1), +9.4 over DeepSeek R1 0528 (13.1).
ARC-AGI-2 (ARC Prize, semi-private set): 15.9%. +7.3 over Claude Opus 4 (8.6%), +9.4 over o3 (high) (6.5%), +11.0 over Gemini 2.5 Pro (4.9%).
ARC-AGI-1 (ARC Prize, semi-private set): 66.7%. +5.9 over o3 (high) (60.8%), +25.7 over Gemini 2.5 Pro (41.0%), +31.0 over Claude Opus 4 (35.7%).