Model Drop

Mistral Medium 3

May 7, 2025 2 min read

Mistral Medium 3, which Mistral AI shipped on May 7, 2025, is a closed-weights multimodal model that Mistral pitched as near-frontier quality at a fraction of the usual price, with the option to self-host it.

Model: Mistral Medium 3 (mistral-medium-2505)
Maker: Mistral AI
Ship date: May 7, 2025
Pricing: $0.40 / $2 per M tokens
Context: 128K tokens
Weights: Closed
Where you can use it:
- Mistral La Plateforme API
- Amazon SageMaker and Le Chat Enterprise
When to reach for it:
- Cheap coding and document or chart reading
- Self-hosting on four or more GPUs
- Skip it for hard science and reasoning work
How it compares:
- Costs about what GPT-4.1 mini does
- Pitched against Claude 3.7 Sonnet and GPT-4o
What makes it different:
- Closed, but deployable in your own VPC
- Can be continuously pretrained for a company
Early reactions: Simon Willison flagged it as a closed release priced like GPT-4.1 mini and found the self-hosting option more interesting than the price.
Read more: Medium is the new large., Mistral Medium 3 model page

The benchmarks

Mistral Medium 3 is a non-reasoning model, so there's no effort setting. Mistral ran every score through its own internal pipeline, reused rivals' published numbers where they existed, and doesn't say whether Claude 3.7 Sonnet used extended thinking. The table also has a Command A column, left out here, and a "DeepSeek 3.1" column, left out because no model by that name existed on launch day.

Mistral AI's numbers

HumanEval (Python coding, 0-shot): 92.1%. +0.6 over GPT-4o (91.5%), +0.0 over Claude 3.7 Sonnet (92.1%), +6.7 over Llama 4 Maverick (85.4%).
LiveCodeBench (fresh coding problems, v6): 30.3%. +1.6 over Llama 4 Maverick (28.7%), −1.1 under GPT-4o (31.4%), −5.7 under Claude 3.7 Sonnet (36.0%).
MATH-500 (competition math, 0-shot): 91.0%. +14.6 over GPT-4o (76.4%), +8.0 over Claude 3.7 Sonnet (83.0%), +1.0 over Llama 4 Maverick (90.0%).
ArenaHard (hard chat prompts, 0-shot): 97.1%. +1.7 over GPT-4o (95.4%), +3.9 over Claude 3.7 Sonnet (93.2%), +5.3 over Llama 4 Maverick (91.8%).
GPQA Diamond (PhD-level science, 5-shot CoT): 57.1%. +4.6 over GPT-4o (52.5%), −4.0 under Llama 4 Maverick (61.1%), −12.6 under Claude 3.7 Sonnet (69.7%).
MMLU-Pro (hard general knowledge, 5-shot CoT): 77.2%. +1.4 over GPT-4o (75.8%), −2.8 under Claude 3.7 Sonnet (80.0%), −3.2 under Llama 4 Maverick (80.4%).
MMMU (college-level visual reasoning): 66.1%. −2.6 under GPT-4o (68.7%), −5.2 under Claude 3.7 Sonnet (71.3%), −5.7 under Llama 4 Maverick (71.8%).
DocVQA (reading scanned documents): 95.3%. +9.4 over GPT-4o (85.9%), +11.0 over Claude 3.7 Sonnet (84.3%), +1.2 over Llama 4 Maverick (94.1%).

Independent numbers

Artificial Analysis Intelligence Index (current v4.3.2, AA estimate): 9.0. −0.3 under Llama 4 Maverick (9.3), −1.2 under GPT-4.1 mini (10.2), −6.3 under Claude 3.7 Sonnet (15.3).
Epoch Capabilities Index (current Epoch AI composite): 134. +2 over Llama 4 Maverick (132), −1 under GPT-4.1 mini (135).