Model Drop

DeepSeek V3

December 26, 2024 2 min read

DeepSeek V3, which DeepSeek shipped on December 26, 2024, is a 671-billion-parameter open-weights mixture-of-experts model that matched leading closed chat models on a small training budget.

Model: DeepSeek V3 (deepseek-chat)
Maker: DeepSeek
Ship date: December 26, 2024
Pricing: $0.14 / $0.28 per M tokens (promo until February 8, 2025, then $0.27 / $1.10)
Context: 128K tokens (64K on DeepSeek's own API)
Where you can use it:
- DeepSeek chat and the DeepSeek API
- Open weights on Hugging Face
When to reach for it:
- Cheap, strong everyday chat and coding
- Self-hosting a frontier-class open model
- Skip it for images; it reads text only
How it compares:
- Trades blows with GPT-4o, Claude 3.5 Sonnet
- Ahead of Llama 3.1 405B and Qwen2.5 72B
What makes it different:
- 671B total, 37B active per token
- Trained on about 2.8M H800 GPU hours
- FP8 training and multi-token prediction
Early reactions: Reviewers fixated on the roughly $5.6 million training bill for a model that benchmarks near Claude 3.5 Sonnet.
Read more: Introducing DeepSeek-V3, technical report

The benchmarks

DeepSeek's own runs with output capped at 8K tokens; it compared against Claude 3.5 Sonnet (October 2024), the May 2024 GPT-4o and Llama 3.1 405B, so GPT-4o is an older build than the one on sale at launch.

DeepSeek's numbers

MMLU-Pro (hard general knowledge): 75.9%. −2.1 under Claude 3.5 Sonnet (78.0%), +3.3 over GPT-4o (72.6%), +2.6 over Llama 3.1 405B (73.3%).
GPQA Diamond (PhD-level science): 59.1%. −5.9 under Claude 3.5 Sonnet (65.0%), +9.2 over GPT-4o (49.9%), +8.0 over Llama 3.1 405B (51.1%).
SWE-bench Verified (real GitHub fixes): 42.0%. −8.8 under Claude 3.5 Sonnet (50.8%), +3.2 over GPT-4o (38.8%), +17.5 over Llama 3.1 405B (24.5%).
LiveCodeBench (fresh coding problems): 40.5%. +4.2 over Claude 3.5 Sonnet (36.3%), +7.1 over GPT-4o (33.4%), +12.1 over Llama 3.1 405B (28.4%).
AIME 2024 (competition math): 39.2%. +23.2 over Claude 3.5 Sonnet (16.0%), +29.9 over GPT-4o (9.3%), +15.9 over Llama 3.1 405B (23.3%).
MATH-500 (math problem solving): 90.2%. +11.9 over Claude 3.5 Sonnet (78.3%), +15.6 over GPT-4o (74.6%), +16.4 over Llama 3.1 405B (73.8%).
SimpleQA (short factual recall): 24.9%. −3.5 under Claude 3.5 Sonnet (28.4%), −13.3 under GPT-4o (38.2%), +7.8 over Llama 3.1 405B (17.1%).

Independent numbers

Artificial Analysis Intelligence Index (current v4.3.2): 8.5. +0.6 over Claude 3.5 Sonnet (7.9), +0.1 over GPT-4o (8.4), +1.2 over Llama 3.1 405B (7.3), +0.8 over Qwen2.5 72B (7.7).
Aider Polyglot (multi-language code edits): 48.4%. +30.2 over GPT-4o (18.2%), −13.3 under o1 (high) (61.7%).