Model Drop

Llama 4 Maverick & Scout

April 5, 2025 3 min read

Llama 4, which Meta released on April 5, 2025, is a pair of open-weight multimodal models, the flagship Maverick and the smaller long-context Scout, and the first Llamas built as mixtures of experts.

Model: Llama 4 Maverick (meta-llama/Llama-4-Maverick-17B-128E-Instruct)

Maker: Meta

Ship date: April 5, 2025

Pricing: Free (open weights)

Context: 1M tokens

Where you can use it:

Downloads on llama.com and Hugging Face
Meta AI in WhatsApp, Messenger, Instagram

When to reach for it:

Self-hosted chat with image understanding
Skip it for hard coding and agent work

How it compares:

Open rival to DeepSeek V3-0324
Aimed at GPT-4o and Gemini 2.0 Flash

What makes it different:

128 experts, 17B active parameters
Text and images fused from pretraining

Early reactions: Researchers flagged that Meta’s LMArena ranking came from an unreleased chat-tuned Maverick, and the public model ranked below months-old rivals.

Read more: The Llama 4 herd · Llama 4 model card

Model: Llama 4 Scout (meta-llama/Llama-4-Scout-17B-16E-Instruct)

Maker: Meta

Ship date: April 5, 2025

Pricing: Free (open weights)

Context: 10M tokens

Where you can use it:

Downloads on llama.com and Hugging Face
One H100 GPU, with Int4 weights

When to reach for it:

Very long inputs, via its 10M window
Single-GPU self-hosting

How it compares:

Ahead of Gemma 3, Mistral 3.1 and Gemini 2.0 Flash-Lite
Behind Maverick on all but one of Meta’s tests

What makes it different:

16 experts, 109B total parameters
The longest context of any Llama

Read more: The Llama 4 herd · Llama 4 model card

The benchmarks

Meta's numbers are for instruction-tuned Maverick, 0-shot at temperature 0 with no test-time compute, against the best self-reported scores of non-thinking rivals Gemini 2.0 Flash, DeepSeek V3-0324 (Meta labeled it "DeepSeek v3.1") and GPT-4o; Meta ran DeepSeek's LiveCodeBench score itself on its date window. Scout’s rows come from the table in Meta’s model card, where both models ran the same tests; Meta charted Scout separately against Gemma 3 27B, Mistral 3.1 24B and Gemini 2.0 Flash-Lite. The independent rows are for Maverick, with GPT-4o as its November 2024 snapshot; the AA and Epoch scores are today's, and Aider's is from April 6, 2025.

Meta's numbers

GPQA Diamond (Maverick, PhD-level science questions): 69.8%. +9.7 over Gemini 2.0 Flash (60.1%), +1.4 over DeepSeek V3-0324 (68.4%), +16.2 over GPT-4o (53.6%).
MMLU-Pro (Maverick, hard knowledge questions): 80.5%. +2.9 over Gemini 2.0 Flash (77.6%), −0.7 under DeepSeek V3-0324 (81.2%).
LiveCodeBench (Maverick, fresh coding problems): 43.4%. +8.9 over Gemini 2.0 Flash (34.5%), −2.4 under DeepSeek V3-0324 (45.8%), +11.1 over GPT-4o (32.3%).
MMMU (Maverick, college-level image reasoning): 73.4%. +1.7 over Gemini 2.0 Flash (71.7%), +4.3 over GPT-4o (69.1%).
MathVista (Maverick, visual math problems): 73.7%. +0.6 over Gemini 2.0 Flash (73.1%), +9.9 over GPT-4o (63.8%).
ChartQA (Maverick, reading charts): 90.0%. +1.7 over Gemini 2.0 Flash (88.3%), +4.3 over GPT-4o (85.7%).
MMMLU (Maverick, multilingual knowledge): 84.6%. +3.1 over GPT-4o (81.5%).
GPQA Diamond (Scout, PhD-level science questions): 57.2%. −12.6 under Llama 4 Maverick (69.8%).
MMLU-Pro (Scout, hard knowledge questions): 74.3%. −6.2 under Llama 4 Maverick (80.5%).
LiveCodeBench (Scout, fresh coding problems): 32.8%. −10.6 under Llama 4 Maverick (43.4%).
MMMU (Scout, college-level image reasoning): 69.4%. −4.0 under Llama 4 Maverick (73.4%).
MathVista (Scout, visual math problems): 70.7%. −3.0 under Llama 4 Maverick (73.7%).
ChartQA (Scout, reading charts): 88.8%. −1.2 under Llama 4 Maverick (90.0%).

Independent numbers

Artificial Analysis Intelligence Index (current v4.3.2, AA estimate, Maverick): 10.0. +1.6 over GPT-4o (8.4), +1.1 over Gemini 2.0 Flash (8.9), +0.3 over DeepSeek V3-0324 (9.7).
Aider Polyglot (Maverick, multi-language code edits): 15.6%. −39.5 under DeepSeek V3-0324 (55.1%), −2.6 under GPT-4o (18.2%).
Epoch Capabilities Index (Maverick, Epoch AI composite): 132. +0 over DeepSeek V3-0324 (132), −3 under Gemini 2.0 Flash (135), +3 over GPT-4o (129).