The benchmark board

Every score from our Model Drops on one board, since Dec 5, 2024: who leads, how fast the best scores move, and which new models jumped furthest past the last generation.

172benchmarks

114models scored

1597scores logged

114Model Drops

Best model overall

Handy AI's weigh-in: every leaderboard on this page folded into one ranking.

Best score over time

Each dot is a model at its release. The line is the record holder at each point. Hollow dots are models without a drop of their own, placed where we first saw them cited.

Closed weights only

Artificial Analysis Intelligence Index

Best score rose 42.8, from 15.2 (o1) to 58 (Claude Opus 5.5).

Closed weights only

GPQA Diamond

Best score rose 18.1 points, from 78% (o1) to 96.1% (GPT-6 Astra).

Closed weights only

SWE-bench Verified

Best score rose 44.9 points, from 49% (Claude 3.5 Sonnet) to 93.9% (Claude Mythos Preview).

Closed weights only

Humanity's Last Exam (no tools)

Best score rose 59.3 points, from 5.1% (Gemini 2.0 Flash) to 64.4% (Claude Opus 5.5).

Closed weights only

DeepSWE v1.1

Best score rose 12 points, from 67% (GPT-5.5) to 79% (Muse Wonder Full).

Closed weights only

AIME 2025

Best score rose 20.6 points, from 79.2% (o1) to 99.8% (Claude Opus 4.6).

Closed weights only

LiveCodeBench

Best score rose 26.8 points, from 63.9% (o1) to 90.7% (Gemini 3 Pro).

Closed weights only

Terminal-Bench 2.0

Best score rose 49.4 points, from 32.6% (Gemini 2.5 Pro) to 82% (Claude Mythos Preview).

Biggest generational jumps

A new model against the model it replaced, on the same chart in its launch post.

GPT-4.1GPT-5

τ²-bench Telecom: 34% to 96.7%

+62.7

Claude Sonnet 5Claude Sonnet 5.5

Terminal-Bench 4.0: 10.3% to 70.6%

+60.3

Claude Haiku 4.5Claude Haiku 5.5

OSWorld 2 (offline subset): 15.7% to 72.4%

+56.7

Gemini 2.0 FlashGemini 2.5 Flash

AIME 2025: 27.5% to 78%

+50.5

GPT-5.6 TerraGPT-6.1 Terra

AA-Omniscience hallucination rate: 88% to 38%

+50

Claude Sonnet 4Claude Sonnet 4.5

τ²-bench Telecom: 49.6% to 98%

+48.4

Claude 3.5 SonnetClaude 3.7 Sonnet

AIME 2024: 16% to 61.3%

+45.3

Claude Sonnet 4.5Claude Sonnet 4.6

ARC-AGI-2: 13.6% to 58.3%

+44.7

Leaderboards

The latest reported score for every model on each benchmark. Tap a card's share button for an image of it.

Closed weights only

Artificial Analysis Intelligence Index

76 models compared.

Claude Opus 5.5 58

Claude Sonnet 5.5max 56

Muse Wonder Fullmax 55.8

GPT-6 Astra 53

Fable 5.1 53

Gemini 4 Argonhigh 53

70 more

Muse Wondermax 52.4

GPT-6.1 Solmax 52

Claude Opus 5 50.8

Fable 5 49.6

Muse Spark 1.3 48.1

GPT-6.1 Terramax 48

GPT-5.6 Sol 47

Grok 4.7 46

Grok 4.6 44

Claude Haiku 5.5max 43

GPT-5.6 Terramax 42

Claude Opus 4.8 41.8

Gemini 3.8 Flash 40.9

Qwen3.8-Max 40.2

Muse Spark 1.2 39.6

Gemini 3.7 Flash 39.1

GPT-5.4 39

Grok 4.5high 38.8

Claude Sonnet 5max 38.2

GPT-6 Lunamax 38

GPT-5.6 Lunamax 37.3

Gemini 3.6 Flash 34

GPT-5.3-Codex 32.5

Claude Opus 4.6max 31.9

Muse Spark 31.3

GPT-5.2xhigh 30.4

Claude Sonnet 4.6 30.1

Gemini 3.1 Pro 29.7

Claude Opus 4.5 29.1

GPT-5.2-Codex 28.5

Gemini 3 Pro 28

Gemini 3 Flash 26.3

GPT-5-Codex 24.9

GPT-5.1 24.7

GPT-5high 23

Claude Opus 4.1 22.8

Grok 4 22.5

o3-pro 21.9

Claude Sonnet 4.5thinking 20.7

Claude Opus 4 20.6

Grok 4.1 Fastreasoning 20.4

o3 20.2

Claude Sonnet 4 18.9

Grok 4 Fast 17.9

Muse Glimmer-30B 17.5

Claude Haiku 4.5 16.9

GPT-5 minihigh 16.8

o4-minihigh 16.7

Gemini 2.5 Pro 16.1

Qwen3-Max 15.6

Claude 3.7 Sonnet 15.3

o1 15.2

Gemini 2.5 Pro Preview 14.5

Grok Code Fast 1 14.1

Gemini 2.5 Flash 13.1

GPT-4.1 12.7

o1-pro 12.4

o3-mini 11

Grok 3 10.4

GPT-4.1 mini 10.2

o1-mini 9.8

GPT-4.5 9.6

Gemini 2.0 Flash Thinking 9.4

Gemini 2.0 Flash 8.9

Claude 3.5 Haiku 8.9

Gemini 2.0 Pro 8.7

GPT-4o 8.4

Claude 3.5 Sonnet 7.9

Gemini 1.5 Pro 7.9

GPT-4o mini 6.7

76 models · 75 drops · Kimi K3.1 Pro, GPT-6.1 Terra, Muse Wonder, DeepSeek V4.1 Flash 1017, Grok 4.6, Muse Spark 1.3, Grok 4.7, Qwen3.8-Max, Gemini 3.8 Flash, GLM-5.3, Muse Spark 1.2, Gemini 3.7 Flash, Muse Spark, Gemini 3.6 Flash, GPT-5.3-Codex, Gemini 3.1 Pro, Claude Sonnet 4.6, GLM-5, Gemini 3 Flash, GPT-5-Codex, GPT-5.1, DeepSeek V3.2, Claude Sonnet 4.5, Gemini 3 Pro, o3-pro, Grok 4, Claude Opus 4.1, Claude Opus 4 & Sonnet 4, Claude Haiku 4.5, Qwen3.8, gpt-oss-120b & gpt-oss-20b, Qwen3-Max, Mistral Medium 3, o3 & o4-mini, o3-mini, Gemini 2.5 Pro, GPT-4.5, Gemini 2.5 Flash, Gemini 2.0 Flash, Grok 3, GPT-4.1, DeepSeek V3, o1

Closed weights only

GPQA Diamond

57 models compared.

GPT-6 Astramax 96.1%

Claude Mythos Previewmax 94.5%

Gemini 3.1 Prohigh 94.3%

GPT-5.6 Solmax 94.1%

Claude Opus 5.5max 93.9%

Fable 5.1max 93.7%

51 more

GPT-5.2 Pro 93.2%

GPT-5.4xhigh 92.8%

GPT-5.3-Codex 92.6%

Qwen3.8-Max 92.6%

Fable 5max 92.6%

GPT-5.2xhigh 92.4%

Claude Opus 4.8max 92%

Gemini 3 Prohigh 91.9%

Claude Opus 4.6max 91.3%

Gemini 3 Flashxhigh 90.4%

Particle-1 90.4%

Claude Sonnet 4.6max 89.9%

Muse Spark 89.5%

GPT-5.1high 88.1%

GPT-5.1 Thinkinghigh 88.1%

Grok 4 87.5%

Claude Opus 4.5high 87%

Gemini 2.5 Pro 86.4%

GPT-5high 85.7%

Grok 4 Fast 85.7%

MAI-Code-1-Flash 84.6%

Grok 4.1 Fast 84.3%

MAI-Thinking-1 84.2%

o3-pro 84%

Muse Glimmer-30Bmax 83.5%

Claude Sonnet 4.5xhigh 83.4%

o3 83.3%

Gemini 2.5 Pro Preview 83%

GPT-5 minihigh 82.3%

o4-minihigh 81.4%

Claude Opus 4.1 81%

Grok 3 80.2%

Grok 3 Beta 80.2%

Claude Opus 4 79.6%

o1-pro 79%

Grok 3 Minihigh 79%

Gemini 2.5 Flash 78.3%

Claude 3.7 Sonnet 78.2%

o1 78%

o3-mini 77%

Claude Sonnet 4 76.1%

o1-preview 73.3%

Claude Haiku 4.5 73.2%

GPT-4.5 69.5%

GPT-4.1 66.3%

Claude 3.5 Sonnet 65%

Gemini 2.0 Flash 60.1%

o1-mini 60%

Gemini 1.5 Pro 59.1%

GPT-4o 52.5%

Gemini 1.5 Flash 51%

57 models · 59 drops · Kimi K3.1 Pro, Claude Mythos Preview, Muse Spark, Qwen3.8-Max, Claude Sonnet 4.6, MAI-Thinking-1, GPT-5.4, Qwen3.8, Gemini 3 Flash, Particle, "Introducing Particle-1", Claude Opus 4.5, GPT-5.2, Grok 4 Fast, GLM-5, Gemini 3 Pro, DeepSeek V3.2, MAI-Code-1-Flash, o3-pro, GPT-5, Claude Opus 4 & Sonnet 4, gpt-oss-120b & gpt-oss-20b, Claude Sonnet 4.5, Gemini 2.5 Pro, Qwen3, Claude Opus 4.1, Gemini 2.5 Flash, Claude Haiku 4.5, o1, GPT-4.1, Claude 3.7 Sonnet, Llama 4 Maverick & Scout, o3-mini, Gemini 2.0 Flash, Mistral Medium 3

Closed weights only

Sentiment Bench

How people received the model in its first week on X, Hacker News, in the news and among trusted reviewers (Handy AI).

Claude Opus 4.5 91.5

Grok 4.6 87

Gemini 4 Carbon 85.8

GPT-5.3-Codex 85.6

Claude Opus 5.5 85.2

Gemini 3 Flash 84.5

50 more

Composer 84.1

Gemini 3 Pro 82.9

Grok 4.5 82.5

GPT-5.5 82.5

Composer 2.5 82.3

GPT-6.1 Sol 80.9

GPT-5.4 79

Gemini 3.7 Flash 77.6

GPT-5.2-Codex 76.7

Composer 2 76.1

Claude Opus 4.6 75.7

Muse Spark 1.1 74

GPT-6 Astra 73.8

Muse Spark 1.3 73.2

Gemini 3.1 Pro 72.3

Claude Haiku 5.5 71.4

Gemini Omni 70.1

Fable 5.1 69.9

Fable 5 69.9

Claude Opus 4.8 69.6

Mythos 5 69.5

GPT-5.1-Codex-Max 69.3

GPT-6.1 Terra 69.2

GPT-6 Luna 69

Claude Sonnet 4.6 67.9

GPT-5.6 Terra 66.5

GPT-5.6 Luna 66.2

MAI-Cyber-1-Flash 65.5

Claude Sonnet 5.5 65.4

Gemini 3.8 Flash 65.3

Claude Mythos Preview 64.8

Muse Spark 64.7

Claude Opus 5 63.9

GPT-Rosalind 62.6

Gemini 4 Argon 61.6

GPT-5.2 61.1

MAI-Code-1-Flash 61

Muse Spark 1.2 59.2

Qwen3.8-Max 56

MAI-Thinking-1 55.6

GPT-6 Sol 55.4

GPT-5.1 50.9

GPT-5.6 Sol 48.2

Grok 4.1 44.5

Gemini 3.5 Flash 43.4

Claude Opus 4.7 43

Gemini 3.6 Flash 39.5

Muse Wonder 37

Grok 4.7 33.4

Claude Sonnet 5 25

56 models · 52 drops · Every model's sources, quotes and method

Closed weights only

SWE-bench Verified

41 models compared.

Claude Mythos Previewmax 93.9%

GPT-6 Astra 86%

Claude Opus 4.5high 80.9%

Claude Opus 4.6max 80.8%

Gemini 3.1 Prohigh 80.6%

GPT-5.2xhigh 80%

35 more

Claude Sonnet 4.6max 79.6%

Gemini 3 Flashxhigh 78%

GPT-5.1-Codex-Maxxhigh 77.9%

Muse Spark 77.4%

Claude Sonnet 4.5xhigh 77.2%

GPT-5.1high 76.3%

GPT-5.1 Thinkinghigh 76.3%

Gemini 3 Proxhigh 76.2%

GPT-5high 74.9%

Claude Opus 4.1 74.5%

GPT-5-Codex 74.5%

GPT-5.1-Codexhigh 73.7%

MAI-Thinking-1 73.5%

Fern 0.1 73.4%

Claude Sonnet 4 72.7%

Claude Opus 4 72.5%

MAI-Code-1-Flash 71.6%

Grok Code Fast 1 70.8%

Qwen3-Max 69.6%

o3high 69.1%

o4-minihigh 68.1%

Claude Haiku 4.5 66.6%

Gemini 2.5 Pro Preview 63.2%

Claude 3.7 Sonnet 62.3%

Gemini 2.5 Flashxhigh 60.4%

Gemini 2.5 Pro 59.6%

GPT-4.1 54.6%

Grok 4.1 Fast 50.6%

o3-minihigh 49.3%

Claude 3.5 Sonnet 49%

o1 48.9%

o1-mini 41.6%

o1-preview 41.3%

GPT-4.5 38%

GPT-4o 33.2%

41 models · 48 drops · Claude Mythos Preview, Mirendil, "Introducing Fern", Qwen3.5, MAI-Thinking-1, Muse Spark, Gemini 3.1 Pro, Gemini 3 Flash, Claude Opus 4.5, Claude Sonnet 4.6, GPT-5.2, DeepSeek V3.2, Claude Sonnet 4.5, Claude Haiku 4.5, GPT-5.1-Codex-Max, Qwen3-Max, MAI-Code-1-Flash, Grok Code Fast 1, GPT-5, gpt-oss-120b & gpt-oss-20b, Claude Opus 4 & Sonnet 4, Gemini 3 Pro, Claude 3.7 Sonnet, o3 & o4-mini, DeepSeek R1, o3-mini, GPT-4.1

Closed weights only

Humanity's Last Exam (no tools)

Expert-level questions, no tools.

Claude Opus 5.5 64.4%

Mythos 5 59%

Fable 5 57.8%

Fable 5.1max 57.6%

Claude Sonnet 5.5 56.9%

Claude Mythos Previewmax 56.8%

37 more

Claude Opus 5 56.3%

GPT-6 Astramax 53.6%

Claude Haiku 5.5max 45.9%

Claude Opus 4.8max 45.7%

GPT-5.6 Sol 44.5%

Gemini 3.1 Pro 44.4%

GPT-5.4xhigh 43.9%

Qwen3.8-Max 43.6%

Muse Spark 42.8%

GPT-5.4 Pro 42.7%

GPT-5.5 41.4%

Claude Opus 4.6max 40%

Gemini 3 Pro 37.5%

GPT-5.2 Pro 36.6%

GPT-5.2xhigh 34.5%

Gemini 3 Flashxhigh 33.7%

Claude Sonnet 4.6max 33.2%

Claude Opus 4.5 30.8%

GPT-5.1 26.5%

GPT-5high 26.3%

GPT-5.1 Thinkinghigh 25.7%

Grok 4 25.4%

Gemini 2.5 Pro 21.6%

o3 20.2%

Grok 4 Fast 20%

Claude Sonnet 4.5 17.7%

Grok 4.1 Fast 17.6%

GPT-5 minihigh 16.7%

o4-mini 14.3%

o3-mini 13.4%

Gemini 2.5 Flash 12.1%

Claude Haiku 4.5 10.2%

Claude 3.7 Sonnet 8.9%

o1-pro 8.1%

GPT-4.5 6.4%

GPT-4.1 5.4%

Gemini 2.0 Flash 5.1%

43 models · 40 drops · Claude Opus 5.5, Fable 5 / Mythos 5, Fable 5.1, Kimi K3.1 Pro, Claude Haiku 5.5, DeepSeek V4.1 Flash 1017, Qwen3.8, Muse Spark, Qwen3.8-Max, GPT-5.4, Claude Sonnet 4.6, Gemini 3 Flash, Gemini 3.1 Pro, Qwen3.5, Gemini 3 Pro, DeepSeek V3.2, GPT-5.2, Kimi K2 Thinking, GPT-5, Grok 4 Fast, Gemini 2.5 Flash, o3 & o4-mini, Gemini 2.5 Pro

Closed weights only

DeepSWE v1.1

Fixing issues in real repositories.

Muse Wonder Fullmax 79%

Muse Wondermax 78.8%

Gemini 4 Argonmax 77.9%

Muse Spark 1.3max 75.4%

GPT-6.1 Solhigh 75.2%

Claude Opus 5.5max 74.2%

24 more

GPT-6 Astramax 74.1%

Claude Opus 5 74%

Gemini 3.8 Flash 73.8%

GPT-5.6 Sol 73%

Particle-1max 72.6%

Grok 4.7high 71%

Fable 5xhigh 69.9%

GPT-5.6 Terramax 69.6%

GPT-6 Solmax 68.8%

Fable 5.1max 67.4%

GPT-5.6 Lunamax 67%

GPT-5.5xhigh 67%

GPT-6 Lunamax 66.6%

Gemini 3.7 Flashhigh 65.3%

Grok 4.6high 65.2%

Claude Opus 4.8max 59%

Qwen3.8-Max 56.6%

Muse Spark 1.2xhigh 55%

Grok 4.5high 54%

Claude Sonnet 5max 53.8%

Gemini 3.6 Flash 48.6%

Fern 0.1 44.2%

Gemini 3.5 Flash 37%

Muse Glimmer-30Bmax 4.4%

30 models · 24 drops · Muse Wonder, Gemini 4 Argon, GPT-6 Astra, GPT-6.1 Sol, Kimi K3.1 Pro, MiniMax M3.1 Flash, Particle, "Introducing Particle-1", Grok 4.7, GPT-6 Sol & GPT-6 Luna, Gemini 3.8 Flash, Gemini 3.6 Flash, GPT-5.6 Sol, Terra, & Luna, Qwen3.8-Max, Muse Spark 1.3, Grok 4.6, Gemini 3.7 Flash, Mirendil, "Introducing Fern", Nanbeige4.3-33B model card

Closed weights only

AIME 2025

28 models compared.

Claude Opus 4.6 99.8%

o4-mini 99.5%

MAI-Thinking-1 97%

Claude Sonnet 4.6max 95.6%

Gemini 3 Pro 95%

GPT-5high 94.6%

22 more

GPT-5.1high 94%

Grok 4 Fast 92%

Grok 4 91.7%

GPT-5 minihigh 91.1%

o3 88.9%

Gemini 2.5 Pro 88%

Claude Sonnet 4.5 87%

Gemini 2.5 Pro Preview 83%

Qwen3-Max 81.6%

Claude Haiku 4.5 80.7%

o1 79.2%

Gemini 2.5 Flash 78%

Claude Opus 4.1 78%

Grok 3 77.3%

Grok 3 Beta 77.3%

o3-minimedium 74.8%

Claude Sonnet 4 70.5%

Claude 3.7 Sonnet 54.8%

Gemini 2.0 Flash Thinking 53.5%

GPT-4.1 46.4%

Claude Opus 4 33.9%

Gemini 2.0 Flash 27.5%

28 models · 24 drops · MAI-Thinking-1, gpt-oss-120b & gpt-oss-20b, DeepSeek V3.2, GPT-5.1, Grok 4 Fast, Kimi K2 Thinking, GPT-5, Claude Sonnet 4.5, Claude Opus 4 & Sonnet 4, Qwen3-Max, Claude Haiku 4.5, Qwen3, Gemini 2.5 Flash, Gemini 2.5 Pro, Grok 3

Closed weights only

LiveCodeBench

27 models compared.

Gemini 3 Pro 90.7%

GPT-5.2 87.7%

MAI-Thinking-1 87.7%

Claude Opus 4.5 84.8%

GPT-5high 84.5%

Grok 4 Fast 80%

21 more

Grok 4 79%

GPT-5 minihigh 77.4%

Gemini 2.5 Pro 74.2%

o3-minihigh 74.1%

o3 72%

Grok 3 70.6%

Grok 3 Beta 70.6%

Qwen3-Max 69%

Claude Sonnet 4.5 64%

o1 63.9%

Gemini 2.5 Flash 63.5%

o1-mini 53.8%

Gemini 2.0 Flash Thinking 45.8%

GPT-4.1 44.7%

Claude Opus 4 44.6%

Claude 3.5 Sonnet 36.3%

Claude 3.7 Sonnet 36%

Gemini 2.0 Flash 34.5%

Gemini 1.5 Pro 34.3%

GPT-4o 31.4%

Gemini 1.5 Flash 30%

27 models · 24 drops · Qwen3.5, MAI-Thinking-1, DeepSeek V3.2, Grok 4 Fast, Grok 4, Gemini 2.5 Pro, Qwen3, Qwen3-Max, Kimi K2 Thinking, Gemini 2.5 Flash, DeepSeek R1, Grok 3, Kimi K2, DeepSeek V3, Mistral Medium 3, Gemini 2.0 Flash

Closed weights only

Terminal-Bench 2.0

Hard tasks in a real terminal (2025 edition).

Claude Mythos Previewmax 82%

GPT-5.3-Codex 77.3%

GPT-5.4xhigh 75.1%

Gemini 3.1 Prohigh 68.5%

Claude Opus 4.6max 65.4%

GPT-5.2-Codex 64%

18 more

GPT-5.2 62.2%

Composer 2 61.7%

Claude Sonnet 4.6max 59.1%

Muse Spark 59%

GPT-5.1-Codex-Maxxhigh 58.1%

Gemini 3 Pro 56.2%

MAI-Code-1-Flash 54.8%

GPT-5.1-Codexhigh 52.8%

Claude Opus 4.5 52.1%

Claude Sonnet 4.5 51%

GPT-5.1 47.6%

Gemini 3 Flashxhigh 47.6%

MAI-Thinking-1 46%

Claude Haiku 4.5 41.6%

Composer 40%

GPT-5high 35.2%

Gemini 2.5 Pro 32.6%

Gemini 2.5 Flashxhigh 16.9%

24 models · 22 drops · Claude Mythos Preview, GPT-5.4, MAI-Thinking-1, Muse Spark, GPT-5.3-Codex, Composer 2, GPT-5.2-Codex, Claude Sonnet 4.6, MAI-Code-1-Flash, GPT-5.1-Codex-Max, Claude Opus 4.5, Gemini 3 Flash, Composer, DeepSeek V3.2, Gemini 3 Pro

Closed weights only

BrowseComp

27 models compared.

GPT-6 Astramax 91.5%

Claude Opus 5.5max 90.8%

GPT-5.6 Solmax 90.4%

GPT-5.6 Terramax 87.5%

Gemini 3.1 Promax 85.9%

GPT-5.5xhigh 84.4%

21 more

Claude Opus 4.6max 84%

GPT-5.6 Lunaxhigh 84%

GPT-5.4 82.7%

GPT-5.2 Pro 77.9%

GPT-5.3-Codex 77.3%

Claude Sonnet 4.6max 74.7%

ChatGPT agent 68.9%

Claude Opus 4.5 67.8%

GPT-5.2xhigh 65.8%

Gemini 3 Pro 59.2%

Gemini 3 Pro Deep Research 59.2%

GPT-5high 54.9%

deep research 51.5%

GPT-5.1 Thinkinghigh 50.8%

o3high 49.7%

Grok 4 Fast 44.9%

Grok 4 43%

o4-minihigh 28.3%

Claude Sonnet 4.5 24.1%

Claude Opus 4 18.8%

GPT-4o 1.9%

27 models · 21 drops · Kimi K3.1 Pro, GPT-5.6 Sol, Terra, & Luna, Gemini 3.1 Pro, GPT-5.4, Claude Opus 4.6, GPT-5, Qwen3.5, DeepSeek V3.2, o3 & o4-mini, GPT-5.2, Grok 4 Fast, GLM-4.5

Closed weights only

ARC-AGI-2

34 models compared.

GPT-5.6 Sol 92.5%

Claude Opus 5 90.4%

Fable 5 89.2%

Gemini 3.8 Flash 89.2%

Gemini 3.7 Flash 84.6%

GPT-5.6 Terramax 83.9%

28 more

GPT-5.4 Pro 83.3%

Gemini 3.1 Prohigh 76.5%

GPT-5.4xhigh 76.1%

Grok 4.6 65.1%

Claude Opus 4.6max 63.3%

Gemini 3.6 Flash 60.4%

GPT-5.6 Lunamax 59.5%

Claude Sonnet 4.6max 58.3%

GPT-5.2 Prohigh 54.2%

GPT-5.2xhigh 52.9%

Grok 4.5high 52.6%

Gemini 3 Deep Think 45.1%

Muse Spark 42.5%

Claude Opus 4.5 37.6%

Gemini 3 Flashxhigh 33.6%

Gemini 3 Proxhigh 31.1%

GPT-5.1high 17.6%

GPT-5.1 Thinkinghigh 17.6%

Grok 4 16%

Claude Sonnet 4.5xhigh 13.6%

GPT-5 9.9%

Claude Opus 4 8.6%

Claude Sonnet 4 5.9%

Grok 4 Fast 5.3%

Gemini 2.5 Pro 4.9%

GPT-5 minihigh 4.4%

o3 2.9%

o4-minimedium 2.3%

34 models · 20 drops · Grok 4.6, Gemini 3.8 Flash, Gemini 3.7 Flash, GPT-5.4, Muse Spark, Gemini 3.6 Flash, Gemini 3.1 Pro, Claude Sonnet 4.6, Claude Opus 4.6, Gemini 3 Flash, Claude Opus 4.5, GPT-5.2, GPT-5.1, GPT-5, Grok 4, Claude Opus 4 & Sonnet 4, Grok 4 Fast, Gemini 3 Pro, o3 & o4-mini

Closed weights only

Terminal-Bench 2.1

22 models compared.

Gemini 3.8 Flashmax 89.4%

Claude Opus 5 89.1%

GPT-5.6 Solmax 88.8%

Muse Spark 1.3max 88.8%

Mythos 5 88%

GPT-5.6 Terramax 87.4%

16 more

Qwen3.8-Max 86.6%

Gemini 3.7 Flashhigh 85.8%

GPT-5.5xhigh 85.6%

GPT-5.6 Lunamax 84.7%

Claude Opus 4.8max 84.6%

Fable 5max 84.6%

Muse Spark 1.2xhigh 82.9%

Grok 4.5high 81.6%

Claude Sonnet 5max 80.4%

Particle-1max 79.9%

Claude Opus 4.6max 78.2%

Gemini 3.6 Flashmax 78%

Gemini 3.5 Flash 76.2%

Muse Spark 1.1xhigh 76.2%

Gemini 3.1 Pro 70.7%

Muse Glimmer-30Bmax 51.7%

22 models · 20 drops · Gemini 3.8 Flash, MiMo-V2.6-Pro, Muse Spark 1.3, Fable 5 / Mythos 5, Qwen3.8-Max, Gemini 3.7 Flash, GPT-5.6 Sol, Terra, & Luna, Gemini 3.6 Flash, Muse Spark 1.2, Particle, "Introducing Particle-1", Qwen3.8

Closed weights only

GDPval-AA

Professional work products, graded head to head (Elo).

Claude Opus 5.5max 1866 Elo

Muse Wonder Fullmax 1857 Elo

Claude Sonnet 5.5 1838 Elo

Muse Wondermax 1770 Elo

Fable 5.1max 1754 Elo

Muse Spark 1.3max 1754 Elo

25 more

Fable 5 1743 Elo

Claude Opus 5 1708 Elo

Grok 4.7xhigh 1695 Elo

Claude Sonnet 4.6max 1633 Elo

Claude Haiku 5.5max 1620 Elo

Muse Spark 1.2xhigh 1615 Elo

Claude Opus 4.6max 1606 Elo

Grok 4.6high 1605 Elo

GPT-5.6 Sol 1588 Elo

GPT-5.6 Lunamax 1584 Elo

GPT-6 Astramax 1573 Elo

Gemini 3.8 Flashmax 1545 Elo

GPT-5.6 Terramax 1528 Elo

Grok 4.5high 1526 Elo

Gemini 3.7 Flashhigh 1525 Elo

GPT-6 Sol 1487 Elo

GPT-5.2xhigh 1462 Elo

Claude Sonnet 5 1449 Elo

GPT-6 Luna 1437 Elo

Gemini 3.6 Flash 1422 Elo

Claude Opus 4.5 1416 Elo

Gemini 3.5 Flash 1349 Elo

Gemini 3.1 Promax 1317 Elo

Gemini 3 Pro 1201 Elo

Claude Haiku 4.5 735 Elo

31 models · 19 drops · Kimi K3.1 Pro, Muse Wonder, Muse Spark 1.3, GLM-5.3, Claude Opus 5.5, Grok 4.7, Gemini 3.1 Pro, Claude Haiku 5.5, MiMo-V2.6-Pro, Gemini 3.6 Flash, Gemini 3.8 Flash, Grok 4.6, Gemini 3.7 Flash, Claude Sonnet 5.5, Claude Opus 4.6, Claude Sonnet 4.6

Closed weights only

Terminal-Bench 4.0

Multi-hour work in a real terminal.

Claude Sonnet 5.5max 70.6%

Claude Opus 5.5max 66.4%

Mythos 5.1 60.9%

GPT-6 Astramax 58.2%

Fable 5.1max 57.9%

Gemini 4 Argonmax 57.4%

16 more

Muse Wonder Fullmax 57.1%

GPT-6.1 Solmax 55.9%

Muse Wondermax 52%

Claude Opus 5 51.8%

GPT-6.1 Terramax 50.3%

GPT-5.6 Sol 39.9%

Claude Haiku 5.5max 39.2%

Grok 4.7xhigh 37.6%

GPT-5.6 Terramax 21.5%

Grok 4.6high 20.3%

Gemini 3.8 Flashmax 19.1%

GPT-6 Luna 16.4%

Claude Sonnet 5 10.3%

Fern 0.1 6.1%

Muse Glimmer-30Bmax 1.6%

Claude Haiku 4.5 0%

22 models · 19 drops · GPT-6.1 Terra, Kimi K3.1 Pro, GPT-6 Astra, Gemini 4 Argon, Muse Wonder, MiniMax M3.1 Flash, Claude Haiku 5.5, Grok 4.7, Gemini 3.8 Flash, Claude Sonnet 5.5, Mirendil, "Introducing Fern", Nanbeige4.3-33B model card

Closed weights only

ARC-AGI-1

27 models compared.

Gemini 3.1 Pro 98%

GPT-5.6 Terramax 96.5%

Gemini 3.7 Flash 95.5%

Claude Opus 4.6high 94%

Gemini 3.6 Flashhigh 91.2%

GPT-5.2xhigh 86.2%

21 more

Claude Sonnet 4.6max 86%

Gemini 3 Flash 84.7%

Claude Opus 4.5 80%

Gemini 3 Pro 75%

GPT-5.1high 72.8%

Grok 4 66.7%

GPT-5 65.7%

Claude Sonnet 4.5thinking 63.7%

GPT-5 minihigh 54.3%

o3 53%

Grok 4 Fast 48.5%

Claude Haiku 4.5 47.7%

Gemini 2.5 Pro 41%

Claude Opus 4 35.7%

o1high 35%

o3-mini 34.5%

Claude 3.7 Sonnet 28.6%

o1-mini 14%

GPT-4.5 10.3%

GPT-4.1 5.5%

GPT-4o 4.5%

27 models · 17 drops · Gemini 3.1 Pro, Gemini 3.7 Flash, Claude Sonnet 4.6, Gemini 3 Flash, Claude Opus 4.5, Gemini 3 Pro, GPT-5, Claude Haiku 4.5, o3 & o4-mini, Grok 4, DeepSeek R1, o3-mini, Claude 3.7 Sonnet, GPT-4.1

Closed weights only

MMMU

27 models compared.

Gemini 3 Pro 87.2%

GPT-5.2 86.7%

GPT-5.1high 85.4%

GPT-5high 84.2%

o3high 82.9%

Gemini 2.5 Pro 82%

21 more

o4-minihigh 81.6%

Claude Opus 4.5 80.7%

Gemini 2.5 Pro Preview 79.6%

Claude Sonnet 4.5 77.8%

o1 77.6%

Claude Opus 4.1 77.1%

Gemini 2.5 Flash 76.7%

Claude Opus 4 76.5%

Grok 3 76%

Grok 3 Beta 76%

Gemini 2.0 Flash Thinking 75.4%

Claude 3.7 Sonnet 75%

GPT-4.1 74.8%

GPT-4.5 74.4%

Claude Sonnet 4 74.4%

Claude Haiku 4.5 73.2%

Gemini 2.0 Flash 71.7%

Claude 3.5 Sonnet 70.4%

GPT-4o 68.7%

Gemini 1.5 Pro 65.9%

Gemini 1.5 Flash 62.3%

27 models · 17 drops · Qwen3.5, Claude Opus 4.5, GPT-5.1, Claude Opus 4.1, Claude Haiku 4.5, Gemini 2.5 Flash, Claude Opus 4 & Sonnet 4, o3 & o4-mini, Claude Sonnet 4.5, Gemini 2.5 Pro, Grok 3, Claude 3.7 Sonnet, Mistral Medium 3, Gemini 2.0 Flash

Closed weights only

Aider Polyglot

17 models compared.

GPT-5high 88%

o3-pro 84.9%

o3 79.6%

Gemini 2.5 Pro 76.9%

Claude Opus 4 70.7%

o4-mini 68.9%

11 more

Claude 3.7 Sonnet 64.9%

o1high 61.7%

Claude Sonnet 4 61.3%

o3-mini 60.4%

GPT-4.1 52.9%

Claude 3.5 Sonnet 51.6%

Gemini 2.5 Flash 51.1%

GPT-4.5 44.9%

o1-mini 32.9%

Gemini 2.0 Flash 22.2%

GPT-4o 18.2%

17 models · 17 drops · GPT-5, o3-pro, DeepSeek R1 0528, Kimi K2, Gemini 2.5 Flash, Gemini 2.5 Pro, GPT-4.1, Claude Opus 4 & Sonnet 4, o3-mini, Claude 3.7 Sonnet, DeepSeek R1

Closed weights only

AutomationBench

Real business automations (Zapier).

Gemini 4 Argonmax 51.3%

Claude Opus 5 50.3%

Muse Spark 1.3max 49.6%

Fable 5 46.2%

GPT-5.6 Sol 45.8%

Claude Sonnet 5.5max 44.7%

11 more

Claude Opus 5.5 42.5%

GPT-6 Astramax 41.4%

GPT-6.1 Solmax 36.1%

GPT-6 Sol 33.2%

Fable 5.1max 31.4%

Gemini 3.7 Flashhigh 30.4%

GPT-5.6 Terra 23.6%

GPT-6 Lunamax 20.7%

Gemini 3.6 Flash 17%

Muse Glimmer-30Bmax 15.2%

Claude Sonnet 5 10.7%

17 models · 16 drops · Gemini 4 Argon, MiniMax M3.1 Flash, Muse Spark 1.3, GLM-5.3, Claude Sonnet 5.5, GPT-6.1 Sol, Gemini 3.7 Flash, GPT-6 Sol & GPT-6 Luna, Nanbeige4.3-33B model card

Closed weights only

SWE-bench Pro

22 models compared.

Fable 5max 80%

Claude Mythos Previewmax 77.8%

Claude Opus 4.8max 69.2%

Qwen3.8-Max 67.7%

Grok 4.5max 64.7%

GPT-5.6 Solmax 64.6%

16 more

Claude Sonnet 5max 63.2%

GPT-5.6 Lunamax 62.7%

Gemini 3.6 Flashmax 58.7%

GPT-5.4 57.7%

GPT-5.3-Codex 56.8%

GPT-5.2-Codex 56.4%

GPT-5.2xhigh 55.6%

Gemini 3.5 Flash 55.1%

Gemini 3.1 Promax 54.2%

Claude Opus 4.6max 53.4%

MAI-Thinking-1 52.8%

MAI-Code-1-Flash 51.2%

Muse Glimmer-30Bmax 51.2%

GPT-5.1 Thinkinghigh 50.8%

GPT-5.1 50.8%

Claude Haiku 4.5 35.2%

22 models · 15 drops · Qwen3.8, Claude Mythos Preview, Qwen3.8-Max, Gemini 3.6 Flash, MAI-Thinking-1, GPT-5.4, GPT-5.3-Codex, MAI-Code-1-Flash, GPT-5.2, GPT-5.2-Codex

Closed weights only

Humanity's Last Exam (with tools)

Expert-level questions, with search and code.

Claude Opus 5.5 67.7%

Fable 5.1max 65.6%

Claude Mythos Preview 64.7%

Mythos 5 64.5%

GPT-5.6 Sol 64.5%

Fable 5 63.9%

14 more

Claude Opus 5 63.6%

Claude Opus 4.8max 57.9%

GPT-6 Astramax 57.2%

Qwen3.8-Max 56.2%

GPT-5.5 52.2%

Gemini 3.1 Pro 51.4%

Gemini 3 Pro 45.8%

GPT-5.2xhigh 45.5%

Claude Opus 4.5 43.2%

GPT-5high 41.7%

Grok 4 41%

Claude Sonnet 4.5 32%

o3 24.9%

o4-mini 17.7%

20 models · 14 drops · Claude Opus 5.5, Kimi K3.1 Pro, Fable 5 / Mythos 5, GLM-5.3, Qwen3.8-Max, Kimi K2.5, Kimi K2 Thinking, gpt-oss-120b & gpt-oss-20b

Closed weights only

AIME 2024

15 models compared.

Grok 4 94.3%

o3-pro 93%

o3 90.3%

Gemini 2.5 Pro 88.7%

o3-minihigh 87.3%

o1-pro 86%

9 more

Grok 3 83.9%

Grok 3 Beta 83.9%

o1 74.3%

o1-mini 63.6%

Claude 3.7 Sonnet 61.3%

o1-preview 42%

GPT-4.5 36.7%

Claude 3.5 Sonnet 16%

GPT-4o 9.3%

15 models · 12 drops · GLM-4.5, o3-pro, GPT-4.5, Grok 3, Qwen3, o3-mini, Claude 3.7 Sonnet, o1

Closed weights only

SimpleQA

17 models compared.

Grok 4 Fast 95%

Grok 4 94%

Grok 3 82%

GPT-4.5 62.5%

Gemini 2.5 Pro 52.9%

o1 47%

11 more

Gemini 2.0 Pro 44.3%

Grok 3 Beta 43.6%

GPT-4.1 42.3%

o1-preview 42%

GPT-4o 38.2%

Gemini 2.0 Flash 29.9%

Gemini 2.5 Flash 29.7%

Claude 3.5 Sonnet 28.4%

Claude Opus 4 22.8%

o3-minihigh 13.8%

o1-mini 7.6%

17 models · 11 drops · Grok 4 Fast, Gemini 2.5 Pro, GPT-4.5, Grok 3, Gemini 2.5 Flash, Kimi K2, o1, DeepSeek V3, o3-mini

Closed weights only

τ²-bench Telecom

12 models compared.

GPT-5.2xhigh 98.7%

Claude Sonnet 4.5 98%

GPT-5high 96.7%

GPT-5.1high 95.6%

GPT-5.1 Thinkinghigh 95.6%

MAI-Code-1-Flash 71.7%

6 more

Claude Opus 4.1 71.5%

o3 58.2%

Claude Opus 4 57%

Claude Haiku 4.5 54.7%

Claude Sonnet 4 49.6%

GPT-4.1 34%

12 models · 11 drops · GPT-5.2, Claude Haiku 4.5, GPT-5.1, MAI-Code-1-Flash, Claude Sonnet 4.5, GPT-5, Kimi K2

Closed weights only

OSWorld-Verified

Using a computer: apps, files and the web (2025 edition).

Claude Opus 5 83.4%

Gemini 3.6 Flashmax 83%

GPT-5.6 Sol 83%

Claude Sonnet 5max 81.2%

Claude Mythos Previewmax 79.6%

Gemini 3.5 Flash 78.4%

13 more

GPT-5.4 75%

GPT-5.3-Codex 74%

Claude Opus 4.6max 72.7%

GPT-5.6 Lunamax 72.6%

Claude Sonnet 4.6max 72.5%

Claude Opus 4.5 66.3%

Muse Glimmer-30Bmax 65.9%

Claude Sonnet 4.5 61.4%

Claude Haiku 4.5 50.7%

GPT-5.2 47.3%

Claude Opus 4.1 44.4%

Claude Sonnet 4 42.2%

GPT-5.2-Codex 38.2%

19 models · 10 drops · MiMo-V2.6-Pro, Gemini 3.6 Flash, Claude Mythos Preview, GPT-5.4, Qwen3.8, Claude Sonnet 4.6, Claude Haiku 4.5, Claude Opus 4.5, GPT-5.3-Codex

Closed weights only

MMLU-Pro

13 models compared.

Claude Opus 4 87.3%

Claude Sonnet 4.6max 87%

Gemini 2.5 Pro 86.2%

o3 85.3%

MAI-Thinking-1 85%

Claude 3.7 Sonnet 80%

7 more

Grok 3no thinking 79.9%

Gemini 2.0 Pro 79.1%

Claude 3.5 Sonnet 78%

Gemini 2.0 Flash 77.6%

Gemini 1.5 Pro 75.8%

GPT-4o 75.8%

Gemini 1.5 Flash 67.3%

13 models · 9 drops · GLM-4.5, MAI-Thinking-1, Mistral Medium 3, Grok 3, Llama 4 Maverick & Scout, Gemini 2.0 Flash

Closed weights only

LMArena text Elo

23 models compared.

Grok 4.1 1483

Gemini 3 Flash 1470

Gemini 2.5 Pro 1452

GPT-5.1 1441

o3 1427

Grok 4 Fast 1425

17 more

Grok 4 1422

Claude Opus 4.1 1419

Claude Sonnet 4.5 1419

GPT-5.2high 1419

GPT-5high 1410

Grok 4.1 Fastreasoning 1410

Gemini 2.5 Flash 1407

Grok 3 1404

GPT-4.5 1398

GPT-5 minihigh 1393

Gemini 2.0 Flash Thinking 1385

Claude Haiku 4.5 1384

GPT-4o 1377

Gemini 2.0 Flash 1355

Claude 3.7 Sonnetthinking 1304

GPT-4o mini 1273

Claude 3.5 Haiku 1238

23 models · 8 drops · Grok 4.1, Gemini 3 Flash, Claude Opus 4.1, Grok 4 Fast, Claude Sonnet 4.5, Gemini 2.5 Pro, Grok 3, Gemini 2.0 Flash

Closed weights only

OSWorld 2

Using a computer: apps, files and the web.

Claude Sonnet 5.5max 83.9%

Claude Opus 5.5 81.8%

GPT-6.1 Solmax 78%

GPT-6.1 Terramax 74.2%

GPT-6 Astra 72.6%

Claude Opus 5 70.6%

6 more

Muse Spark 1.3max 66.9%

GPT-5.6 Sol 65.7%

Gemini 3.8 Flashmax 59%

GPT-5.6 Terramax 57.1%

Claude Sonnet 5 57%

GPT-6 Lunamax 55.5%

12 models · 8 drops · GPT-6.1 Terra, Claude Sonnet 5.5, GPT-6 Astra, MiniMax M3.1 Flash, Muse Spark 1.3, Gemini 3.8 Flash

Closed weights only

Terminal-Bench

Tasks in a real terminal (first edition).

Claude Sonnet 4.5 51%

Claude Opus 4.1 46.5%

GPT-5high 43.8%

Claude Haiku 4.5 41%

Claude Opus 4 39.2%

Claude Sonnet 4 36.4%

5 more

Claude 3.7 Sonnet 35.2%

GPT-4.1 30.3%

o3 30.2%

Gemini 2.5 Pro Preview 25.3%

Gemini 2.5 Pro 25.3%

11 models · 8 drops · Kimi K2 Thinking, Claude Sonnet 4.5, Claude Haiku 4.5, Claude Opus 4.1, Claude Opus 4 & Sonnet 4

Closed weights only

IFBench

7 models compared.

Qwen3.8-Max 82.8%

GPT-5.2 75.4%

GPT-5.6 Solmax 72.7%

Gemini 3 Pro 70.4%

Fable 5max 63.5%

Claude Opus 4.8max 62.2%

1 more

Claude Opus 4.5 58%

7 models · 8 drops · Qwen3.8-Max, Qwen3.5

Closed weights only

Harvey Legal Agent Benchmark

Legal work end to end.

Grok 4.7xhigh 19.6%

Gemini 4 Argonmax 19.6%

Fable 5.1max 19.1%

Grok 4.6high 15.8%

Muse Wonder Fullmax 13.1%

Grok 4.5high 12.9%

10 more

Fable 5max 11.3%

Qwen3.8 Max 10.4%

Gemini 3.8 Flashmax 10%

Muse Wondermax 10%

GPT-6 Astramax 8.6%

Claude Opus 5max 6.7%

Claude Sonnet 5max 5%

Claude Opus 5.5max 4.2%

GPT-5.6 Solmax 2.5%

GPT-5.6 Terramax 0.8%

16 models · 7 drops · Grok 4.7, Gemini 4 Argon, Kimi K3.1 Pro, Muse Wonder, Grok 4.6, Mistral Large 4, Gemini 3.8 Flash

Closed weights only

MMMLU

13 models compared.

Claude Mythos Previewmax 92.7%

Claude Opus 4.6 91.1%

Claude Opus 4.1 89.5%

Claude Opus 4 88.8%

o3 88.8%

o1 87.7%

7 more

Claude Sonnet 4 86.5%

Claude 3.7 Sonnet 85.9%

GPT-4.5 85.1%

GPT-4.1 83.7%

Claude 3.5 Sonnet 82.1%

GPT-4o 81.5%

o3-minihigh 81.1%

13 models · 6 drops · Claude Mythos Preview, Claude Opus 4.1, Claude 3.7 Sonnet, Claude Opus 4 & Sonnet 4, GPT-4.5, Llama 4 Maverick & Scout

Closed weights only

MCP Atlas

10 models compared.

Muse Spark 1.2xhigh 90.3%

Claude Opus 5xhigh 85.8%

Fable 5 83.3%

GPT-5.6 Sol 81.8%

GPT-5.2xhigh 68%

Gemini 3 Prohigh 66.6%

4 more

Claude Opus 4.5 62.3%

Gemini 3 Flashxhigh 57.4%

Claude Sonnet 4.5xhigh 43.8%

Claude Opus 4.1 40.9%

10 models · 6 drops · Muse Spark 1.2, GLM-5, Claude Opus 4.5, Gemini 3 Flash

Closed weights only

TAU-bench Retail

10 models compared.

Claude Opus 4.1 82.4%

Claude Opus 4 81.4%

Claude 3.7 Sonnet 81.2%

Claude Sonnet 4 80.5%

Claude 3.5 Sonnet 71.5%

o1 70.8%

4 more

o3high 70.4%

GPT-4.1 68%

o4-minihigh 65.6%

o3-mini 57.6%

10 models · 6 drops · Claude Opus 4.1, Claude Opus 4 & Sonnet 4, Claude 3.7 Sonnet, o3 & o4-mini, gpt-oss-120b & gpt-oss-20b

Closed weights only

Terminal-Bench Science

Scientific computing tasks in a terminal.

GPT-6 Astramax 68.1%

Claude Opus 5.5max 63.3%

GPT-6.1 Terramax 58.3%

GPT-6.1 Solmax 57%

Fable 5.1max 40%

Claude Opus 5 29%

3 more

GPT-6 Sol 27.6%

Fable 5 24.7%

GPT-5.6 Sol 22.4%

9 models · 6 drops · GPT-6.1 Terra, Kimi K3.1 Pro, Claude Opus 5.5, GPT-6.1 Sol, Fable 5.1

Closed weights only

Epoch Capabilities Index

66 models compared.

GPT-6 Astra 169.1

Claude Opus 5.5 168.4

GPT-6.1 Terra 168.1

GPT-6.1 Sol 167.8

Claude Sonnet 5.5 166.7

Fable 5.1 166.4

60 more

Gemini 4 Argon 166.1

Claude Opus 5 164.5

GPT-6 Sol 163.8

Fable 5 163

GPT-5.6 Sol 162.6

Muse Wonder 161.7

Claude Haiku 5.5 160.4

GPT-5.6 Terra 160.1

GPT-5.4 Pro 159.8

Claude Opus 4.8 159.6

GPT-5.5 159.4

Gemini 3.7 Flash 158.2

Gemini 3.8 Flash 157.6

Muse Spark 1.3 157.4

GPT-5.4 157.1

Claude Sonnet 5 157

Grok 4.6 156.9

GPT-6 Luna 156.8

Claude Opus 4.7 156.7

GPT-5.6 Luna 156.7

GPT-5.2 Pro 156.5

Qwen3.8-Max 156.2

Claude Opus 4.6 155.9

Gemini 3.1 Pro 155.2

Muse Spark 1.2 155.2

Gemini 3.5 Flash 155

Gemini 3.6 Flash 154.7

Grok 4.7 154.6

Grok 4.5 154.4

Muse Spark 1.1 154.2

GPT-5.2 153.8

Gemini 3 Pro 153.2

Claude Sonnet 4.6 152.9

Muse Spark 152.2

Gemini 3 Flash 152

Claude Opus 4.5 150.2

GPT-5 150

GPT-5.1 149.9

o3-pro 147.4

o3 146.9

Claude Sonnet 4.5 146.8

Grok 4 146.3

o4-mini 145.6

Grok 4 Fast 144.2

Claude Opus 4.1 143.9

Claude Opus 4 142.4

Qwen3-Max 142.4

Gemini 2.5 Pro 142.2

Claude Haiku 4.5 142.2

o1 141.8

Claude Sonnet 4 141.6

Claude 3.7 Sonnet 141

o3-mini 140.3

Gemini 2.5 Flash 139.9

Grok 3 138.3

GPT-4.1 136.7

GPT-4.5 136.7

GPT-4.1 mini 135

Gemini 2.0 Flash 134.7

GPT-4o 129

66 models · 5 drops · Epoch Capabilities Index (2026-11-04), Epoch Capabilities Index (Nov 3), Mistral Medium 3, Llama 4 Maverick & Scout

Closed weights only

AA-LCR

16 models compared.

Fable 5.1max 85.3%

Claude Opus 5.5max 84.7%

Muse Wonder Fullmax 83.7%

Muse Glimmer-30Bmax 83.3%

GPT-6 Lunamax 83.3%

GPT-6.1 Solmax 83%

10 more

GPT-6.1 Terramax 82.7%

Claude Sonnet 5.5max 82.7%

Claude Haiku 5.5max 82.7%

Gemini 3.8 Flashhigh 81.3%

Muse Wondermax 80.7%

GPT-6 Astramax 80.7%

Qwen3.8-Max 80.3%

Gemini 4 Argonhigh 79.7%

Gemini 4 Carbonhigh 79.3%

Grok 4.7xhigh 76.7%

16 models · 5 drops · Artificial Analysis (2026-11-04), Nanbeige4.3-33B model card

Closed weights only

Codeforces

8 models compared.

o3-pro 2748

o4-minihigh 2719

o3high 2706

o3-minimedium 2036

Gemini 2.5 Pro 2001

o1 1891

2 more

o1-mini 1820

o1-pro 1707

8 models · 5 drops · o3-pro, gpt-oss-120b & gpt-oss-20b, Qwen3, DeepSeek R1

Closed weights only

CyberGym

7 models compared.

MAI-Cyber-1-Flash 96%

GPT-5.5 Cyber 85.6%

Mythos 5 83.8%

GPT-5.6 Sol 83.6%

Gemini 3.5 Flash Cyber 83.2%

Claude Mythos Previewmax 83.1%

1 more

Claude Opus 4.6 66.6%

7 models · 5 drops · MAI-Cyber-1-Flash, GLM-5.3, MiniMax M3.1 Flash, Claude Mythos Preview

Closed weights only

ProgramBench

7 models compared.

Claude Opus 5.5max 65%

Muse Wonder Fullmax 59%

Fable 5.1max 53.5%

Muse Wondermax 50%

GPT-6 Astramax 50%

Claude Opus 5 37%

1 more

GPT-5.6 Sol 23%

7 models · 5 drops · Kimi K3.1 Pro, Muse Wonder, DeepSeek V4.1 Flash 1017

Closed weights only

SWE-bench Multilingual

7 models compared.

GPT-5.4high 76.8%

Claude Opus 4.6high 75.8%

Composer 2 73.7%

Claude Sonnet 4.5 68%

MAI-Code-1-Flash 65.5%

Claude Haiku 4.5 62.7%

1 more

Composer 56.9%

7 models · 5 drops · Composer 2, MiniMax-M2, MAI-Code-1-Flash, Composer

Closed weights only

MATH-500

5 models compared.

o1 96.4%

o1-mini 90%

Claude 3.7 Sonnet 83%

Claude 3.5 Sonnet 78.3%

GPT-4o 76.4%

5 models · 5 drops · DeepSeek R1, Mistral Medium 3, DeepSeek V3

Closed weights only

OSWorld 2 (offline subset)

Using a computer, on the 82 OSWorld 2.1 tasks that need no internet.

Claude Opus 5.5max 87.2%

Claude Sonnet 5.5 83.9%

Gemini 4 Carbonmax 81.5%

GPT-6 Astramax 80.9%

Fable 5.1max 76.8%

Gemini 4 Argonmax 74.7%

7 more

Claude Haiku 5.5max 72.4%

GPT-6.1 Solmax 71.4%

Claude Opus 5max 70.2%

GPT-5.6 Sol 65.7%

GPT-6 Solmax 64.4%

GPT-6 Luna 48.9%

Claude Haiku 4.5 15.7%

13 models · 4 drops · Gemini 4 Carbon, Claude Haiku 5.5, GPT-6.1 Sol, GPT-6 Sol & GPT-6 Luna

Closed weights only

AA-Omniscience hallucination rate

Lower is better.

Muse Wonder Fullmax 23%

Muse Wondermax 26.5%

Grok 4.7 29%

Grok 4.6 34%

GPT-6.1 Terra 38%

Claude Sonnet 5.5 47%

4 more

GPT-6 Astra 51.3%

Claude Opus 5.5 58.6%

Fable 5.1 72.6%

GPT-5.6 Terra 88%

10 models · 4 drops · Muse Wonder, Grok 4.7, GPT-6.1 Terra

Closed weights only

CharXiv Reasoning

10 models compared.

Muse Spark 86.4%

GPT-5.6 Terra 85.9%

Gemini 3.6 Flashmax 85.2%

Gemini 3.7 Flashhigh 84.5%

GPT-5.4xhigh 82.8%

GPT-5.6 Lunamax 82.7%

4 more

Grok 4.5max 81.6%

Gemini 3.1 Prohigh 80.2%

Claude Sonnet 5max 77%

Claude Opus 4.6max 65.3%

10 models · 4 drops · Muse Spark, Gemini 3.7 Flash, Gemini 3.6 Flash

Closed weights only

MMMU-Pro

9 models compared.

Claude Opus 5.5max 87.7%

GPT-6 Astramax 86.9%

Fable 5.1max 86.3%

Gemini 3 Flashxhigh 81.2%

Gemini 3 Proxhigh 81%

GPT-5.2xhigh 79.5%

3 more

Claude Opus 4.5 74%

Claude Sonnet 4.5xhigh 68%

Grok 4.1 Fast 63%

9 models · 4 drops · Kimi K3.1 Pro, Gemini 3 Flash, Kimi K2.5

Closed weights only

MMLU

7 models compared.

o3 93.4%

o4-mini 93%

o1 91.8%

o1-preview 90.8%

o3-mini 87%

Particle-1 86.1%

1 more

o1-mini 85.2%

7 models · 4 drops · gpt-oss-120b & gpt-oss-20b, o1, Particle, "Introducing Particle-1", o3-mini

Closed weights only

τ²-bench Retail

7 models compared.

Claude Opus 4.1 86.8%

Claude Sonnet 4.5 86.2%

Claude Sonnet 4 83.8%

Claude Opus 4 81.8%

GPT-5high 81.1%

GPT-5.1high 77.9%

1 more

GPT-4.1 74.8%

7 models · 4 drops · Claude Sonnet 4.5, Kimi K2, GPT-5.1

Closed weights only

NL2Repo

6 models compared.

Claude Opus 5 75.3%

GPT-5.6 Sol 56.8%

Claude Opus 4.6 49.8%

GPT-5.4 41.3%

Gemini 3.1 Pro 33.4%

Muse Glimmer-30Bmax 27.4%

6 models · 4 drops · MiniMax M3.1 Flash, GLM-5.1, Nanbeige4.3-33B model card

Closed weights only

IFEval

5 models compared.

Claude 3.7 Sonnet 93.2%

Claude 3.5 Sonnet 90.2%

GPT-4.5 88.2%

GPT-4.1 87.4%

GPT-4o 81%

5 models · 4 drops · Claude 3.7 Sonnet, GPT-4.1

Closed weights only

Agents' Last Exam

4 models compared.

Claude Opus 5.5max 42.6%

GPT-6 Astramax 32.8%

Claude Opus 5 32.2%

GPT-5.6 Sol 30.6%

4 models · 4 drops · Kimi K3.1 Pro, MiniMax M3.1 Flash

Closed weights only

AIME 2026

3 models compared.

Muse Glimmer-30Bmax 94.7%

MAI-Code-1-Flash 92.5%

Claude Haiku 4.5 83.3%

3 models · 4 drops · Nanbeige4.3-33B model card, MAI-Code-1-Flash

Closed weights only

JobBench

3 models compared.

Claude Opus 5max 65.7%

Muse Spark 1.3max 64.9%

GPT-5.6 Solmax 45.4%

3 models · 4 drops · MiMo-V2.6-Pro, Muse Spark 1.3

Closed weights only

Terminal-Bench 4.0 (standard harness)

Terminal-Bench run by a third party on one shared harness.

Gemini 4 Carbonhigh 65.2%

Claude Sonnet 5.5max 63.6%

Claude Opus 5.5max 59.6%

GPT-6 Astramax 59.1%

Gemini 4 Argonhigh 57.1%

Muse Wonder Fullmax 57.1%

9 more

GPT-6.1 Solmax 56.1%

GPT-6.1 Terramax 55.6%

Fable 5.1max 52%

Muse Wondermax 52%

Qwen3.8-Max 38.9%

Claude Haiku 5.5max 32.8%

Grok 4.7xhigh 25.8%

Gemini 3.8 Flashhigh 19.7%

GPT-6 Lunamax 12.6%

15 models · 3 drops · Artificial Analysis (2026-11-04)

Closed weights only

GDP.pdf

Work products built from PDFs (Surge AI).

GPT-6 Astramax 34.2%

Gemini 3.7 Flashhigh 34%

GPT-6.1 Solhigh 32%

Claude Opus 5.5max 30.6%

Claude Sonnet 5 28%

GPT-6 Sol 28%

4 more

Fable 5.1max 27.6%

GPT-5.6 Terra 24.7%

Gemini 3.6 Flash 22%

Muse Spark 1.2 16%

10 models · 3 drops · Kimi K3.1 Pro, Gemini 3.7 Flash, GPT-6.1 Sol

Closed weights only

FrontierCode v1.1

Hard, long coding tasks.

Claude Opus 5.5 54.4%

GPT-6 Astra 53.3%

Claude Sonnet 5.5xhigh 52.1%

Fable 5.1 50.3%

GPT-6 Sol 49.3%

Claude Opus 5 48%

3 more

Claude Haiku 5.5max 46.4%

Claude Sonnet 5 42.4%

GPT-6 Luna 42.4%

9 models · 3 drops · Claude Sonnet 5.5, Claude Opus 5.5, Claude Haiku 5.5

Closed weights only

GPQA

9 models compared.

GPT-5.2 92.4%

Gemini 3 Pro 91.9%

Grok 4 87.7%

Claude Opus 4.5 87%

Gemini 2.5 Pro 84.4%

o3 82.7%

3 more

o3-minihigh 79.7%

GPT-4.5 71.4%

GPT-4o 53.6%

9 models · 3 drops · Qwen3.5, GLM-4.5, GPT-4.5

Closed weights only

Terminal-Bench 3.0

9 models compared.

Claude Opus 5 43.5%

GPT-5.6 Solmax 34.6%

Fable 5 33.7%

Grok 4.6high 26%

GPT-5.6 Terra 20.8%

Grok 4.5high 15.7%

3 more

Gemini 3.7 Flashhigh 14.9%

Claude Sonnet 5 14.6%

Gemini 3.6 Flash 5.4%

9 models · 3 drops · Grok 4.6, GLM-5.3, Gemini 3.7 Flash

Closed weights only

ExploitBench

8 models compared.

GPT-6 Astramax 100%

Claude Opus 5.5max 91%

Fable 5.1max 83%

Mythos 5 78%

GPT-5.6 Sol 76.5%

Claude Mythos Preview 69%

2 more

Claude Opus 4.8 40%

GPT-5.5 34%

8 models · 3 drops · Kimi K3.1 Pro, GLM-5.3, Fable 5 / Mythos 5

Closed weights only

BabyVision (with tools)

6 models compared.

Claude Opus 5.5max 97.7%

GPT-6 Astramax 96.6%

Fable 5.1max 95.1%

Claude Opus 5 94.1%

Particle-1 89.2%

GPT-5.6 Sol 88.9%

6 models · 3 drops · Kimi K3.1 Pro, DeepSeek V4.1 Flash 1017, Particle, "Introducing Particle-1"

Closed weights only

ZeroBench (with tools)

5 models compared.

Claude Opus 5.5max 61%

Fable 5.1max 59%

GPT-6 Astramax 58%

GPT-5.6 Sol 53%

Claude Opus 5 52%

5 models · 3 drops · Kimi K3.1 Pro, DeepSeek V4.1 Flash 1017

Closed weights only

GDPval

Work products judged against industry experts (win or tie rate).

GPT-5.4 83%

GPT-5.2xhigh 70.9%

GPT-5.3-Codex 70.9%

GPT-5xhigh 38.8%

4 models · 3 drops · GPT-5.4, GPT-5.2

Closed weights only

AA-Briefcase v1.1

17 models compared.

Claude Sonnet 5.5max 1813 Elo

Claude Opus 5.5max 1800 Elo

Muse Wonder Fullmax 1729 Elo

Fable 5.1max 1669 Elo

Muse Wondermax 1659 Elo

Grok 4.7xhigh 1637 Elo

11 more

Gemini 4 Carbonhigh 1610 Elo

Qwen3.8-Max 1607 Elo

Claude Haiku 5.5max 1574 Elo

GPT-6 Astramax 1569 Elo

GPT-6.1 Solmax 1555 Elo

Grok 4.6high 1546 Elo

GPT-5.6 Solmax 1487 Elo

Gemini 4 Argonhigh 1484 Elo

GPT-6.1 Terramax 1401 Elo

GPT-6 Lunamax 1337 Elo

Gemini 3.8 Flashhigh 1203 Elo

17 models · 2 drops · Artificial Analysis (2026-11-04), Grok 4.7

Closed weights only

AutomationBench-AA

Business automations, run by Artificial Analysis.

Gemini 4 Carbonhigh 77.8%

Gemini 4 Argonhigh 77.5%

Claude Sonnet 5.5max 71.8%

Muse Wonder Fullmax 71.3%

Claude Opus 5.5max 69.5%

GPT-6 Astramax 68.5%

9 more

Muse Wondermax 68.1%

Grok 4.7xhigh 65.6%

GPT-6.1 Solmax 64.9%

GPT-6.1 Terramax 60.2%

Gemini 3.8 Flashhigh 59.9%

Fable 5.1max 59.4%

Qwen3.8-Max 56.2%

GPT-6 Lunamax 53.2%

Claude Haiku 5.5max 35.4%

15 models · 2 drops · Artificial Analysis (2026-11-04)

Closed weights only

Agents' Last Exam v1

9 models compared.

Claude Opus 5.5max 63.2%

GPT-6.1 Terramax 60%

GPT-6 Astramax 59.3%

GPT-6.1 Solmax 58.6%

GPT-6 Solmax 56.4%

GPT-5.6 Solmax 52.8%

3 more

Claude Opus 5max 52.7%

GPT-6 Lunamax 50.9%

GPT-5.6 Terramax 50.7%

9 models · 2 drops · GPT-6.1 Terra, GPT-6 Sol & GPT-6 Luna

Closed weights only

APEX-Agents

8 models compared.

Claude Opus 5.5max 73.5%

Fable 5.1max 68.6%

GPT-6 Astramax 64.7%

Claude Opus 5 60.6%

Fable 5max 59.2%

Grok 4.6high 57.5%

2 more

GPT-5.6 Solmax 56.7%

Grok 4.5high 47.1%

8 models · 2 drops · Kimi K3.1 Pro, Grok 4.6

Closed weights only

HealthBench Professional

8 models compared.

Mythos 5 66%

Claude Mythos Preview 64.7%

Fable 5.1max 62.1%

GPT-5.6 Solmax 60.5%

Claude Opus 4.8 56.9%

Grok 4.7xhigh 56.7%

2 more

GPT-5.5 51.8%

Grok 4.6high 48.5%

8 models · 2 drops · Fable 5 / Mythos 5, Grok 4.7

Closed weights only

DeepSearchQA

7 models compared.

GPT-5.6 Solmax 93.1%

Claude Opus 5max 90.4%

Muse Spark 1.3max 90.3%

Muse Spark 74.8%

Claude Opus 4.6max 73.7%

GPT-5.4xhigh 73.6%

1 more

Gemini 3.1 Prohigh 69.7%

7 models · 2 drops · Muse Spark 1.3, Muse Spark

Closed weights only

HLE-Verified

7 models compared.

Gemini 3.8 Flashmax 54.9%

GPT-5.6 Solmax 54.5%

Claude Opus 5max 54.4%

Gemini 3.7 Flashhigh 53.6%

Gemini 3.6 Flash 51.2%

GPT-5.6 Terramax 51.1%

1 more

Claude Sonnet 5max 31%

7 models · 2 drops · Gemini 3.8 Flash, Gemini 3.7 Flash

Closed weights only

HMMT 2025

7 models compared.

Grok 4 Fast 93.3%

GPT-5high 93.3%

Grok 4 90%

GPT-5 minihigh 87.8%

Gemini 2.5 Pro 82.5%

o3 77.5%

1 more

Claude Opus 4 58.3%

7 models · 2 drops · Grok 4 Fast, Grok 4

Closed weights only

SimpleQA Verified

7 models compared.

Gemini 3 Pro 72.1%

Gemini 3 Flashxhigh 68.7%

Gemini 2.5 Pro 54.5%

GPT-5.2xhigh 38%

GPT-5.1 34.9%

Claude Sonnet 4.5xhigh 29.3%

1 more

Grok 4.1 Fast 19.5%

7 models · 2 drops · Gemini 3 Pro, Gemini 3 Flash

Closed weights only

Terminal-Bench 2.1 (standard harness)

7 models compared.

Claude Opus 5 89.1%

GPT-5.6 Terra 88%

GPT-5.6 Sol 88%

Gemini 3.8 Flashhigh 87.6%

Muse Spark 1.3xhigh 85.4%

Grok 4.5 81.6%

1 more

Muse Spark 1.2 80.1%

7 models · 2 drops · Muse Spark 1.3, Muse Spark 1.2

Closed weights only

CursorBench 4.0

Long coding tasks inside Cursor.

Claude Opus 5.5 57.8%

Fable 5.1max 51.8%

Claude Opus 5 46.6%

Grok 4.7xhigh 46.3%

GPT-5.6 Solmax 41.7%

Grok 4.6high 40.4%

6 models · 2 drops · Claude Opus 5.5, Grok 4.7

Closed weights only

OSWorld

6 models compared.

Claude Opus 5.5max 81.8%

Fable 5.1max 80.7%

GPT-6 Astramax 75.8%

Claude Sonnet 4.5 61.4%

Claude Opus 4.1 44.4%

Claude Sonnet 4 42.2%

6 models · 2 drops · Kimi K3.1 Pro, Claude Sonnet 4.5

Closed weights only

TAU-bench Airline

6 models compared.

Claude Sonnet 4 60%

Claude Opus 4 59.6%

Claude 3.7 Sonnet 58.4%

Claude Opus 4.1 56%

o3 52%

GPT-4.1 49.4%

6 models · 2 drops · Claude Opus 4.1, Claude Opus 4 & Sonnet 4

Closed weights only

Video-MMMU

6 models compared.

Gemini 3 Pro 87.6%

GPT-5.2xhigh 85.9%

Claude Opus 4.5 84.4%

Gemini 2.5 Pro 83.6%

GPT-5.1 80.4%

Claude Sonnet 4.5 77.8%

6 models · 2 drops · Kimi K2.5, Gemini 3 Pro

Closed weights only

AA-Briefcase

5 models compared.

Claude Opus 5 1715 Elo

Grok 4.6high 1577 Elo

Fable 5max 1574 Elo

GPT-5.6 Solmax 1502 Elo

Grok 4.5high 1313 Elo

5 models · 2 drops · Grok 4.6

Closed weights only

Artificial Analysis Coding Agent Index

5 models compared.

Claude Opus 5.5 62

Gemini 4 Carbon 59

Grok 4.7 56

Gemini 4 Argon 53

Grok 4.6 47

5 models · 2 drops · Gemini 4 Carbon, Grok 4.7

Closed weights only

Chartography (with tools)

5 models compared.

GPT-6 Astramax 91%

Claude Opus 5.5max 89%

Fable 5.1max 88.4%

Claude Opus 5 84%

GPT-5.6 Sol 79.9%

5 models · 2 drops · Kimi K3.1 Pro, DeepSeek V4.1 Flash 1017

Closed weights only

CursorBench

5 models compared.

GPT-5.4high 63.9%

Composer 2 61.3%

GPT-5.3-Codex 59.1%

Claude Opus 4.6high 58.2%

Composer 38%

5 models · 2 drops · Composer 2, Composer

Closed weights only

FrontierMath

5 models compared.

GPT-5.1high 26.7%

GPT-5high 26.3%

o3-minihigh 9.2%

o1-mini 5.8%

o1 5.5%

5 models · 2 drops · GPT-5.1, o3-mini

Closed weights only

FrontierSWE v2

Frontier software engineering tasks.

GPT-6 Astramax 65.5%

Claude Opus 5.5max 62.3%

Gemini 4 Carbonmax 61.2%

Fable 5.1max 56.3%

Gemini 4 Argonmax 55%

5 models · 2 drops · Gemini 4 Carbon

Closed weights only

Professional Capture-the-Flag Challenges

5 models compared.

GPT-5.2-Codex 88.1%

GPT-5.2 82.4%

GPT-5.1-Codex-Max 75.6%

GPT-5-Codex 50.5%

GPT-5.1xhigh 43%

5 models · 2 drops · GPT-5.2-Codex, GPT-5.1-Codex-Max

Closed weights only

SWE-Lancer IC Diamond

5 models compared.

GPT-5.3-Codex 81.4%

GPT-5.1-Codex-Maxxhigh 79.9%

GPT-5.2-Codex 76%

GPT-5.2high 74.6%

GPT-5.1-Codexhigh 66.3%

5 models · 2 drops · GPT-5.3-Codex, GPT-5.1-Codex-Max

Closed weights only

SWE-Marathon

5 models compared.

Muse Wonder Fullxhigh 57.5%

Muse Wondermax 53.1%

Claude Opus 5.5max 46.3%

Fable 5.1max 45.6%

GPT-6 Astramax 42.5%

5 models · 2 drops · Muse Wonder, Kimi K3.1 Pro

Closed weights only

Agents' Last Exam (CLI)

4 models compared.

Claude Opus 5.5max 40.7%

GPT-6 Astramax 38.8%

GPT-5.6 Sol 28.6%

Fable 5 23.8%

4 models · 2 drops · Kimi K3.1 Pro, GLM-5.3

Closed weights only

MRCR (1M)

4 models compared.

Gemini 1.5 Pro 82.6%

Gemini 1.5 Flash 71.9%

Gemini 2.5 Flash 66.3%

Gemini 2.0 Flash 48.2%

4 models · 2 drops · Gemini 2.0 Flash, Gemini 2.5 Flash

Closed weights only

PaperBench-10

4 models compared.

GPT-5.2-Codex 43%

GPT-5.1-Codex-Maxxhigh 40%

GPT-5.2 39%

GPT-5.1xhigh 34%

4 models · 2 drops · GPT-5.2-Codex, GPT-5.1-Codex-Max

Closed weights only

Vals Index

4 models compared.

Gemini 4 Argonmax 68.9%

Claude Opus 5.5 67%

Fable 5.1 65.8%

GPT-6 Astra 63.1%

4 models · 2 drops · Gemini 4 Argon

Closed weights only

BFCL v3

3 models compared.

o1 67.8%

o3-minimedium 64.6%

Gemini 2.5 Pro 62.9%

3 models · 2 drops · Qwen3

Closed weights only

CyBench

3 models compared.

Grok 4 43%

Grok 4.1 39%

Grok Code Fast 1 22.5%

3 models · 2 drops · Grok 4.1, Grok Code Fast 1

Closed weights only

LiveBench Coding

3 models compared.

o3-minihigh 84.6%

o1 67.4%

o1-preview 52.3%

3 models · 2 drops · o3-mini, o1

Closed weights only

MASK dishonesty rate

Lower is better.

Grok 4 43%

Grok 4.1 49%

Grok Code Fast 1 71.9%

3 models · 2 drops · Grok 4.1, Grok Code Fast 1

Closed weights only

Multi-IF

3 models compared.

Gemini 2.5 Pro 77.8%

o1 48.8%

o3-minimedium 48.4%

3 models · 2 drops · Qwen3

Closed weights only

Toolathlon-Verified

3 models compared.

Claude Opus 5 80.6%

GPT-5.6 Sol 74.9%

Muse Glimmer-30Bmax 47.2%

3 models · 2 drops · MiMo-V2.6-Pro, Nanbeige4.3-33B model card

Closed weights only

SWE-Atlas-QnA

2 models compared.

Fern 0.1 53.2%

Muse Glimmer-30Bmax 17.7%

2 models · 2 drops · Mirendil, "Introducing Fern", Nanbeige4.3-33B model card

From a single launch

Closed weights only

Artificial Analysis Intelligence Index v5.0

15 models compared.

Gemini 4 Carbonhigh 60

Claude Opus 5.5max 59.1

GPT-6 Astramax 56.9

GPT-6.1 Solmax 54.6

Claude Sonnet 5.5max 54.5

GPT-6.1 Terramax 51.3

9 more

Gemini 4 Argonhigh 50.4

Muse Wonder Fullmax 50.1

Fable 5.1max 49.2

Muse Wondermax 43.6

Claude Haiku 5.5max 36.5

Grok 4.7xhigh 35.5

Qwen3.8-Max 33.7

Gemini 3.8 Flashhigh 30.8

GPT-6 Lunamax 27.9

15 models · 1 drop · Artificial Analysis (2026-11-04)

Closed weights only

FrontierCode 2.0

Real coding tasks in open-source repos (Cognition), run by Artificial Analysis.

Claude Opus 5.5max 46.4%

GPT-6 Astramax 42.4%

GPT-6.1 Terramax 42.3%

Claude Sonnet 5.5max 41.7%

Gemini 4 Carbonhigh 40.6%

GPT-6.1 Solmax 40.6%

9 more

Muse Wonder Fullmax 37.8%

Claude Haiku 5.5max 34.9%

Fable 5.1max 31.6%

Grok 4.7xhigh 28.7%

Muse Wondermax 27.9%

Gemini 4 Argonhigh 25.7%

GPT-6 Lunamax 25.2%

Qwen3.8-Max 23.4%

Gemini 3.8 Flashhigh 18.7%

15 models · 1 drop · Artificial Analysis (2026-11-04)

Closed weights only

Vals Index v2.2

14 models compared.

Claude Sonnet 5.5 70.9%

Gemini 4 Carbon 70.5%

Claude Opus 5.5 70.2%

Gemini 4 Argon 69.2%

Muse Wonder Full 66.8%

GPT-6 Astra 65%

8 more

Muse Wonder 64.9%

GPT-6.1 Sol 63.9%

GPT-6.1 Terra 60.5%

Qwen3.8-Max 59.1%

Claude Haiku 5.5 58.7%

Grok 4.7 58.6%

Gemini 3.8 Flash 56%

GPT-6 Luna 52.3%

14 models · 1 drop · Vals Index v2.2 (2026-11-04)

Closed weights only

Artificial Analysis Cyber Index

6 models compared.

Grok 4.7xhigh 56.4

GPT-6 Lunamax 52.7

GPT-6 Astramax 33.4

Claude Opus 5.5max 28.7

Gemini 3.8 Flashhigh 25.1

Fable 5.1max 25

6 models · 1 drop · Artificial Analysis (2026-11-04)

Closed weights only

Artificial Analysis Coding Agent Index v2.0

5 models compared.

Claude Opus 5.5max 62

Claude Sonnet 5.5max 59

GPT-6.1 Solmax 54

GPT-6.1 Terramax 48

GPT-6 Lunamax 31

5 models · 1 drop · GPT-6.1 Terra

Closed weights only

Factual error rate

Lower is better.

GPT-6 Astramax 4.2%

GPT-6.1 Terramax 4.3%

GPT-6.1 Solmax 4.4%

GPT-6 Lunamax 18.4%

GPT-5.6 Terramax 22.3%

5 models · 1 drop · GPT-6.1 Terra

Closed weights only

MemoryBench

5 models compared.

Muse Wondermax 85.6%

Muse Wonder Fullmax 85.5%

Claude Opus 5.5max 81.5%

GPT-6 Astramax 81.4%

Fable 5.1max 79%

5 models · 1 drop · Muse Wonder

Closed weights only

MMMU-Pro Vision

5 models compared.

GPT-6 Astramax 86.7%

GPT-6.1 Solmax 86.3%

GPT-6.1 Terramax 85.5%

GPT-5.6 Terramax 81.7%

GPT-6 Lunamax 79.9%

5 models · 1 drop · GPT-6.1 Terra

Closed weights only

Personal Superintelligence Index

5 models compared.

Muse Wonder Fullmax 59.1

Claude Opus 5.5max 55.2

GPT-6 Astramax 54.5

Muse Wondermax 54.3

Fable 5.1max 49.9

5 models · 1 drop · Muse Wonder

Closed weights only

SWE-sweep

5 models compared.

Gemini 4 Carbonmax 9.6%

GPT-6 Astramax 9.5%

Gemini 4 Argonmax 4.3%

Claude Opus 5.5max 3.6%

Fable 5.1max 2.9%

5 models · 1 drop · Gemini 4 Carbon

Closed weights only

Tokens per second

5 models compared.

Grok Code Fast 1 160

Gemini 2.5 Pro 92.4

Claude Sonnet 4 78.7

GPT-5 50.1

Grok 4 40.6

5 models · 1 drop · Grok Code Fast 1

Closed weights only

Vals Finance Agent v2

5 models compared.

Gemini 3.8 Flashmax 61.4%

Claude Opus 5max 58.6%

GPT-5.6 Terramax 54.4%

Claude Sonnet 5max 53.9%

GPT-5.6 Solmax 53.8%

5 models · 1 drop · Gemini 3.8 Flash

Closed weights only

Agents' Last Exam v1.1

4 models compared.

Gemini 4 Carbonmax 77.4%

GPT-6 Astramax 73.1%

Claude Opus 5.5max 71.6%

Gemini 4 Argonmax 69%

4 models · 1 drop · Gemini 4 Carbon

Closed weights only

Chartography

4 models compared.

Claude Sonnet 5.5 61.6%

Claude Haiku 5.5max 46.4%

GPT-6 Luna 29.1%

Claude Haiku 4.5 6.4%

4 models · 1 drop · Claude Haiku 5.5

Closed weights only

CursorBench 3.2

4 models compared.

Fable 5max 70.5%

Grok 4.6high 69.9%

GPT-5.6 Solmax 67.2%

Grok 4.5high 66.7%

4 models · 1 drop · Grok 4.6

Closed weights only

EEBench

4 models compared.

Grok 4.7xhigh 64%

Fable 5.1max 56.4%

Grok 4.6high 53%

GPT-5.6 Solmax 39.4%

4 models · 1 drop · Grok 4.7

Closed weights only

EQ-Bench3

4 models compared.

Grok 4.1 1586

Gemini 2.5 Pro 1460

Claude Opus 4 1304

Grok 4 1206

4 models · 1 drop · Grok 4.1

Closed weights only

GDM-MRCR v2

4 models compared.

Gemini 3.6 Flashmax 91.8%

Grok 4.5max 81.4%

GPT-5.6 Lunamax 74.8%

Claude Sonnet 5max 71.6%

4 models · 1 drop · Gemini 3.6 Flash

Closed weights only

HealthBench Hard

4 models compared.

Muse Spark 42.8%

GPT-5.4xhigh 40.1%

Gemini 3.1 Prohigh 20.6%

Claude Opus 4.6max 14.8%

4 models · 1 drop · Muse Spark

Closed weights only

MathArena Apex

4 models compared.

Gemini 3 Pro 23.4%

Claude Sonnet 4.5 1.6%

GPT-5.1 1%

Gemini 2.5 Pro 0.5%

4 models · 1 drop · Gemini 3 Pro

Closed weights only

Meta Internal Coding Bench

4 models compared.

Claude Opus 5max 79.4%

Muse Spark 1.2xhigh 70.6%

Muse Spark 1.1xhigh 68.3%

GPT-5.6 Terramax 65.4%

4 models · 1 drop · Muse Spark 1.2

Closed weights only

MLE-Bench

4 models compared.

Claude Sonnet 5max 66.9%

Gemini 3.6 Flashmax 63.9%

GPT-5.6 Lunamax 47.6%

Grok 4.5max 43.2%

4 models · 1 drop · Gemini 3.6 Flash

Closed weights only

PaperBench

4 models compared.

Qwen3.8-Max 93%

GPT-5.6 Solmax 90.5%

Fable 5max 88.8%

Claude Opus 4.8max 80.3%

4 models · 1 drop · Qwen3.8-Max

Closed weights only

SEC-Bench Pro

4 models compared.

GPT-5.6 Solmax 71.2%

GPT-5.6 Terramax 57.7%

GPT-5.6 Lunamax 48.9%

GPT-5.5xhigh 45.8%

4 models · 1 drop · GPT-5.6 Sol, Terra, & Luna

Closed weights only

SimpleQA hallucination rate

Lower is better.

GPT-4.5 37.1%

o1 44%

GPT-4o 61.8%

o3-mini 80.3%

4 models · 1 drop · GPT-4.5

Closed weights only

USAMO 2025

4 models compared.

Gemini Deep Think 49.4%

Grok 4 37.5%

Gemini 2.5 Pro 34.5%

o3 21.7%

4 models · 1 drop · Grok 4

Closed weights only

AIME 2024 4/4 reliability

3 models compared.

o3-pro 90%

o3medium 80%

o1-pro 80%

3 models · 1 drop · o3-pro

Closed weights only

ARC-AGI-3

3 models compared.

GPT-6 Astra 99.9%

Claude Opus 5 30.2%

GPT-5.6 Sol 7.8%

3 models · 1 drop · GPT-6 Astra

Closed weights only

BenchCAD (with tools)

3 models compared.

Claude Opus 5.5max 96.2%

GPT-6 Astramax 95.9%

Fable 5.1max 92.6%

3 models · 1 drop · Kimi K3.1 Pro

Closed weights only

BioMysteryBench hard

3 models compared.

Mythos 5 46.1%

Claude Opus 4.8 40%

Claude Mythos Preview 29.6%

3 models · 1 drop · Fable 5 / Mythos 5

Closed weights only

BioMysteryBench human-solved

3 models compared.

Mythos 5 83.9%

Claude Mythos Preview 82.6%

Claude Opus 4.8 80.4%

3 models · 1 drop · Fable 5 / Mythos 5

Closed weights only

Codeforces 4/4 reliability

3 models compared.

o3-pro 2301

o3medium 2011

o1-pro 1423

3 models · 1 drop · o3-pro

Closed weights only

Creative Writing v3

3 models compared.

Grok 4.1 1721.9

o3 1696.4

Claude Sonnet 4.5 1648.7

3 models · 1 drop · Grok 4.1

Closed weights only

CVE-Bench

3 models compared.

GPT-5.2-Codex 87%

GPT-5.1-Codex-Max 80%

GPT-5.2 69%

3 models · 1 drop · GPT-5.2-Codex

Closed weights only

Cyber Range

3 models compared.

GPT-5.1-Codex-Max 81.8%

GPT-5.2-Codex 72.7%

GPT-5.2 63.6%

3 models · 1 drop · GPT-5.2-Codex

Closed weights only

CyberGym Level 1 leaderboard

3 models compared.

Wiz Atlas 90.9%

MAI-Cyber-1-Flash 86.3%

Claude Mythos Preview 83.1%

3 models · 1 drop · MAI-Cyber-1-Flash

Closed weights only

Cybersecurity Capture The Flag Challenges

3 models compared.

GPT-5.3-Codex 77.6%

GPT-5.2high 67.7%

GPT-5.2-Codex 67.4%

3 models · 1 drop · GPT-5.3-Codex

Closed weights only

Frontier Finance

3 models compared.

Fable 5.1max 60.8%

Claude Opus 5.5max 56.7%

GPT-6 Astramax 45%

3 models · 1 drop · Kimi K3.1 Pro

Closed weights only

FrontierMath Tier 4

3 models compared.

GPT-6 Astra 97.6%

Fable 5.1 87.8%

GPT-5.6 Sol 83%

3 models · 1 drop · GPT-6 Astra

Closed weights only

GPQA Diamond 4/4 reliability

3 models compared.

o3-pro 76%

o1-pro 74%

o3medium 67%

3 models · 1 drop · o3-pro

Closed weights only

MATH

3 models compared.

Gemini 2.0 Flash 89.7%

Gemini 1.5 Pro 86.5%

Gemini 1.5 Flash 77.9%

3 models · 1 drop · Gemini 2.0 Flash

Closed weights only

MLE-Bench-30

3 models compared.

GPT-5.1-Codex-Max 17%

GPT-5.2 16%

GPT-5.2-Codex 10%

3 models · 1 drop · GPT-5.2-Codex

Closed weights only

MRCR 512K-1M

3 models compared.

Muse Spark 1.3max 98.1%

GPT-5.6 Solmax 73.8%

Muse Spark 1.2xhigh 55.5%

3 models · 1 drop · Muse Spark 1.3

Closed weights only

MultiChallenge

3 models compared.

GPT-4.5 43.8%

GPT-4.1 38.3%

GPT-4o 27.8%

3 models · 1 drop · GPT-4.1

Closed weights only

Natural2Code

3 models compared.

Gemini 2.0 Flash 92.9%

Gemini 1.5 Pro 85.4%

Gemini 1.5 Flash 79.8%

3 models · 1 drop · Gemini 2.0 Flash

Closed weights only

OfficeQA Pro v2

3 models compared.

Claude Opus 5.5max 62.2%

Fable 5.1max 54.5%

GPT-6 Astramax 54.4%

3 models · 1 drop · Kimi K3.1 Pro

Closed weights only

OpenAI-MRCR

3 models compared.

GPT-4.5 38.5

GPT-4o 31.9

GPT-4.1 2

3 models · 1 drop · GPT-4.1

Closed weights only

SWE-Lancer Diamond

3 models compared.

GPT-4.5 32.6%

GPT-4o 23.3%

o3-minihigh 10.8%

3 models · 1 drop · GPT-4.5

Closed weights only

τ²-bench Airline

3 models compared.

o3 64.8%

GPT-5high 62.6%

GPT-4.1 56%

3 models · 1 drop · GPT-5

Closed weights only

AA-Omniscience

2 models compared.

Gemini 4 Carbon 54

Claude Opus 5.5 48

2 models · 1 drop · Gemini 4 Carbon

Closed weights only

Apex Bench v1.1

2 models compared.

Claude Opus 5 65.8%

GPT-5.6 Sol 54.3%

2 models · 1 drop · DeepSeek V4.1 Flash 1017

Closed weights only

ArenaHard

2 models compared.

GPT-4o 95.4%

Claude 3.7 Sonnet 93.2%

2 models · 1 drop · Mistral Medium 3

Closed weights only

Artifacts Bench

2 models compared.

Claude Haiku 4.5 36.6%

MAI-Code-1-Flash 36.4%

2 models · 1 drop · MAI-Code-1-Flash

Closed weights only

BankerToolBench

2 models compared.

Claude Opus 5 76.2

GPT-5.6 Sol 74.6

2 models · 1 drop · MiniMax M3.1 Flash

Closed weights only

ChartQA

2 models compared.

Gemini 2.0 Flash 88.3%

GPT-4o 85.7%

2 models · 1 drop · Llama 4 Maverick & Scout

Closed weights only

Code refactoring tasks

2 models compared.

GPT-5-Codex 51.3%

GPT-5high 33.9%

2 models · 1 drop · GPT-5-Codex

Closed weights only

Code review comments per PR

2 models compared.

GPT-5high 1.3

GPT-5-Codex 0.9

2 models · 1 drop · GPT-5-Codex

Closed weights only

Code review high-impact comments

2 models compared.

GPT-5-Codex 52.4%

GPT-5high 39.4%

2 models · 1 drop · GPT-5-Codex

Closed weights only

Code review incorrect comments

Lower is better.

GPT-5-Codex 4.4%

GPT-5high 13.7%

2 models · 1 drop · GPT-5-Codex

Closed weights only

DocVQA

2 models compared.

GPT-4o 85.9%

Claude 3.7 Sonnet 84.3%

2 models · 1 drop · Mistral Medium 3

Closed weights only

FActScore

Lower is better.

Grok 4.1 3%

Grok 4 Fastnon reasoning 9.9%

2 models · 1 drop · Grok 4.1

Closed weights only

Hallucination Rate

Lower is better.

Grok 4.1 4.2%

Grok 4 Fastnon reasoning 12.1%

2 models · 1 drop · Grok 4.1

Closed weights only

HumanEval

2 models compared.

Claude 3.7 Sonnet 92.1%

GPT-4o 91.5%

2 models · 1 drop · Mistral Medium 3

Closed weights only

Irregular medium cyber challenges

2 models compared.

GPT-5xhigh 11

GPT-5.1-Codex-Maxxhigh 9

2 models · 1 drop · GPT-5.1-Codex-Max

Closed weights only

MathVista

2 models compared.

Gemini 2.0 Flash 73.1%

GPT-4o 63.8%

2 models · 1 drop · Llama 4 Maverick & Scout

Closed weights only

MGSM

2 models compared.

o1-preview 90.8%

o1 89.3%

2 models · 1 drop · o1

Closed weights only

OpenAI PRs

2 models compared.

GPT-5.1-Codex-Maxxhigh 53%

GPT-5.1xhigh 45%

2 models · 1 drop · GPT-5.1-Codex-Max

Closed weights only

SuperGPQA

2 models compared.

Qwen3-Max 65.1%

Claude Opus 4 56.5%

2 models · 1 drop · Qwen3-Max

Closed weights only

SWE-bench Pro v2 Hard

2 models compared.

Claude Opus 5.5 100%

Fern 0.1 76.5%

2 models · 1 drop · Mirendil, "Introducing Fern"

Closed weights only

Sycophancy rate

Lower is better.

Grok 4 7%

Grok 4.1 19%

2 models · 1 drop · Grok 4.1

Closed weights only

UK AISI "The Last Ones" cyber range

2 models compared.

Claude Mythos Preview 22

Claude Opus 4.6 16

2 models · 1 drop · Claude Mythos Preview

Closed weights only

Video-MME

2 models compared.

GPT-4.1 72%

GPT-4o 65.3%

2 models · 1 drop · GPT-4.1

Closed weights only

WMDP Cyber

2 models compared.

Grok 4 79%

Grok Code Fast 1 62.1%

2 models · 1 drop · Grok Code Fast 1

Closed weights only

τ²-Bench

2 models compared.

Claude Sonnet 4.5 84.7%

GPT-5 80.1%

2 models · 1 drop · MiniMax-M2

Closed weights only

τ²-Bench (Weighted)

2 models compared.

Qwen3-Max 74.8%

Claude Opus 4 67.7%

2 models · 1 drop · Qwen3-Max

Closed weights only

CritPt v2 Hard

Unpublished research-level physics problems, hard subset, run by Artificial Analysis.

GPT-6 Astramax 70.5%

GPT-6.1 Solmax 69.5%

Gemini 4 Carbonhigh 69%

GPT-6.1 Terramax 68%

Claude Opus 5.5max 68%

Claude Sonnet 5.5max 59%

9 more

Muse Wonder Fullmax 56.5%

Gemini 4 Argonhigh 52.5%

Fable 5.1max 52.5%

Muse Wondermax 50.5%

Gemini 3.8 Flashhigh 46.5%

Claude Haiku 5.5max 43.5%

GPT-6 Lunamax 42.5%

Grok 4.7xhigh 33.5%

Qwen3.8-Max 29.5%

15 models · 0 drops · Artificial Analysis (2026-11-04)

Closed weights only

Terminal-Bench Science (standard harness)

Terminal-Bench Science run by a third party on one shared harness.

Gemini 4 Carbonhigh 64.8%

GPT-6 Astramax 63.3%

Claude Opus 5.5max 59%

GPT-6.1 Solmax 58.1%

GPT-6.1 Terramax 57.6%

Claude Sonnet 5.5max 53.3%

9 more

Gemini 4 Argonhigh 52.9%

Fable 5.1max 43.3%

Muse Wonder Fullmax 37.1%

Muse Wondermax 25.7%

Claude Haiku 5.5max 20%

Qwen3.8-Max 11.9%

Grok 4.7xhigh 11.4%

Gemini 3.8 Flashhigh 10%

GPT-6 Lunamax 8.6%

15 models · 0 drops · Artificial Analysis (2026-11-04)

Release pace

Every Model Drop by month, marked with its maker.

2 Dec 2024

1 Jan 2025

3 Feb 2025

1 Mar 2025

3 Apr 2025

1 May 2025

1 Jun 2025

1 Jul 2025

3 Aug 2025

4 Sep 2025

2 Oct 2025

5 Nov 2025

3 Dec 2025

0 Jan 2026

4 Feb 2026

2 Mar 2026

6 Apr 2026

5 May 2026

4 Jun 2026

6 Jul 2026

4 Aug 2026

9 Sep 2026

7 Oct 2026

How this board works

Scores come from the benchmark section of each handyai.news Model Drop, mostly the makers' launch tables. When drops disagree, the newest drop wins, since later launches re-run rivals on the current version of a benchmark; the model's own launch number is kept as launch_score. Settings (effort level, harness) vary between makers, so compare within a board with care. Models without their own drop are dated by the first drop that cites them.

The data is open: /api/benchmarks (JSON), /benchmarks.md (Markdown), one benchmark at /api/benchmarks/{benchmark}, the best model overall at /api/benchmarks/weigh-in (or /weigh-in/{type} for a topic), a PNG of any card by adding .png, and the list_benchmarks, get_benchmark and get_weigh_in tools on our MCP server.