The benchmark board

Every score from our Model Drops on one board, since Dec 5, 2024: who leads, how fast the best scores move, and which new models jumped furthest past the last generation.

194benchmarks
195models scored
2536scores logged
114Model Drops

Best model overall

Handy AI's weigh-in: every leaderboard on this page folded into one ranking.

Biggest generational jumps

A new model against the model it replaced, on the same chart in its launch post.

Gemma 3 27BGemma 4
τ²-bench Retail: 6.6% to 86.4%
+79.8
GPT-4.1GPT-5
τ²-bench Telecom: 34% to 96.7%
+62.7
Claude Sonnet 5Claude Sonnet 5.5
Terminal-Bench 4.0: 10.3% to 70.6%
+60.3
Claude Haiku 4.5Claude Haiku 5.5
OSWorld 2 (offline subset): 15.7% to 72.4%
+56.7
Gemini 2.0 FlashGemini 2.5 Flash
AIME 2025: 27.5% to 78%
+50.5
GPT-5.6 TerraGPT-6.1 Terra
AA-Omniscience hallucination rate: 88% to 38%
+50
Claude Sonnet 4Claude Sonnet 4.5
τ²-bench Telecom: 49.6% to 98%
+48.4
Claude 3.5 SonnetClaude 3.7 Sonnet
AIME 2024: 16% to 61.3%
+45.3

Leaderboards

The latest reported score for every model on each benchmark. Tap a card's share button for an image of it.

Sentiment Bench

How people received the model in its first week on X, Hacker News, in the news and among trusted reviewers (Handy AI).

Claude Opus 4.5 91.5
GLM 5.2 88
Grok 4.6 87
Gemini 4 Carbon 85.8
GPT-5.3-Codex 85.6
Claude Opus 5.5 85.2
77 more
Gemini 3 Flash 84.5
Composer 84.1
DeepSeek V4.1 Flash 1017 83.9
Kimi K2.5 83.7
Qwen3.8 83.6
Gemini 3 Pro 82.9
Grok 4.5 82.5
Kimi K2.7 Code 82.5
GPT-5.5 82.5
Composer 2.5 82.3
Kimi K2.6 82.3
Gemma 4 82.3
GPT-6.1 Sol 80.9
Kimi K3 80.3
Kimi K2 Thinking 79.6
GPT-5.4 79
GLM-5.3 78.4
Gemini 3.7 Flash 77.6
Nemotron 3.5 Lightning 77.3
MiniMax-M2 77.3
GPT-5.2-Codex 76.7
Composer 2 76.1
Claude Opus 4.6 75.7
DeepSeek V3.2 74.8
GLM-5.1 74.3
Muse Spark 1.1 74
DeepSeek V4 74
GPT-6 Astra 73.8
Muse Spark 1.3 73.2
Gemini 3.1 Pro 72.3
Nemotron 3 Super 71.6
Claude Haiku 5.5 71.4
MiMo-V2.6-Pro 71.3
Nemotron 3 Ultra 70.7
Gemini Omni 70.1
Fable 5.1 69.9
Fable 5 69.9
Claude Opus 4.8 69.6
Mythos 5 69.5
GPT-5.1-Codex-Max 69.3
GPT-6.1 Terra 69.2
GPT-6 Luna 69
GLM-5 68.5
Nemotron 3.5 Super VL 68.2
Nemotron 3 Nano 68
Claude Sonnet 4.6 67.9
Qwen3.5 67.9
Kimi K3.1 Pro 67.5
GPT-5.6 Terra 66.5
GPT-5.6 Luna 66.2
MAI-Cyber-1-Flash 65.5
Claude Sonnet 5.5 65.4
Gemini 3.8 Flash 65.3
Claude Mythos Preview 64.8
Muse Spark 64.7
Claude Opus 5 63.9
GPT-Rosalind 62.6
Gemini 4 Argon 61.6
MiniMax M3.1 Flash 61.1
GPT-5.2 61.1
MAI-Code-1-Flash 61
Muse Spark 1.2 59.2
Qwen3.8-Max 56
MAI-Thinking-1 55.6
GPT-6 Sol 55.4
Muse Spark Open 52.7
GPT-5.1 50.9
Mistral Large 4 49.4
GPT-5.6 Sol 48.2
Inkling 48.1
Grok 4.1 44.5
Gemini 3.5 Flash 43.4
Claude Opus 4.7 43
Gemini 3.6 Flash 39.5
Muse Wonder 37
Grok 4.7 33.4
Claude Sonnet 5 25

83 models · 79 drops · Every model's sources, quotes and method

Artificial Analysis Intelligence Index

138 models compared.

Claude Opus 5.5 58
Claude Sonnet 5.5max 56
Muse Wonder Fullmax 55.8
Kimi K3.1 Pro 55
GPT-6 Astra 53
Fable 5.1 53
132 more
Gemini 4 Argonhigh 53
Muse Wondermax 52.4
GPT-6.1 Solmax 52
Claude Opus 5 50.8
Fable 5 49.6
Muse Spark 1.3 48.1
GPT-6.1 Terramax 48
GPT-5.6 Sol 47
MiMo-V2.6-Pro 46.3
Grok 4.7 46
DeepSeek V4.1 Flash 1017 45.1
GLM-5.3 44.8
Kimi K3 44
Grok 4.6 44
Claude Haiku 5.5max 43
MiniMax M3.1 Flash 42.7
GPT-5.6 Terramax 42
Claude Opus 4.8 41.8
Gemini 3.8 Flash 40.9
Qwen3.8-Max 40.2
Qwen3.8-2.4T-A95B 39.9
Muse Spark 1.2 39.6
DeepSeek V4.1 Flash 39.5
Gemini 3.7 Flash 39.1
GPT-5.4 39
Muse Spark Open 38.9
Grok 4.5high 38.8
Mistral Large 4 38.4
Claude Sonnet 5max 38.2
GPT-6 Lunamax 38
GPT-5.6 Lunamax 37.3
DeepSeek V4 Pro 0813 36
Gemini 3.6 Flash 34
GLM-5.2 33.7
Qwen3.8 33.7
GPT-5.3-Codex 32.5
Claude Opus 4.6max 31.9
Muse Spark 31.3
GPT-5.2xhigh 30.4
DeepSeek V4 Pro 30.4
Claude Sonnet 4.6 30.1
Gemini 3.1 Pro 29.7
Claude Opus 4.5 29.1
GPT-5.2-Codex 28.5
Gemini 3 Pro 28
GLM-5 27.9
Kimi K2.6 27
Gemini 3 Flash 26.3
GLM-5.1 26.1
Inkling 25
GPT-5-Codex 24.9
GPT-5.1 24.7
Kimi K2.5 23.5
GPT-5high 23
Qwen3.5 27B 22.9
Nemotron 3 Ultra 22.9
Claude Opus 4.1 22.8
MiniMax M2.7 22.8
Grok 4 22.5
GLM-4.7 22.2
Kimi K2 Thinking 22
o3-pro 21.9
DeepSeek V3.2 21.5
Qwen3.6-27B 21.4
MiniMax-M2.1 20.9
Claude Sonnet 4.5thinking 20.7
Claude Opus 4 20.6
Grok 4.1 Fastreasoning 20.4
o3 20.2
Claude Sonnet 4 18.9
MiniMax-M2 18.6
GLM-4.6 18.5
Qwen3.5 18.4
Qwen3.5 397B 18.4
Qwen3.6 35B A3B 18.2
Grok 4 Fast 17.9
Muse Glimmer-30B 17.5
Claude Haiku 4.5 16.9
GPT-5 minihigh 16.8
o4-minihigh 16.7
Gemma 4 26B A4B 16.7
DeepSeek V3.2-Exp 16.6
Gemini 2.5 Pro 16.1
Qwen3-Max 15.6
Qwen3.5-122B-A10B 15.6
Claude 3.7 Sonnet 15.3
Kimi K2 0905 15.3
o1 15.2
Gemma 4 14.7
Gemini 2.5 Pro Preview 14.5
Grok Code Fast 1 14.1
DeepSeek V3.1 13.5
Gemini 2.5 Flash 13.1
DeepSeek R1 0528 13.1
Nemotron 3.5 Lightning 12.9
GLM-4.5 12.8
Nemotron 3 Super 12.8
GPT-4.1 12.7
Kimi K2 12.7
Qwen3-235B-A22B-Thinking-2507 12.7
o1-pro 12.4
Qwen3-Coder 11.9
gpt-oss-120b 11.6
DeepSeek R1 11.4
o3-mini 11
Grok 3 10.4
GPT-4.1 mini 10.2
Llama 4 Maverick 10
o1-mini 9.8
Qwen3-30B-A3B-Thinking-2507 9.8
DeepSeek V3-0324 9.7
DeepSeek V3 0324 9.7
GPT-4.5 9.6
Qwen3 9.5
Gemini 2.0 Flash Thinking 9.4
Mistral Medium 3 9
gpt-oss-20bhigh 9
Gemini 2.0 Flash 8.9
Claude 3.5 Haiku 8.9
Nemotron 3 Nano 8.9
Gemini 2.0 Pro 8.7
DeepSeek V3 8.5
GPT-4o 8.4
Qwen3 235B 8.3
Claude 3.5 Sonnet 7.9
Gemini 1.5 Pro 7.9
Qwen2.5 72B 7.7
Llama 3.1 Nemotron Ultra 7.5
Nemotron Nano 2 7.4
Llama 3.1 405B 7.3
GPT-4o mini 6.7
Qwen3-8B 5.2

138 models · 75 drops · Kimi K3.1 Pro, GPT-6.1 Terra, Muse Wonder, DeepSeek V4.1 Flash 1017, Grok 4.6, Muse Spark 1.3, Grok 4.7, Muse Spark Open, Qwen3.8-Max, Gemini 3.8 Flash, GLM-5.3, Muse Spark 1.2, Gemini 3.7 Flash, Muse Spark, Gemini 3.6 Flash, Mistral Large 4, Qwen3.8, GPT-5.3-Codex, Gemini 3.1 Pro, Claude Sonnet 4.6, GLM-5, Qwen3.5, Inkling, Gemini 3 Flash, Nemotron 3 Ultra, GPT-5-Codex, GPT-5.1, GLM-5.1, DeepSeek V3.2, Gemma 4, Claude Sonnet 4.5, Gemini 3 Pro, Kimi K2.5, o3-pro, Grok 4, Claude Opus 4.1, Claude Opus 4 & Sonnet 4, Kimi K2 Thinking, Nemotron 3.5 Lightning, Claude Haiku 4.5, gpt-oss-120b & gpt-oss-20b, Qwen3-Max, Nemotron 3 Super, Mistral Medium 3, o3 & o4-mini, DeepSeek V3.1, Grok Code Fast 1, DeepSeek R1 0528, o3-mini, Gemini 2.5 Pro, Kimi K2, Nemotron 3 Nano, Llama 4 Maverick & Scout, GPT-4.5, Qwen3, Gemini 2.5 Flash, Gemini 2.0 Flash, Grok 3, DeepSeek V3, GPT-4.1, o1, Llama 3.1 Nemotron Ultra, Nemotron Nano 2

GPQA Diamond

107 models compared.

GPT-6 Astramax 96.1%
Claude Mythos Previewmax 94.5%
Gemini 3.1 Prohigh 94.3%
Kimi K3.1 Promax 94.2%
GPT-5.6 Solmax 94.1%
Claude Opus 5.5max 93.9%
101 more
Fable 5.1max 93.7%
Qwen3.8-2.4T-A95Bxhigh 93.5%
Kimi K3max 93.5%
GPT-5.2 Pro 93.2%
GPT-5.4xhigh 92.8%
Qwen3.8-Flash-Next 92.7%
GPT-5.3-Codex 92.6%
Qwen3.8-Max 92.6%
Fable 5max 92.6%
GPT-5.2xhigh 92.4%
DeepSeek V4.1 Flash 1017max 92.2%
Claude Opus 4.8max 92%
Gemini 3 Prohigh 91.9%
GLM-5.3max 91.7%
Claude Opus 4.6max 91.3%
Kimi K2.6 91.1%
Gemini 3 Flashxhigh 90.4%
Particle-1 90.4%
Qwen3.7-Plusmax 90.3%
Claude Sonnet 4.6max 89.9%
Nemotron 3.5 Super VL 89.5%
Muse Spark 89.5%
GLM-5.2 89.5%
Nanbeige4.3-33Bmax 89.5%
Qwen3.8max 89.2%
Muse Spark Openxhigh 89%
GPT-5.1high 88.1%
GPT-5.1 Thinkinghigh 88.1%
Qwen3.6-27Bmax 87.8%
Kimi K2.5 87.6%
Grok 4 87.5%
Inkling 87.2%
Qwen3.5 397B 87.1%
Claude Opus 4.5high 87%
Nemotron 3 Ultra 87%
Qwen3.5-122B-A10B 86.7%
Gemini 2.5 Pro 86.4%
GLM-5.1 86.1%
GLM-5high 86%
GPT-5high 85.7%
Grok 4 Fast 85.7%
MAI-Code-1-Flash 84.6%
Kimi K2 Thinking 84.5%
Grok 4.1 Fast 84.3%
Gemma 4 84.3%
MAI-Thinking-1 84.2%
o3-pro 84%
Clover-35B-A3B-Preview 83.8%
Muse Glimmer-30Bmax 83.5%
Claude Sonnet 4.5xhigh 83.4%
Qwen3.6 35B A3B 83.4%
o3 83.3%
Gemini 2.5 Pro Preview 83%
DeepSeek V3.2 82.4%
GPT-5 minihigh 82.3%
o4-minihigh 81.4%
DeepSeek R1 0528 81%
Claude Opus 4.1 81%
Grok 3 80.2%
Grok 3 Beta 80.2%
gpt-oss-120b 80.1%
DeepSeek V3.1thinking 80.1%
Gemma 4 26B A4B 79.6%
Claude Opus 4 79.6%
Nemotron 3 Super 79.2%
o1-pro 79%
Grok 3 Minihigh 79%
Gemini 2.5 Flash 78.3%
Claude 3.7 Sonnet 78.2%
o1 78%
o3-mini 77%
Claude Sonnet 4 76.1%
Llama 3.1 Nemotron Ultra 76%
Nemotron 3.5 Lightning 75.4%
Kimi K2 75.1%
Nemotron 3 Nano 74.1%
o1-preview 73.3%
Claude Haiku 4.5 73.2%
DeepSeek R1 71.5%
gpt-oss-20b 71.5%
Qwen3 71.1%
Qwen3 235B 71.1%
GPT-4.5 69.5%
DeepSeek V3-0324 68.4%
DeepSeek V3 0324 68.4%
GPT-4.1 66.3%
Claude 3.5 Sonnet 65%
Nemotron Nano 2 64%
LFM3-1.7B 63.4%
Llama 4 Maverick 61.1%
Gemini 2.0 Flash 60.1%
o1-mini 60%
Qwen3-8B 59.6%
Gemini 1.5 Pro 59.1%
DeepSeek V3 59.1%
Llama 4 Scout 57.2%
Mistral Medium 3 57.1%
GPT-4o 52.5%
Gemini 1.5 Flash 51%
Llama 3.1 405B 43.4%
Gemma 3 27B 42.4%

107 models · 59 drops · Kimi K3.1 Pro, Claude Mythos Preview, Muse Spark, Qwen3.8-Max, Muse Spark Open, Claude Sonnet 4.6, MAI-Thinking-1, Nemotron 3.5 Super VL, GPT-5.4, Qwen3.8, Inkling, Gemini 3 Flash, Particle, "Introducing Particle-1", Nanbeige4.3-33B model card, Claude Opus 4.5, GPT-5.2, GLM-5, Grok 4 Fast, Nemotron 3 Ultra, Gemini 3 Pro, DeepSeek V3.2, MAI-Code-1-Flash, Gemma 4, o3-pro, DiffuSpace, "Clover-35B-A3B-Preview", Nemotron 3.5 Lightning, GPT-5, Claude Opus 4 & Sonnet 4, gpt-oss-120b & gpt-oss-20b, DeepSeek V3.1, Claude Sonnet 4.5, Gemini 2.5 Pro, Qwen3, Nemotron 3 Super, Claude Opus 4.1, Gemini 2.5 Flash, Claude Haiku 4.5, Llama 3.1 Nemotron Ultra, Kimi K2, o1, DeepSeek R1 0528, GPT-4.1, Llama 4 Maverick & Scout, Claude 3.7 Sonnet, Nemotron Nano 2, Liquid AI, "Introducing LFM3", Mistral Medium 3, o3-mini, Gemini 2.0 Flash, DeepSeek V3

SWE-bench Verified

75 models compared.

Claude Mythos Previewmax 93.9%
GPT-6 Astra 86%
Claude Opus 4.5high 80.9%
Claude Opus 4.6max 80.8%
Gemini 3.1 Prohigh 80.6%
DeepSeek V4 Pro 80.6%
69 more
Kimi K2.6 80.2%
GPT-5.2xhigh 80%
Claude Sonnet 4.6max 79.6%
GLM-5.3-Flash 79.1%
Gemini 3 Flashxhigh 78%
GPT-5.1-Codex-Maxxhigh 77.9%
GLM-5high 77.8%
Inkling 77.6%
Muse Spark 77.4%
Claude Sonnet 4.5xhigh 77.2%
Kimi K2.5 76.8%
Qwen3.5 76.4%
GPT-5.1high 76.3%
GPT-5.1 Thinkinghigh 76.3%
Gemini 3 Proxhigh 76.2%
GPT-5high 74.9%
Claude Opus 4.1 74.5%
GPT-5-Codex 74.5%
Nemotron 3 Ultra 73.9%
GPT-5.1-Codexhigh 73.7%
Clover-35B-A3B-Preview 73.6%
MAI-Thinking-1 73.5%
Fern 0.1 73.4%
DeepSeek V3.2 73.1%
Claude Sonnet 4 72.7%
Claude Opus 4 72.5%
Qwen3.5-122B-A10B 71.9%
MAI-Code-1-Flash 71.6%
Kimi K2 Thinkinghigh 71.3%
Nemotron 3.5 Super VL 71.1%
Grok Code Fast 1 70.8%
Qwen3.6 35B A3B 70.1%
Qwen3-Max 69.6%
MiniMax-M2 69.4%
Kimi K2 0905 69.2%
o3high 69.1%
o4-minihigh 68.1%
GLM-4.6 68%
DeepSeek V3.2-Exp 67.8%
Claude Haiku 4.5 66.6%
DeepSeek V3.1 66%
Kimi K2 65.4%
GLM-4.5 64.2%
Gemini 2.5 Pro Preview 63.2%
Claude 3.7 Sonnet 62.3%
Nemotron 3 Super 60.5%
Gemini 2.5 Flashxhigh 60.4%
Gemini 2.5 Pro 59.6%
Gemma 4 26B A4B 57.4%
GPT-4.1 54.6%
gpt-oss-20b 52.4%
Qwen3-235B-A22B-Instruct-2507 52.2%
Nemotron 3.5 Lightning 51.6%
Grok 4.1 Fast 50.6%
o3-minihigh 49.3%
DeepSeek R1 49.2%
Claude 3.5 Sonnet 49%
o1 48.9%
DeepSeek V3 0324 45.4%
DeepSeek R1 0528 44.6%
LFM3-1.7B 43.4%
DeepSeek V3 42%
gpt-oss-120b 41.9%
o1-mini 41.6%
o1-preview 41.3%
GPT-4.5 38%
Nemotron 3 Nano 34.1%
GPT-4o 33.2%
Llama 3.1 405B 24.5%

75 models · 48 drops · Claude Mythos Preview, Mirendil, "Introducing Fern", Qwen3.5, MAI-Thinking-1, Muse Spark, Inkling, Gemini 3.1 Pro, Nemotron 3.5 Super VL, Gemini 3 Flash, Claude Opus 4.5, GLM-5, Claude Sonnet 4.6, GPT-5.2, DeepSeek V3.2, Claude Sonnet 4.5, Claude Haiku 4.5, GPT-5.1-Codex-Max, DiffuSpace, "Clover-35B-A3B-Preview", Qwen3-Max, MAI-Code-1-Flash, Grok Code Fast 1, Nemotron 3.5 Lightning, MiniMax-M2, Kimi K2 Thinking, GPT-5, gpt-oss-120b & gpt-oss-20b, GLM-4.5, Claude Opus 4 & Sonnet 4, Nemotron 3 Super, Gemini 3 Pro, DeepSeek R1 0528, Claude 3.7 Sonnet, o3 & o4-mini, DeepSeek V3.1, Liquid AI, "Introducing LFM3", DeepSeek V3, DeepSeek R1, o3-mini, GPT-4.1

Humanity's Last Exam (no tools)

Expert-level questions, no tools.

Claude Opus 5.5 64.4%
Mythos 5 59%
Fable 5 57.8%
Fable 5.1max 57.6%
Claude Sonnet 5.5 56.9%
Claude Mythos Previewmax 56.8%
71 more
Claude Opus 5 56.3%
GPT-6 Astramax 53.6%
Kimi K3.1 Promax 51.1%
Claude Haiku 5.5max 45.9%
Claude Opus 4.8max 45.7%
GPT-5.6 Sol 44.5%
Gemini 3.1 Pro 44.4%
GPT-5.4xhigh 43.9%
Qwen3.8-Max 43.6%
Qwen3.8max 43.6%
Kimi K3 43.5%
Muse Spark 42.8%
GPT-5.4 Pro 42.7%
DeepSeek V4.1 Flash 1017 42.6%
GPT-5.5 41.4%
GLM-5.2 40.1%
Claude Opus 4.6max 40%
Qwen3.8-Flash-Next 39.6%
Gemini 3 Pro 37.5%
DeepSeek V4.1 Flash 36.8%
GPT-5.2 Pro 36.6%
Kimi K2.6 35.9%
GPT-5.2xhigh 34.5%
Gemini 3 Flashxhigh 33.7%
Claude Sonnet 4.6max 33.2%
Nemotron 3.5 Super VL 31.8%
Kimi K2.5 31.5%
Claude Opus 4.5 30.8%
GLM-5high 30.5%
Inkling 29.7%
Nemotron 3 Ultra 29.2%
Qwen3.5 28.7%
Qwen3.5 397B 28.5%
GLM-5.1 27.2%
GPT-5.1 26.5%
GPT-5high 26.3%
GPT-5.1 Thinkinghigh 25.7%
Grok 4 25.4%
DeepSeek V3.2 25.1%
Qwen3.5-122B-A10B 24.2%
Kimi K2 Thinkinghigh 23.9%
Gemini 2.5 Pro 21.6%
o3 20.2%
Grok 4 Fast 20%
Gemma 4 19.5%
Nemotron 3 Super 18.3%
DeepSeek R1 0528 17.7%
Claude Sonnet 4.5 17.7%
Grok 4.1 Fast 17.6%
GPT-5 minihigh 16.7%
DeepSeek V3.1thinking 15.9%
gpt-oss-120b 14.9%
GLM-4.5 14.4%
o4-mini 14.3%
o3-mini 13.4%
Gemini 2.5 Flash 12.1%
Qwen3 235B 11.8%
gpt-oss-20b 10.9%
Nemotron 3 Nano 10.6%
Claude Haiku 4.5 10.2%
Qwen3-30B-A3B-Thinking-2507 9.8%
Claude 3.7 Sonnet 8.9%
Gemma 4 26B A4B 8.7%
DeepSeek R1 8.5%
o1-pro 8.1%
LFM3-1.7B 7.2%
Nemotron Nano 2 6.5%
GPT-4.5 6.4%
GPT-4.1 5.4%
Gemini 2.0 Flash 5.1%
Qwen3-8B 4.4%

77 models · 40 drops · Claude Opus 5.5, Fable 5 / Mythos 5, Fable 5.1, Kimi K3.1 Pro, Claude Haiku 5.5, DeepSeek V4.1 Flash 1017, Qwen3.8, Muse Spark, Qwen3.8-Max, GPT-5.4, Inkling, Nemotron 3.5 Super VL, Claude Sonnet 4.6, Gemini 3 Flash, Gemini 3.1 Pro, GLM-5.1, Qwen3.5, GLM-5, Nemotron 3 Ultra, Gemini 3 Pro, DeepSeek V3.2, GPT-5.2, Kimi K2 Thinking, GPT-5, Grok 4 Fast, Gemma 4, Nemotron 3 Super, DeepSeek V3.1, GLM-4.5, Gemini 2.5 Flash, o3 & o4-mini, DeepSeek R1 0528, Nemotron 3 Nano, Liquid AI, "Introducing LFM3", Nemotron Nano 2, Gemini 2.5 Pro

LiveCodeBench

61 models compared.

DeepSeek V4 93.5%
Gemini 3 Pro 90.7%
Kimi K2.6 89.6%
Nemotron 3 Ultra 89%
GPT-5.2 87.7%
MAI-Thinking-1 87.7%
55 more
GLM-5.1 85.7%
Kimi K2.5 85%
Claude Opus 4.5 84.8%
GPT-5high 84.5%
Qwen3.5 83.6%
DeepSeek V3.2 83.3%
Kimi K2 Thinking 82.6%
Grok 4 Fast 80%
Gemma 4 80%
Qwen3.5 397B 79.3%
Grok 4 79%
GPT-5 minihigh 77.4%
Gemma 4 26B A4B 77.1%
Gemini 2.5 Pro 74.2%
o3-minihigh 74.1%
DeepSeek V3.2-Exp 74.1%
LFM3-1.7B 73.5%
DeepSeek R1 0528 73.3%
o3 72%
Nemotron Nano 2 71.1%
Qwen3 70.7%
Grok 3 70.6%
Grok 3 Beta 70.6%
Qwen3-Max 69%
Nemotron 3 Nano 68.3%
Qwen3 235B 66.5%
Llama 3.1 Nemotron Ultra 66.3%
Qwen3-30B-A3B-Thinking-2507 66%
Claude Sonnet 4.5 64%
o1 63.9%
DeepSeek R1 63.5%
Gemini 2.5 Flash 63.5%
gpt-oss-20b 61%
Qwen3-8B 59.5%
o1-mini 53.8%
Kimi K2 53.7%
DeepSeek V3.1 52.3%
Qwen3-235B-A22B-Instruct-2507 51.8%
DeepSeek V3 0324 46.9%
Gemini 2.0 Flash Thinking 45.8%
DeepSeek V3-0324 45.8%
GPT-4.1 44.7%
Claude Opus 4 44.6%
DeepSeek V3 40.5%
Claude 3.5 Sonnet 36.3%
Claude 3.7 Sonnet 36%
Gemini 2.0 Flash 34.5%
Gemini 1.5 Pro 34.3%
Llama 4 Scout 32.8%
GPT-4o 31.4%
Mistral Medium 3 30.3%
Gemini 1.5 Flash 30%
Gemma 3 27B 29.1%
Llama 4 Maverick 28.7%
Llama 3.1 405B 28.4%

61 models · 24 drops · MAI-Thinking-1, Qwen3.5, Nemotron 3 Ultra, DeepSeek V3.2, Grok 4 Fast, Gemma 4, Grok 4, Gemini 2.5 Pro, Kimi K2 Thinking, Liquid AI, "Introducing LFM3", DeepSeek R1 0528, Nemotron Nano 2, Qwen3, Qwen3-Max, Nemotron 3 Nano, Llama 3.1 Nemotron Ultra, Gemini 2.5 Flash, DeepSeek R1, Kimi K2, Grok 3, Llama 4 Maverick & Scout, DeepSeek V3, Mistral Medium 3, Gemini 2.0 Flash

AIME 2025

46 models compared.

Claude Opus 4.6 99.8%
o4-mini 99.5%
gpt-oss-120b 97.9%
MAI-Thinking-1 97%
Claude Sonnet 4.6max 95.6%
Gemini 3 Pro 95%
40 more
GPT-5high 94.6%
Kimi K2 Thinkinghigh 94.5%
GPT-5.1high 94%
DeepSeek V3.2 93.1%
Grok 4 Fast 92%
Grok 4 91.7%
gpt-oss-20b 91.7%
GPT-5 minihigh 91.1%
DeepSeek V3.2-Exp 89.3%
Nemotron 3 Nano 89.1%
o3 88.9%
Gemini 2.5 Pro 88%
DeepSeek R1 0528 87.5%
Claude Sonnet 4.5 87%
Qwen3-30B-A3B-Thinking-2507 85%
Gemini 2.5 Pro Preview 83%
Qwen3-Max 81.6%
Qwen3 81.5%
Qwen3 235B 81.5%
Claude Haiku 4.5 80.7%
o1 79.2%
Gemini 2.5 Flash 78%
Claude Opus 4.1 78%
Grok 3 77.3%
Grok 3 Beta 77.3%
o3-minimedium 74.8%
Llama 3.1 Nemotron Ultra 72.5%
Nemotron Nano 2 72.1%
Claude Sonnet 4 70.5%
Qwen3-235B-A22B-Instruct-2507 70.3%
DeepSeek R1 70%
Qwen3-8B 69.3%
Claude 3.7 Sonnet 54.8%
Gemini 2.0 Flash Thinking 53.5%
DeepSeek V3.1 49.8%
Kimi K2 49.5%
DeepSeek V3 0324 46.7%
GPT-4.1 46.4%
Claude Opus 4 33.9%
Gemini 2.0 Flash 27.5%

46 models · 24 drops · MAI-Thinking-1, gpt-oss-120b & gpt-oss-20b, DeepSeek V3.2, GPT-5.1, Grok 4 Fast, Kimi K2 Thinking, Nemotron 3 Nano, GPT-5, Claude Sonnet 4.5, DeepSeek V3.1, Claude Opus 4 & Sonnet 4, Qwen3-Max, Qwen3, DeepSeek R1 0528, Claude Haiku 4.5, Gemini 2.5 Flash, Gemini 2.5 Pro, Llama 3.1 Nemotron Ultra, Nemotron Nano 2, Grok 3, Kimi K2

DeepSWE v1.1

Fixing issues in real repositories.

Muse Wonder Fullmax 79%
Muse Wondermax 78.8%
Gemini 4 Argonmax 77.9%
DeepSeek V4.1 Flash 1017max 77%
Kimi K3.1 Promax 75.5%
Muse Spark 1.3max 75.4%
39 more
GPT-6.1 Solhigh 75.2%
Claude Opus 5.5max 74.2%
DeepSeek V4.1 Flash 74.2%
GPT-6 Astramax 74.1%
Claude Opus 5 74%
Gemini 3.8 Flash 73.8%
MiniMax M3.1 Flash 73.7%
GPT-5.6 Sol 73%
Particle-1max 72.6%
MiMo-V2.6-Pro 71.9%
Grok 4.7high 71%
Fable 5xhigh 69.9%
GPT-5.6 Terramax 69.6%
GPT-6 Solmax 68.8%
Kimi K3 67.5%
Fable 5.1max 67.4%
GPT-5.6 Lunamax 67%
GPT-5.5xhigh 67%
GLM-5.3max 66.9%
GPT-6 Lunamax 66.6%
Nanbeige4.3-33Bmax 65.5%
Gemini 3.7 Flashhigh 65.3%
Grok 4.6high 65.2%
Mistral Large 4 62%
Muse Spark Openxhigh 60%
Claude Opus 4.8max 59%
DeepSeek V4 Pro 0813 57%
Qwen3.8-2.4T-A95Bxhigh 57%
Qwen3.8-Max 56.6%
Muse Spark 1.2xhigh 55%
Grok 4.5high 54%
Claude Sonnet 5max 53.8%
Gemini 3.6 Flash 48.6%
Fern 0.1 44.2%
Qwen3.8max 42.5%
Gemini 3.5 Flash 37%
Muse Glimmer-30Bmax 4.4%
Nemotron 3 Ultra 3.5%
Gemma 4max 2.7%

45 models · 24 drops · Muse Wonder, Gemini 4 Argon, Kimi K3.1 Pro, GPT-6 Astra, GPT-6.1 Sol, MiniMax M3.1 Flash, Particle, "Introducing Particle-1", MiMo-V2.6-Pro, Grok 4.7, GPT-6 Sol & GPT-6 Luna, Gemini 3.8 Flash, Gemini 3.6 Flash, GPT-5.6 Sol, Terra, & Luna, Nanbeige4.3-33B model card, Muse Spark Open, Qwen3.8-Max, Mistral Large 4, Muse Spark 1.3, Grok 4.6, Gemini 3.7 Flash, Mirendil, "Introducing Fern"

Terminal-Bench 2.0

Hard tasks in a real terminal (2025 edition).

Claude Mythos Previewmax 82%
GPT-5.3-Codex 77.3%
GPT-5.4xhigh 75.1%
Gemini 3.1 Prohigh 68.5%
Claude Opus 4.6max 65.4%
GPT-5.2-Codex 64%
27 more
GLM-5.1 63.5%
GPT-5.2 62.2%
Composer 2 61.7%
Claude Sonnet 4.6max 59.1%
Muse Spark 59%
GPT-5.1-Codex-Maxxhigh 58.1%
Gemini 3 Pro 56.2%
GLM-5high 56.2%
MAI-Code-1-Flash 54.8%
GPT-5.1-Codexhigh 52.8%
Qwen3.5 52.5%
Claude Opus 4.5 52.1%
Claude Sonnet 4.5 51%
Kimi K2.5 50.8%
GPT-5.1 47.6%
Gemini 3 Flashxhigh 47.6%
DeepSeek V3.2 46.4%
MAI-Thinking-1 46%
Claude Haiku 4.5 41.6%
Composer 40%
Qwen3.5-122B-A10B 37.5%
Kimi K2 Thinking 35.7%
GPT-5high 35.2%
Gemini 2.5 Pro 32.6%
Nemotron 3 Super 31%
gpt-oss-120b 18.7%
Gemini 2.5 Flashxhigh 16.9%

33 models · 22 drops · Claude Mythos Preview, GPT-5.4, MAI-Thinking-1, Muse Spark, GPT-5.3-Codex, GLM-5.1, Composer 2, GPT-5.2-Codex, Claude Sonnet 4.6, GLM-5, MAI-Code-1-Flash, GPT-5.1-Codex-Max, Qwen3.5, Claude Opus 4.5, Gemini 3 Flash, Composer, Nemotron 3 Super, DeepSeek V3.2, Gemini 3 Pro

BrowseComp

49 models compared.

Kimi K3.1 Promax 92.1%
LongCat-2.5 91.9%
GPT-6 Astramax 91.5%
Claude Opus 5.5max 90.8%
GPT-5.6 Solmax 90.4%
GPT-5.6 Terramax 87.5%
43 more
Gemini 3.1 Promax 85.9%
GPT-5.5xhigh 84.4%
Claude Opus 4.6max 84%
GPT-5.6 Lunaxhigh 84%
GPT-5.4 82.7%
GPT-5.2 Pro 77.9%
GPT-5.3-Codex 77.3%
Claude Sonnet 4.6max 74.7%
Qwen3.5 69%
ChatGPT agent 68.9%
Claude Opus 4.5 67.8%
GPT-5.2xhigh 65.8%
GLM-5high 62%
Kimi K2.6 61.3%
Kimi K2.5 60.6%
Kimi K2 Thinkinghigh 60.2%
GLM-5.1 59.4%
Gemini 3 Pro 59.2%
Gemini 3 Pro Deep Research 59.2%
GPT-5high 54.9%
deep research 51.5%
DeepSeek V3.2 51.4%
GPT-5.1 Thinkinghigh 50.8%
o3high 49.7%
Qwen3.6 35B A3B 48.7%
GLM-4.6 45.1%
Grok 4 Fast 44.9%
Nemotron 3 Ultra 44.4%
MiniMax-M2 44%
Grok 4 43%
Qwen3.5 397B 40.5%
DeepSeek V3.2-Exp 40.1%
Nemotron 3.5 Lightning 37%
DeepSeek V3.1thinking 30%
o4-minihigh 28.3%
GLM-4.5 26.4%
Gemma 4 26B A4B 26.3%
Claude Sonnet 4.5 24.1%
Claude Opus 4 18.8%
Nemotron 3 Nano 13.7%
DeepSeek R1 0528 8.9%
Kimi K2 7.9%
GPT-4o 1.9%

49 models · 21 drops · Kimi K3.1 Pro, Meituan LongCat-2.5 launch post, GPT-5.6 Sol, Terra, & Luna, Gemini 3.1 Pro, GPT-5.4, Claude Opus 4.6, Qwen3.5, GPT-5, GLM-5, Nemotron 3 Ultra, GLM-5.1, DeepSeek V3.2, o3 & o4-mini, GPT-5.2, Nemotron 3.5 Lightning, MiniMax-M2, Grok 4 Fast, Kimi K2 Thinking, DeepSeek V3.1, GLM-4.5

Terminal-Bench 2.1

47 models compared.

MiMo-V2.6-Pro 89.9%
Gemini 3.8 Flashmax 89.4%
Claude Opus 5 89.1%
GPT-5.6 Solmax 88.8%
Muse Spark 1.3max 88.8%
LongCat-2.5 88.4%
41 more
Mythos 5 88%
GPT-5.6 Terramax 87.4%
Qwen3.8-Max 86.6%
Gemini 3.7 Flashhigh 85.8%
GPT-5.5xhigh 85.6%
Kimi K3max 85%
GPT-5.6 Lunamax 84.7%
Claude Opus 4.8max 84.6%
Fable 5max 84.6%
GLM-5.3max 83.9%
Muse Spark 1.2xhigh 82.9%
GLM-5.2 82.7%
Qwen3.8-2.4T-A95Bxhigh 82%
Grok 4.5high 81.6%
Nanbeige4.3-33Bmax 80.9%
Claude Sonnet 5max 80.4%
Particle-1max 79.9%
Claude Opus 4.6max 78.2%
Gemini 3.6 Flashmax 78%
Muse Spark Openxhigh 76.4%
Gemini 3.5 Flash 76.2%
Muse Spark 1.1xhigh 76.2%
Qwen3.8max 73%
Kimi K2.6 71.3%
Gemini 3.1 Pro 70.7%
Qwen3.7-Plusmax 64%
Inkling 63.8%
Qwen3.6-27Bmax 63.4%
GLM-5.1 59.3%
Nemotron 3 Ultra 56.4%
Muse Glimmer-30Bmax 51.7%
Clover-35B-A3B-Preview 51.7%
Qwen3.5 397B 49.9%
Gemma 4max 47.2%
Qwen3.6 35B A3B 44.4%
SenseNova-Loop-9B-Preview 40.5%
Gemma 4 26B A4B 37.2%
Nemotron 3.5 Lightning 24.6%
gpt-oss-20b 15.2%
LFM3-1.7B 12.9%
Nemotron 3 Nano 8.3%

47 models · 20 drops · MiMo-V2.6-Pro, Gemini 3.8 Flash, Muse Spark 1.3, Meituan LongCat-2.5 launch post, Fable 5 / Mythos 5, Qwen3.8-Max, Gemini 3.7 Flash, GPT-5.6 Sol, Terra, & Luna, Muse Spark Open, Gemini 3.6 Flash, Inkling, Muse Spark 1.2, Nanbeige4.3-33B model card, Particle, "Introducing Particle-1", Qwen3.8, Nemotron 3 Ultra, DiffuSpace, "Clover-35B-A3B-Preview", Nemotron 3.5 Lightning, SenseNova-Loop-9B-Preview model card, Liquid AI, "Introducing LFM3"

ARC-AGI-2

34 models compared.

GPT-5.6 Sol 92.5%
Claude Opus 5 90.4%
Fable 5 89.2%
Gemini 3.8 Flash 89.2%
Gemini 3.7 Flash 84.6%
GPT-5.6 Terramax 83.9%
28 more
GPT-5.4 Pro 83.3%
Gemini 3.1 Prohigh 76.5%
GPT-5.4xhigh 76.1%
Grok 4.6 65.1%
Claude Opus 4.6max 63.3%
Gemini 3.6 Flash 60.4%
GPT-5.6 Lunamax 59.5%
Claude Sonnet 4.6max 58.3%
GPT-5.2 Prohigh 54.2%
GPT-5.2xhigh 52.9%
Grok 4.5high 52.6%
Gemini 3 Deep Think 45.1%
Muse Spark 42.5%
Claude Opus 4.5 37.6%
Gemini 3 Flashxhigh 33.6%
Gemini 3 Proxhigh 31.1%
GPT-5.1high 17.6%
GPT-5.1 Thinkinghigh 17.6%
Grok 4 16%
Claude Sonnet 4.5xhigh 13.6%
GPT-5 9.9%
Claude Opus 4 8.6%
Claude Sonnet 4 5.9%
Grok 4 Fast 5.3%
Gemini 2.5 Pro 4.9%
GPT-5 minihigh 4.4%
o3 2.9%
o4-minimedium 2.3%

34 models · 20 drops · Grok 4.6, Gemini 3.8 Flash, Gemini 3.7 Flash, GPT-5.4, Muse Spark, Gemini 3.6 Flash, Gemini 3.1 Pro, Claude Sonnet 4.6, Claude Opus 4.6, Gemini 3 Flash, Claude Opus 4.5, GPT-5.2, GPT-5.1, GPT-5, Grok 4, Claude Opus 4 & Sonnet 4, Grok 4 Fast, Gemini 3 Pro, o3 & o4-mini

GDPval-AA

Professional work products, graded head to head (Elo).

Claude Opus 5.5max 1866 Elo
Muse Wonder Fullmax 1857 Elo
Kimi K3.1 Promax 1846 Elo
Claude Sonnet 5.5 1838 Elo
Muse Wondermax 1770 Elo
GLM-5.3 1769 Elo
33 more
Fable 5.1max 1754 Elo
Muse Spark 1.3max 1754 Elo
DeepSeek V4.1 Flash 1017max 1746 Elo
Fable 5 1743 Elo
Claude Opus 5 1708 Elo
Grok 4.7xhigh 1695 Elo
MiMo-V2.6-Pro 1673 Elo
Claude Sonnet 4.6max 1633 Elo
Claude Haiku 5.5max 1620 Elo
Muse Spark 1.2xhigh 1615 Elo
Qwen3.8-2.4T-A95Bxhigh 1613 Elo
Claude Opus 4.6max 1606 Elo
Grok 4.6high 1605 Elo
GPT-5.6 Sol 1588 Elo
GPT-5.6 Lunamax 1584 Elo
GPT-6 Astramax 1573 Elo
Gemini 3.8 Flashmax 1545 Elo
Kimi K3max 1537 Elo
GPT-5.6 Terramax 1528 Elo
Grok 4.5high 1526 Elo
Gemini 3.7 Flashhigh 1525 Elo
GPT-6 Sol 1487 Elo
GPT-5.2xhigh 1462 Elo
Claude Sonnet 5 1449 Elo
GPT-6 Luna 1437 Elo
Mistral Large 4 1424 Elo
Gemini 3.6 Flash 1422 Elo
Claude Opus 4.5 1416 Elo
Muse Spark Openxhigh 1401 Elo
Gemini 3.5 Flash 1349 Elo
Gemini 3.1 Promax 1317 Elo
Gemini 3 Pro 1201 Elo
Claude Haiku 4.5 735 Elo

39 models · 19 drops · Kimi K3.1 Pro, Muse Wonder, GLM-5.3, Muse Spark 1.3, Claude Opus 5.5, Grok 4.7, MiMo-V2.6-Pro, Gemini 3.1 Pro, Claude Haiku 5.5, Muse Spark Open, Gemini 3.6 Flash, Gemini 3.8 Flash, Grok 4.6, Gemini 3.7 Flash, Claude Sonnet 5.5, Claude Opus 4.6, Claude Sonnet 4.6

Terminal-Bench 4.0

Multi-hour work in a real terminal.

Claude Sonnet 5.5max 70.6%
Claude Opus 5.5max 66.4%
Kimi K3.1 Promax 65.7%
Mythos 5.1 60.9%
GPT-6 Astramax 58.2%
Fable 5.1max 57.9%
31 more
Gemini 4 Argonmax 57.4%
Muse Wonder Fullmax 57.1%
GPT-6.1 Solmax 55.9%
Muse Wondermax 52%
Claude Opus 5 51.8%
GPT-6.1 Terramax 50.3%
Hy4 44.4%
DeepSeek V4.1 Flash 1017max 42.3%
GLM-5.3max 41.8%
GPT-5.6 Sol 39.9%
Claude Haiku 5.5max 39.2%
LongCat-2.5 38.5%
MiniMax M3.1 Flash 38.1%
Grok 4.7xhigh 37.6%
MiMo-V2.6-Pro 34.9%
DeepSeek V4.1 Flash 31.2%
Mistral Large 4 28.3%
Nanbeige4.3-33Bmax 23.8%
GPT-5.6 Terramax 21.5%
Grok 4.6high 20.3%
Gemini 3.8 Flashmax 19.1%
GPT-6 Luna 16.4%
Kimi K3 12.6%
Claude Sonnet 5 10.3%
DeepSeek V4 Pro 0813 10%
Qwen3.8max 6.3%
Fern 0.1 6.1%
Muse Glimmer-30Bmax 1.6%
Claude Haiku 4.5 0%
Gemma 4max 0%
Nemotron 3 Ultra 0%

37 models · 19 drops · GPT-6.1 Terra, Kimi K3.1 Pro, GPT-6 Astra, Gemini 4 Argon, Muse Wonder, MiniMax M3.1 Flash, Tencent Hunyuan Hy4 launch post, Claude Haiku 5.5, Meituan LongCat-2.5 launch post, Grok 4.7, MiMo-V2.6-Pro, Mistral Large 4, Nanbeige4.3-33B model card, Gemini 3.8 Flash, Claude Sonnet 5.5, Mirendil, "Introducing Fern"

MMMU

32 models compared.

Gemini 3 Pro 87.2%
GPT-5.2 86.7%
GPT-5.1high 85.4%
Qwen3.5 85%
Kimi K2.5 84.3%
GPT-5high 84.2%
26 more
o3high 82.9%
Gemini 2.5 Pro 82%
o4-minihigh 81.6%
Claude Opus 4.5 80.7%
Gemini 2.5 Pro Preview 79.6%
Claude Sonnet 4.5 77.8%
o1 77.6%
Claude Opus 4.1 77.1%
Gemini 2.5 Flash 76.7%
Claude Opus 4 76.5%
Grok 3 76%
Grok 3 Beta 76%
Gemini 2.0 Flash Thinking 75.4%
Claude 3.7 Sonnet 75%
GPT-4.1 74.8%
GPT-4.5 74.4%
Claude Sonnet 4 74.4%
Claude Haiku 4.5 73.2%
Llama 4 Maverick 71.8%
Gemini 2.0 Flash 71.7%
Claude 3.5 Sonnet 70.4%
Llama 4 Scout 69.4%
GPT-4o 68.7%
Mistral Medium 3 66.1%
Gemini 1.5 Pro 65.9%
Gemini 1.5 Flash 62.3%

32 models · 17 drops · Qwen3.5, Claude Opus 4.5, GPT-5.1, Claude Opus 4.1, Claude Haiku 4.5, Gemini 2.5 Flash, Claude Opus 4 & Sonnet 4, o3 & o4-mini, Claude Sonnet 4.5, Gemini 2.5 Pro, Grok 3, Mistral Medium 3, Claude 3.7 Sonnet, Llama 4 Maverick & Scout, Gemini 2.0 Flash

ARC-AGI-1

28 models compared.

Gemini 3.1 Pro 98%
GPT-5.6 Terramax 96.5%
Gemini 3.7 Flash 95.5%
Claude Opus 4.6high 94%
Gemini 3.6 Flashhigh 91.2%
GPT-5.2xhigh 86.2%
22 more
Claude Sonnet 4.6max 86%
Gemini 3 Flash 84.7%
Claude Opus 4.5 80%
Gemini 3 Pro 75%
GPT-5.1high 72.8%
Grok 4 66.7%
GPT-5 65.7%
Claude Sonnet 4.5thinking 63.7%
GPT-5 minihigh 54.3%
o3 53%
Grok 4 Fast 48.5%
Claude Haiku 4.5 47.7%
Gemini 2.5 Pro 41%
Claude Opus 4 35.7%
o1high 35%
o3-mini 34.5%
Claude 3.7 Sonnet 28.6%
DeepSeek R1 15.8%
o1-mini 14%
GPT-4.5 10.3%
GPT-4.1 5.5%
GPT-4o 4.5%

28 models · 17 drops · Gemini 3.1 Pro, Gemini 3.7 Flash, Claude Sonnet 4.6, Gemini 3 Flash, Claude Opus 4.5, Gemini 3 Pro, GPT-5, Claude Haiku 4.5, o3 & o4-mini, Grok 4, DeepSeek R1, o3-mini, Claude 3.7 Sonnet, GPT-4.1

Aider Polyglot

27 models compared.

GPT-5high 88%
o3-pro 84.9%
o3 79.6%
Gemini 2.5 Pro 76.9%
DeepSeek V3.1thinking 76.3%
DeepSeek R1 0528 71.6%
21 more
Claude Opus 4 70.7%
o4-mini 68.9%
Qwen3 235B 65%
Claude 3.7 Sonnet 64.9%
o1high 61.7%
Claude Sonnet 4 61.3%
o3-mini 60.4%
Kimi K2 60%
Qwen3no thinking 59.6%
DeepSeek R1 57%
DeepSeek V3-0324 55.1%
DeepSeek V3 0324 55.1%
GPT-4.1 52.9%
Claude 3.5 Sonnet 51.6%
Gemini 2.5 Flash 51.1%
DeepSeek V3 48.4%
GPT-4.5 44.9%
o1-mini 32.9%
Gemini 2.0 Flash 22.2%
GPT-4o 18.2%
Llama 4 Maverick 15.6%

27 models · 17 drops · GPT-5, o3-pro, DeepSeek R1 0528, DeepSeek V3.1, Kimi K2, Gemini 2.5 Flash, Gemini 2.5 Pro, GPT-4.1, Claude Opus 4 & Sonnet 4, o3-mini, Qwen3, Llama 4 Maverick & Scout, Claude 3.7 Sonnet, GPT-4.5, DeepSeek R1

AutomationBench

Real business automations (Zapier).

MiniMax M3.1 Flash 62.2%
DeepSeek V4.1 Flash 1017 60.5%
Mistral Large 4 59.9%
DeepSeek V4 Pro 0813 56.7%
Hy4 56.2%
DeepSeek V4.1 Flash 54.8%
25 more
Nanbeige4.3-33Bmax 54.2%
MiMo-V2.6-Pro 53.1%
LongCat-2.5 51.8%
Gemini 4 Argonmax 51.3%
Claude Opus 5 50.3%
Muse Spark 1.3max 49.6%
GLM-5.3 48.8%
Kimi K3 46.7%
Fable 5 46.2%
GPT-5.6 Sol 45.8%
Claude Sonnet 5.5max 44.7%
Claude Opus 5.5 42.5%
GPT-6 Astramax 41.4%
Qwen3.8max 38.3%
GPT-6.1 Solmax 36.1%
GPT-6 Sol 33.2%
Fable 5.1max 31.4%
Gemini 3.7 Flashhigh 30.4%
GPT-5.6 Terra 23.6%
GPT-6 Lunamax 20.7%
Gemini 3.6 Flash 17%
Muse Glimmer-30Bmax 15.2%
SenseNova-Loop-9B-Preview 13.7%
Claude Sonnet 5 10.7%
Gemma 4max 9.7%

31 models · 16 drops · MiniMax M3.1 Flash, Mistral Large 4, Tencent Hunyuan Hy4 launch post, Nanbeige4.3-33B model card, MiMo-V2.6-Pro, Meituan LongCat-2.5 launch post, Gemini 4 Argon, Muse Spark 1.3, GLM-5.3, Claude Sonnet 5.5, GPT-6.1 Sol, Gemini 3.7 Flash, GPT-6 Sol & GPT-6 Luna, SenseNova-Loop-9B-Preview model card

SWE-bench Pro

36 models compared.

Fable 5max 80%
Claude Mythos Previewmax 77.8%
Claude Opus 4.8max 69.2%
Qwen3.8-Max 67.7%
Grok 4.5max 64.7%
GPT-5.6 Solmax 64.6%
30 more
Claude Sonnet 5max 63.2%
GPT-5.6 Lunamax 62.7%
GLM-5.2 62.1%
Qwen3.8max 61.7%
Qwen3.8-Flash-Next 60.4%
Nanbeige4.3-33Bmax 60.3%
GLM-5.3-Flash 59.7%
Gemini 3.6 Flashmax 58.7%
Kimi K2.6 58.6%
GLM-5.1 58.4%
GPT-5.4 57.7%
Qwen3.7-Plusmax 57.6%
GPT-5.3-Codex 56.8%
GPT-5.2-Codex 56.4%
GPT-5.2xhigh 55.6%
Gemini 3.5 Flash 55.1%
Inkling 54.3%
Gemini 3.1 Promax 54.2%
Kimi K2.5 53.8%
Qwen3.6-27Bmax 53.5%
Claude Opus 4.6max 53.4%
MAI-Thinking-1 52.8%
Nemotron 3.5 Super VL 51.8%
MAI-Code-1-Flash 51.2%
Muse Glimmer-30Bmax 51.2%
Gemma 4max 51.2%
GPT-5.1 Thinkinghigh 50.8%
GPT-5.1 50.8%
Nemotron 3 Ultra 49.7%
Claude Haiku 4.5 35.2%

36 models · 15 drops · Qwen3.8, Claude Mythos Preview, Qwen3.8-Max, Gemini 3.6 Flash, Inkling, Nemotron 3.5 Super VL, Nanbeige4.3-33B model card, MAI-Thinking-1, GLM-5.1, GPT-5.4, GPT-5.3-Codex, MAI-Code-1-Flash, GPT-5.2, GPT-5.2-Codex

Humanity's Last Exam (with tools)

Expert-level questions, with search and code.

Kimi K3.1 Promax 70%
Claude Opus 5.5 67.7%
Fable 5.1max 65.6%
Claude Mythos Preview 64.7%
Mythos 5 64.5%
GPT-5.6 Sol 64.5%
34 more
Fable 5 63.9%
Claude Opus 5 63.6%
GLM-5.3 62.5%
Kimi K3 59.8%
DeepSeek V4.1 Flash 1017max 58.8%
Claude Opus 4.8max 57.9%
GPT-6 Astramax 57.2%
Qwen3.8-Max 56.2%
GLM-5.2 54.7%
Kimi K2.6 54%
GPT-5.5 52.2%
Kimi K2.5 51.8%
Gemini 3.1 Pro 51.4%
GLM-5high 50.4%
Inkling 46%
Gemini 3 Pro 45.8%
GPT-5.2xhigh 45.5%
Kimi K2 Thinkinghigh 44.9%
Claude Opus 4.5 43.2%
GPT-5high 41.7%
Grok 4 41%
DeepSeek V3.2 40.8%
Nemotron 3 Ultra 37.4%
Claude Sonnet 4.5 32%
MiniMax-M2 31.8%
GLM-4.6 30.4%
DeepSeek V3.1thinking 29.8%
o3 24.9%
DeepSeek R1 0528 24.8%
DeepSeek V3.2-Exp 20.3%
gpt-oss-120b 19%
o4-mini 17.7%
gpt-oss-20b 17.3%
LFM3-1.7B 13.9%

40 models · 14 drops · Kimi K3.1 Pro, Claude Opus 5.5, Fable 5 / Mythos 5, GLM-5.3, Qwen3.8-Max, Inkling, GLM-5, Kimi K2.5, Kimi K2 Thinking, MiniMax-M2, DeepSeek V3.1, gpt-oss-120b & gpt-oss-20b, Liquid AI, "Introducing LFM3"

AIME 2024

23 models compared.

Grok 4 94.3%
o3-pro 93%
GLM-4.5 91%
o3 90.3%
DeepSeek R1 0528 89.3%
Gemini 2.5 Pro 88.7%
17 more
o3-minihigh 87.3%
o1-pro 86%
Qwen3 85.7%
Qwen3 235B 85.7%
Grok 3 83.9%
Grok 3 Beta 83.9%
Llama 3.1 Nemotron Ultra 80.8%
DeepSeek R1 79.8%
o1 74.3%
o1-mini 63.6%
Claude 3.7 Sonnet 61.3%
o1-preview 42%
DeepSeek V3 39.2%
GPT-4.5 36.7%
Llama 3.1 405B 23.3%
Claude 3.5 Sonnet 16%
GPT-4o 9.3%

23 models · 12 drops · GLM-4.5, o3-pro, GPT-4.5, Qwen3, DeepSeek R1 0528, Grok 3, Llama 3.1 Nemotron Ultra, o3-mini, Claude 3.7 Sonnet, o1, DeepSeek V3

τ²-bench Telecom

26 models compared.

GPT-5.2xhigh 98.7%
GLM-5.3-Flash 98.5%
Claude Sonnet 4.5 98%
GPT-5high 96.7%
GPT-5.1high 95.6%
GPT-5.1 Thinkinghigh 95.6%
20 more
Kimi K2 Thinking 93%
Qwen3.5-122B-A10B 87.2%
MiniMax-M2 86.8%
Nemotron 3 Ultra 83%
Nemotron 3.5 Super VL 73.2%
MAI-Code-1-Flash 71.7%
Claude Opus 4.1 71.5%
gpt-oss-120b 66%
Kimi K2 65.8%
Nemotron 3 Super 64.4%
o3 58.2%
Claude Opus 4 57%
Claude Haiku 4.5 54.7%
gpt-oss-20b 49.7%
Claude Sonnet 4 49.6%
Nemotron 3 Nano 42.2%
GPT-4.1 34%
DeepSeek V3.2-Exp 33.9%
DeepSeek V3 0324 32.5%
Qwen3-30B-A3B-Thinking-2507 26.3%

26 models · 11 drops · GPT-5.2, Nemotron 3.5 Super VL, Claude Haiku 4.5, GPT-5.1, Kimi K2 Thinking, MAI-Code-1-Flash, Claude Sonnet 4.5, Nemotron 3 Super, Kimi K2, GPT-5, Nemotron 3 Nano

SimpleQA

23 models compared.

Grok 4 Fast 95%
Grok 4 94%
Grok 3 82%
GPT-4.5 62.5%
Gemini 2.5 Pro 52.9%
o1 47%
17 more
Gemini 2.0 Pro 44.3%
Grok 3 Beta 43.6%
GPT-4.1 42.3%
o1-preview 42%
GPT-4o 38.2%
Kimi K2 31%
DeepSeek R1 30.1%
Gemini 2.0 Flash 29.9%
Gemini 2.5 Flash 29.7%
Claude 3.5 Sonnet 28.4%
DeepSeek R1 0528 27.8%
DeepSeek V3 0324 27.7%
DeepSeek V3 24.9%
Claude Opus 4 22.8%
Llama 3.1 405B 17.1%
o3-minihigh 13.8%
o1-mini 7.6%

23 models · 11 drops · Grok 4 Fast, Gemini 2.5 Pro, GPT-4.5, Grok 3, Gemini 2.5 Flash, Kimi K2, o1, DeepSeek R1 0528, DeepSeek V3, o3-mini

OSWorld-Verified

Using a computer: apps, files and the web (2025 edition).

Kimi K3 84.8%
Qwen3.8max 84.3%
Claude Opus 5 83.4%
Gemini 3.6 Flashmax 83%
GPT-5.6 Sol 83%
MiMo-V2.6-Pro 82%
18 more
Claude Sonnet 5max 81.2%
Claude Mythos Previewmax 79.6%
Gemini 3.5 Flash 78.4%
GPT-5.4 75%
GPT-5.3-Codex 74%
Qwen3.7-Plusmax 73.3%
Claude Opus 4.6max 72.7%
GPT-5.6 Lunamax 72.6%
Claude Sonnet 4.6max 72.5%
Claude Opus 4.5 66.3%
Muse Glimmer-30Bmax 65.9%
Qwen3.6-27Bmax 63.9%
Claude Sonnet 4.5 61.4%
Claude Haiku 4.5 50.7%
GPT-5.2 47.3%
Claude Opus 4.1 44.4%
Claude Sonnet 4 42.2%
GPT-5.2-Codex 38.2%

24 models · 10 drops · MiMo-V2.6-Pro, Qwen3.8, Gemini 3.6 Flash, Claude Mythos Preview, GPT-5.4, Claude Sonnet 4.6, Claude Haiku 4.5, Claude Opus 4.5, GPT-5.3-Codex

MMLU-Pro

25 models compared.

Claude Opus 4 87.3%
Claude Sonnet 4.6max 87%
Gemini 2.5 Pro 86.2%
o3 85.3%
Gemma 4 85.2%
MAI-Thinking-1 85%
19 more
DeepSeek R1 0528 84.9%
Clover-35B-A3B-Preview 84.8%
GLM-4.5 84.6%
Gemma 4 26B A4B 82.6%
DeepSeek V3-0324 81.2%
Llama 4 Maverick 80.4%
Claude 3.7 Sonnet 80%
Grok 3no thinking 79.9%
Gemini 2.0 Pro 79.1%
Claude 3.5 Sonnet 78%
Gemini 2.0 Flash 77.6%
Mistral Medium 3 77.2%
DeepSeek V3 75.9%
Gemini 1.5 Pro 75.8%
GPT-4o 75.8%
Llama 4 Scout 74.3%
Llama 3.1 405B 73.3%
Gemma 3 27B 67.6%
Gemini 1.5 Flash 67.3%

25 models · 9 drops · GLM-4.5, MAI-Thinking-1, Gemma 4, DiffuSpace, "Clover-35B-A3B-Preview", Llama 4 Maverick & Scout, Mistral Medium 3, Grok 3, Gemini 2.0 Flash, DeepSeek V3

IFBench

24 models compared.

Nemotron 3 Ultra 82.9%
Qwen3.8-Max 82.8%
Inkling 79.8%
Qwen3.5 397B 78.2%
GLM-5.1 76.6%
Qwen3.5 76.5%
18 more
Kimi K2.6 76%
GPT-5.2 75.4%
GLM-5.2 73.3%
LFM3-1.7B 73%
GPT-5.6 Solmax 72.7%
Nemotron 3 Super 72.6%
Nemotron 3 Nano 71.5%
Nemotron 3.5 Super VL 70.9%
Gemini 3 Pro 70.4%
Kimi K2.5 70.2%
gpt-oss-120b 68.3%
Qwen3.5-122B-A10B 67.6%
gpt-oss-20b 65%
Fable 5max 63.5%
Claude Opus 4.8max 62.2%
Claude Opus 4.5 58%
GLM-5.3-Flash 57.7%
Qwen3-30B-A3B-Thinking-2507 51%

24 models · 8 drops · Nemotron 3.5 Super VL, Qwen3.8-Max, Inkling, Nemotron 3 Ultra, Qwen3.5, Liquid AI, "Introducing LFM3", Nemotron 3 Super, Nemotron 3 Nano

LMArena text Elo

23 models compared.

Grok 4.1 1483
Gemini 3 Flash 1470
Gemini 2.5 Pro 1452
GPT-5.1 1441
o3 1427
Grok 4 Fast 1425
17 more
Grok 4 1422
Claude Opus 4.1 1419
Claude Sonnet 4.5 1419
GPT-5.2high 1419
GPT-5high 1410
Grok 4.1 Fastreasoning 1410
Gemini 2.5 Flash 1407
Grok 3 1404
GPT-4.5 1398
GPT-5 minihigh 1393
Gemini 2.0 Flash Thinking 1385
Claude Haiku 4.5 1384
GPT-4o 1377
Gemini 2.0 Flash 1355
Claude 3.7 Sonnetthinking 1304
GPT-4o mini 1273
Claude 3.5 Haiku 1238

23 models · 8 drops · Grok 4.1, Gemini 3 Flash, Claude Opus 4.1, Grok 4 Fast, Claude Sonnet 4.5, Gemini 2.5 Pro, Grok 3, Gemini 2.0 Flash

Terminal-Bench

Tasks in a real terminal (first edition).

Claude Sonnet 4.5 51%
Kimi K2 Thinkinghigh 47.1%
Claude Opus 4.1 46.5%
MiniMax-M2 46.3%
Kimi K2 0905 44.5%
GPT-5high 43.8%
15 more
Claude Haiku 4.5 41%
GLM-4.6 40.5%
Claude Opus 4 39.2%
DeepSeek V3.2-Exp 37.7%
GLM-4.5 37.5%
Claude Sonnet 4 36.4%
Claude 3.7 Sonnet 35.2%
DeepSeek V3.1 31.3%
GPT-4.1 30.3%
o3 30.2%
Gemini 2.5 Pro Preview 25.3%
Gemini 2.5 Pro 25.3%
Kimi K2 25%
DeepSeek V3 0324 13.3%
DeepSeek R1 0528 5.7%

21 models · 8 drops · Kimi K2 Thinking, Claude Sonnet 4.5, MiniMax-M2, Claude Haiku 4.5, Claude Opus 4.1, GLM-4.5, Claude Opus 4 & Sonnet 4, DeepSeek V3.1

OSWorld 2

Using a computer: apps, files and the web.

Claude Sonnet 5.5max 83.9%
Claude Opus 5.5 81.8%
Hy4 78.9%
GPT-6.1 Solmax 78%
GPT-6.1 Terramax 74.2%
GPT-6 Astra 72.6%
10 more
Claude Opus 5 70.6%
Muse Spark 1.3max 66.9%
GPT-5.6 Sol 65.7%
MiniMax M3.1 Flash 61.7%
Gemini 3.8 Flashmax 59%
Kimi K3 58.3%
GPT-5.6 Terramax 57.1%
Claude Sonnet 5 57%
GPT-6 Lunamax 55.5%
DeepSeek V4.1 Flash 54.3%

16 models · 8 drops · GPT-6.1 Terra, Claude Sonnet 5.5, Tencent Hunyuan Hy4 launch post, GPT-6 Astra, MiniMax M3.1 Flash, Muse Spark 1.3, Gemini 3.8 Flash

MCP Atlas

17 models compared.

Muse Spark 1.2xhigh 90.3%
Muse Spark Openxhigh 89.2%
Claude Opus 5xhigh 85.8%
Qwen3.8-2.4T-A95Bxhigh 84.5%
GLM-5.3max 84.2%
Fable 5 83.3%
11 more
Kimi K3max 82.3%
GPT-5.6 Sol 81.8%
GPT-5.2xhigh 68%
GLM-5high 67.8%
Gemini 3 Prohigh 66.6%
Kimi K2.5 63.8%
Claude Opus 4.5 62.3%
Gemini 3 Flashxhigh 57.4%
Claude Sonnet 4.5xhigh 43.8%
Claude Opus 4.1 40.9%
LFM3-1.7B 32.6%

17 models · 6 drops · Muse Spark 1.2, Muse Spark Open, GLM-5, Claude Opus 4.5, Gemini 3 Flash, Liquid AI, "Introducing LFM3"

Epoch Capabilities Index

93 models compared.

GPT-6 Astra 169.1
Claude Opus 5.5 168.4
GPT-6.1 Terra 168.1
GPT-6.1 Sol 167.8
Claude Sonnet 5.5 166.7
Fable 5.1 166.4
87 more
Gemini 4 Argon 166.1
Claude Opus 5 164.5
GPT-6 Sol 163.8
Fable 5 163
GPT-5.6 Sol 162.6
Muse Wonder 161.7
Claude Haiku 5.5 160.4
GPT-5.6 Terra 160.1
GPT-5.4 Pro 159.8
Claude Opus 4.8 159.6
GPT-5.5 159.4
DeepSeek V4.1 Flash 1017 158.8
Kimi K3 158.3
Gemini 3.7 Flash 158.2
Gemini 3.8 Flash 157.6
Muse Spark 1.3 157.4
GPT-5.4 157.1
Claude Sonnet 5 157
Grok 4.6 156.9
GPT-6 Luna 156.8
Claude Opus 4.7 156.7
GPT-5.6 Luna 156.7
GPT-5.2 Pro 156.5
Qwen3.8-Max 156.2
GLM-5.3 156.2
Claude Opus 4.6 155.9
Gemini 3.1 Pro 155.2
Muse Spark 1.2 155.2
Muse Spark Open 155.1
Gemini 3.5 Flash 155
Gemini 3.6 Flash 154.7
Grok 4.7 154.6
Grok 4.5 154.4
Muse Spark 1.1 154.2
GPT-5.2 153.8
Gemini 3 Pro 153.2
Claude Sonnet 4.6 152.9
Muse Spark 152.2
GLM 5.2 152.1
Gemini 3 Flash 152
Qwen3.8 151.8
Kimi K2.6 151.1
Mistral Large 4 150.6
Kimi K2.7 Code 150.2
Claude Opus 4.5 150.2
GPT-5 150
GPT-5.1 149.9
GLM-5.1 149.9
Inkling 148.5
Kimi K2.5 148.1
o3-pro 147.4
o3 146.9
Claude Sonnet 4.5 146.8
Grok 4 146.3
DeepSeek V3.2 146.1
Nemotron 3 Ultra 146
Kimi K2 Thinking 145.9
GLM-5 145.8
o4-mini 145.6
Grok 4 Fast 144.2
Claude Opus 4.1 143.9
Claude Opus 4 142.4
Qwen3-Max 142.4
Gemini 2.5 Pro 142.2
Claude Haiku 4.5 142.2
Gemma 4 26B A4B 142
o1 141.8
Claude Sonnet 4 141.6
Claude 3.7 Sonnet 141
o3-mini 140.3
Gemini 2.5 Flash 139.9
Kimi K2 139.9
gpt-oss-120b 139.8
DeepSeek V3.1 139.8
DeepSeek R1 138.8
Grok 3 138.3
gpt-oss-20b 137.8
GPT-4.1 136.7
GPT-4.5 136.7
GPT-4.1 mini 135
Gemini 2.0 Flash 134.7
Mistral Medium 3 134
Llama 4 Maverick 132.3
DeepSeek V3 132.3
DeepSeek V3-0324 132
Llama 4 Scout 129.6
GPT-4o 129

93 models · 5 drops · Epoch Capabilities Index (2026-11-04), Epoch Capabilities Index (Nov 3), Mistral Medium 3, Llama 4 Maverick & Scout

AA-LCR

37 models compared.

Kimi K3max 88.7%
MiMo-V2.6-Pro 86.3%
Kimi K3.1 Pro 86.3%
Fable 5.1max 85.3%
Claude Opus 5.5max 84.7%
Nanbeige4.3-33Bmax 84%
31 more
Muse Wonder Fullmax 83.7%
Muse Glimmer-30Bmax 83.3%
GPT-6 Lunamax 83.3%
GPT-6.1 Solmax 83%
GPT-6.1 Terramax 82.7%
DeepSeek V4.1 Flash 1017max 82.7%
Claude Sonnet 5.5max 82.7%
Claude Haiku 5.5max 82.7%
Ling-3.1-flash-1019 82.3%
Qwen3.8xhigh 82%
Gemini 3.8 Flashhigh 81.3%
Muse Wondermax 80.7%
GPT-6 Astramax 80.7%
Qwen3.8-Max 80.3%
MiniMax M3.1 Flash 79.7%
Gemini 4 Argonhigh 79.7%
GLM-5.3max 79.7%
Gemini 4 Carbonhigh 79.3%
Muse Spark Openxhigh 78.3%
Inklingxhigh 77.3%
Grok 4.7xhigh 76.7%
Gemma 4max 72%
Clover-35B-A3B-Preview 72%
Nemotron 3.5 Super VL 69%
Qwen3.6 35B A3B 61.1%
Qwen3-30B-A3B-Thinking-2507 59%
Gemma 4 26B A4B 57.6%
LFM3-1.7B 52.8%
Nemotron 3.5 Lightning 52%
gpt-oss-20b 32.9%
Nemotron 3 Nano 32.8%

37 models · 5 drops · Artificial Analysis (2026-11-04), Nanbeige4.3-33B model card, DiffuSpace, "Clover-35B-A3B-Preview", Nemotron 3.5 Lightning, Nemotron 3 Nano, Liquid AI, "Introducing LFM3"

CyberGym

13 models compared.

MAI-Cyber-1-Flash 96%
GPT-5.5 Cyber 85.6%
GLM-5.3 84.5%
Mythos 5 83.8%
GPT-5.6 Sol 83.6%
Gemini 3.5 Flash Cyber 83.2%
7 more
Claude Mythos Previewmax 83.1%
MiniMax M3.1 Flash 82.2%
Kimi K3 80%
GLM-5.1 68.7%
Claude Opus 4.6 66.6%
Kimi K2.5 41.3%
DeepSeek V3.2 17.3%

13 models · 5 drops · MAI-Cyber-1-Flash, MiniMax M3.1 Flash, GLM-5.3, Claude Mythos Preview, GLM-5.1

MATH-500

13 models compared.

Nemotron Nano 2 97.8%
DeepSeek R1 97.3%
Llama 3.1 Nemotron Ultra 97%
o1 96.4%
Qwen3-8B 96.3%
Mistral Medium 3 91%
7 more
DeepSeek V3 90.2%
o1-mini 90%
Llama 4 Maverick 90%
Claude 3.7 Sonnet 83%
Claude 3.5 Sonnet 78.3%
GPT-4o 76.4%
Llama 3.1 405B 66.2%

13 models · 5 drops · Nemotron Nano 2, Llama 3.1 Nemotron Ultra, DeepSeek R1, Mistral Medium 3, DeepSeek V3

SWE-bench Multilingual

13 models compared.

GPT-5.4high 76.8%
Claude Opus 4.6high 75.8%
Composer 2 73.7%
Claude Sonnet 4.5 68%
MAI-Code-1-Flash 65.5%
Kimi K2.5 65.1%
7 more
Claude Haiku 4.5 62.7%
DeepSeek V3.2-Exp 57.9%
Composer 56.9%
MiniMax-M2 56.5%
Kimi K2 0905 55.9%
GLM-4.6 53.8%
LFM3-1.7B 23%

13 models · 5 drops · Composer 2, MiniMax-M2, MAI-Code-1-Flash, Composer, Liquid AI, "Introducing LFM3"

Codeforces

12 models compared.

o3-pro 2748
o4-minihigh 2719
o3high 2706
gpt-oss-120b 2622
gpt-oss-20b 2516
Qwen3 2056
6 more
o3-minimedium 2036
DeepSeek R1 2029
Gemini 2.5 Pro 2001
o1 1891
o1-mini 1820
o1-pro 1707

12 models · 5 drops · o3-pro, gpt-oss-120b & gpt-oss-20b, Qwen3, DeepSeek R1

MMMU-Pro

16 models compared.

Claude Opus 5.5max 87.7%
GPT-6 Astramax 86.9%
Fable 5.1max 86.3%
Kimi K3.1 Promax 84.1%
Gemini 3 Flashxhigh 81.2%
Gemini 3 Proxhigh 81%
10 more
DeepSeek V4.1 Flash 1017max 80.6%
GPT-5.2xhigh 79.5%
Kimi K2.5 78.5%
Gemma 4 76.9%
Claude Opus 4.5 74%
Gemma 4 26B A4B 73.8%
Qwen3-VL-235B-A22B-Thinking 69.3%
Claude Sonnet 4.5xhigh 68%
Grok 4.1 Fast 63%
Gemma 3 27B 49.7%

16 models · 4 drops · Kimi K3.1 Pro, Gemini 3 Flash, Kimi K2.5, Gemma 4

NL2Repo

15 models compared.

Claude Opus 5 75.3%
DeepSeek V4.1 Flash 1017 68.2%
DeepSeek V4.1 Flash 65.4%
MiniMax M3.1 Flash 61.2%
Kimi K3 58%
GLM-5.3 58%
9 more
GPT-5.6 Sol 56.8%
Nanbeige4.3-33Bmax 53%
Claude Opus 4.6 49.8%
Qwen3.8max 48.3%
GLM-5.1 42.7%
GPT-5.4 41.3%
Gemini 3.1 Pro 33.4%
Kimi K2.5 32%
Muse Glimmer-30Bmax 27.4%

15 models · 4 drops · MiniMax M3.1 Flash, Nanbeige4.3-33B model card, GLM-5.1

CharXiv Reasoning

14 models compared.

Muse Spark 86.4%
GPT-5.6 Terra 85.9%
Gemini 3.6 Flashmax 85.2%
Gemini 3.7 Flashhigh 84.5%
Qwen3.5-122B-A10B 83.5%
GPT-5.4xhigh 82.8%
8 more
GPT-5.6 Lunamax 82.7%
GLM-5.3-Flash 81.9%
Grok 4.5max 81.6%
Nemotron 3.5 Super VL 80.8%
Gemini 3.1 Prohigh 80.2%
Inkling Small 77.8%
Claude Sonnet 5max 77%
Claude Opus 4.6max 65.3%

14 models · 4 drops · Muse Spark, Gemini 3.7 Flash, Nemotron 3.5 Super VL, Gemini 3.6 Flash

OSWorld 2 (offline subset)

Using a computer, on the 82 OSWorld 2.1 tasks that need no internet.

Claude Opus 5.5max 87.2%
Claude Sonnet 5.5 83.9%
Gemini 4 Carbonmax 81.5%
GPT-6 Astramax 80.9%
Fable 5.1max 76.8%
Gemini 4 Argonmax 74.7%
7 more
Claude Haiku 5.5max 72.4%
GPT-6.1 Solmax 71.4%
Claude Opus 5max 70.2%
GPT-5.6 Sol 65.7%
GPT-6 Solmax 64.4%
GPT-6 Luna 48.9%
Claude Haiku 4.5 15.7%

13 models · 4 drops · Gemini 4 Carbon, Claude Haiku 5.5, GPT-6.1 Sol, GPT-6 Sol & GPT-6 Luna

AA-Omniscience hallucination rate

Lower is better.

Muse Wonder Fullmax 23%
Muse Wondermax 26.5%
Grok 4.7 29%
Grok 4.6 34%
GPT-6.1 Terra 38%
Claude Sonnet 5.5 47%
6 more
GPT-6 Astra 51.3%
Claude Opus 5.5 58.6%
Kimi K2.5 64%
Fable 5.1 72.6%
Kimi K2 Thinking 74%
GPT-5.6 Terra 88%

12 models · 4 drops · Muse Wonder, Grok 4.7, GPT-6.1 Terra, Kimi K2.5

AIME 2026

12 models compared.

GLM-5.2 99.2%
Inkling 97.1%
Kimi K2.6 96.4%
Nanbeige4.3-33Bmax 95.2%
Muse Glimmer-30Bmax 94.7%
Qwen3.8max 94.5%
6 more
Nemotron 3 Ultra 94.2%
MAI-Code-1-Flash 92.5%
Gemma 4 89.2%
Gemma 4 26B A4B 88.3%
Claude Haiku 4.5 83.3%
Gemma 3 27B 20.8%

12 models · 4 drops · Inkling, Nanbeige4.3-33B model card, MAI-Code-1-Flash, Gemma 4

τ²-bench Retail

12 models compared.

Claude Opus 4.1 86.8%
Gemma 4 86.4%
Claude Sonnet 4.5 86.2%
Gemma 4 26B A4B 85.5%
Claude Sonnet 4 83.8%
Claude Opus 4 81.8%
6 more
GPT-5high 81.1%
GPT-5.1high 77.9%
GPT-4.1 74.8%
Kimi K2 70.6%
DeepSeek V3 0324 69.1%
Gemma 3 27B 6.6%

12 models · 4 drops · Claude Sonnet 4.5, Gemma 4, Kimi K2, GPT-5.1

JobBench

11 models compared.

Claude Opus 5max 65.7%
Muse Spark 1.3max 64.9%
MiMo-V2.6-Pro 62%
Muse Spark Openxhigh 61.8%
GLM-5.3max 61.4%
Kimi K3max 54.3%
5 more
Qwen3.8-2.4T-A95Bxhigh 52.7%
GPT-5.6 Solmax 45.4%
Nanbeige4.3-33Bmax 44.8%
Qwen3.8max 34.5%
Gemma 4max 9%

11 models · 4 drops · MiMo-V2.6-Pro, Muse Spark 1.3, Muse Spark Open, Nanbeige4.3-33B model card

Terminal-Bench 4.0 (standard harness)

Terminal-Bench run by a third party on one shared harness.

Kimi K3.1 Pro 67.2%
Gemini 4 Carbonhigh 65.2%
Claude Sonnet 5.5max 63.6%
Claude Opus 5.5max 59.6%
GPT-6 Astramax 59.1%
Gemini 4 Argonhigh 57.1%
23 more
Muse Wonder Fullmax 57.1%
GPT-6.1 Solmax 56.1%
GPT-6.1 Terramax 55.6%
Fable 5.1max 52%
Muse Wondermax 52%
GLM-5.3max 41.9%
Qwen3.8-Max 38.9%
MiMo-V2.6-Pro 34.8%
DeepSeek V4.1 Flash 1017max 33.8%
Ling-3.1-flash-1019 32.8%
Claude Haiku 5.5max 32.8%
DeepSeek V4.1 Flash 27%
Grok 4.7xhigh 25.8%
MiniMax M3.1 Flash 25.8%
Mistral Large 4high 24.7%
Gemini 3.8 Flashhigh 19.7%
DeepSeek V4 Pro 0813 14.1%
Kimi K3max 12.6%
GPT-6 Lunamax 12.6%
Muse Spark Openxhigh 8.1%
Qwen3.8xhigh 5.6%
Inklingxhigh 1%
Nemotron 3.5 Super VL 0%

29 models · 3 drops · Artificial Analysis (2026-11-04), Grok 4.7, Mistral Large 4

GDP.pdf

Work products built from PDFs (Surge AI).

Kimi K3.1 Promax 34.6%
GPT-6 Astramax 34.2%
Gemini 3.7 Flashhigh 34%
GPT-6.1 Solhigh 32%
Claude Opus 5.5max 30.6%
Claude Sonnet 5 28%
7 more
GPT-6 Sol 28%
Fable 5.1max 27.6%
GPT-5.6 Terra 24.7%
Gemini 3.6 Flash 22%
DeepSeek V4.1 Flash 1017max 17.2%
Muse Spark 1.2 16%
GLM-5.3max 11.2%

13 models · 3 drops · Kimi K3.1 Pro, Gemini 3.7 Flash, GPT-6.1 Sol

GPQA

13 models compared.

GPT-5.2 92.4%
Gemini 3 Pro 91.9%
Qwen3.5 88.4%
Grok 4 87.7%
Kimi K2.5 87.6%
Claude Opus 4.5 87%
7 more
Gemini 2.5 Pro 84.4%
o3 82.7%
DeepSeek R1 0528 81.3%
o3-minihigh 79.7%
GLM-4.5 79.1%
GPT-4.5 71.4%
GPT-4o 53.6%

13 models · 3 drops · Qwen3.5, GLM-4.5, GPT-4.5

ExploitBench

11 models compared.

GPT-6 Astramax 100%
Claude Opus 5.5max 91%
Fable 5.1max 83%
Mythos 5 78%
GPT-5.6 Sol 76.5%
Kimi K3.1 Promax 74.7%
5 more
Claude Mythos Preview 69%
GLM-5.3 54.4%
Claude Opus 4.8 40%
GPT-5.5 34%
Kimi K3 32.2%

11 models · 3 drops · Kimi K3.1 Pro, GLM-5.3, Fable 5 / Mythos 5

Terminal-Bench 3.0

11 models compared.

Claude Opus 5 43.5%
GPT-5.6 Solmax 34.6%
Fable 5 33.7%
GLM-5.3 28.3%
Grok 4.6high 26%
GPT-5.6 Terra 20.8%
5 more
Kimi K3 17.4%
Grok 4.5high 15.7%
Gemini 3.7 Flashhigh 14.9%
Claude Sonnet 5 14.6%
Gemini 3.6 Flash 5.4%

11 models · 3 drops · Grok 4.6, GLM-5.3, Gemini 3.7 Flash

ZeroBench (with tools)

11 models compared.

Kimi K3.1 Promax 63%
Claude Opus 5.5max 61%
Fable 5.1max 59%
GPT-6 Astramax 58%
GPT-5.6 Sol 53%
Claude Opus 5 52%
5 more
DeepSeek V4.1 Flash 1017 51%
Muse Spark Openxhigh 50%
DeepSeek V4.1 Flash 49%
Kimi K3max 41%
Inkling 14%

11 models · 3 drops · Kimi K3.1 Pro, DeepSeek V4.1 Flash 1017, Muse Spark Open

BabyVision (with tools)

10 models compared.

Claude Opus 5.5max 97.7%
GPT-6 Astramax 96.6%
Kimi K3.1 Promax 95.4%
Fable 5.1max 95.1%
Claude Opus 5 94.1%
DeepSeek V4.1 Flash 1017 92.5%
4 more
DeepSeek V4.1 Flash 89.6%
Particle-1 89.2%
GPT-5.6 Sol 88.9%
Kimi K3 85.7%

10 models · 3 drops · Kimi K3.1 Pro, DeepSeek V4.1 Flash 1017, Particle, "Introducing Particle-1"

FrontierCode v1.1

Hard, long coding tasks.

Claude Opus 5.5 54.4%
GPT-6 Astra 53.3%
Claude Sonnet 5.5xhigh 52.1%
Fable 5.1 50.3%
GPT-6 Sol 49.3%
Claude Opus 5 48%
3 more
Claude Haiku 5.5max 46.4%
Claude Sonnet 5 42.4%
GPT-6 Luna 42.4%

9 models · 3 drops · Claude Sonnet 5.5, Claude Opus 5.5, Claude Haiku 5.5

GDPval

Work products judged against industry experts (win or tie rate).

GPT-5.4 83%
GPT-5.2xhigh 70.9%
GPT-5.3-Codex 70.9%
GPT-5xhigh 38.8%

4 models · 3 drops · GPT-5.4, GPT-5.2

AA-Briefcase v1.1

29 models compared.

Claude Sonnet 5.5max 1813 Elo
Claude Opus 5.5max 1800 Elo
Muse Wonder Fullmax 1729 Elo
Kimi K3.1 Pro 1675 Elo
Fable 5.1max 1669 Elo
Muse Wondermax 1659 Elo
23 more
Grok 4.7xhigh 1637 Elo
Gemini 4 Carbonhigh 1610 Elo
Qwen3.8-Max 1607 Elo
Claude Haiku 5.5max 1574 Elo
DeepSeek V4.1 Flash 1017max 1570 Elo
GPT-6 Astramax 1569 Elo
GPT-6.1 Solmax 1555 Elo
Grok 4.6high 1546 Elo
Ling-3.1-flash-1019 1522 Elo
MiMo-V2.6-Pro 1512 Elo
GLM-5.3max 1503 Elo
Kimi K3max 1497 Elo
GPT-5.6 Solmax 1487 Elo
Gemini 4 Argonhigh 1484 Elo
MiniMax M3.1 Flash 1475 Elo
GPT-6.1 Terramax 1401 Elo
Qwen3.8xhigh 1393 Elo
Mistral Large 4high 1348 Elo
GPT-6 Lunamax 1337 Elo
Muse Spark Openxhigh 1278 Elo
Gemini 3.8 Flashhigh 1203 Elo
Nemotron 3.5 Super VL 999 Elo
Inklingxhigh 833 Elo

29 models · 2 drops · Artificial Analysis (2026-11-04), Grok 4.7

AutomationBench-AA

Business automations, run by Artificial Analysis.

Gemini 4 Carbonhigh 77.8%
Gemini 4 Argonhigh 77.5%
Kimi K3.1 Pro 73.3%
MiniMax M3.1 Flash 72.5%
Claude Sonnet 5.5max 71.8%
Muse Wonder Fullmax 71.3%
21 more
Claude Opus 5.5max 69.5%
GPT-6 Astramax 68.5%
Muse Wondermax 68.1%
Ling-3.1-flash-1019 67.5%
DeepSeek V4.1 Flash 1017max 66.6%
Grok 4.7xhigh 65.6%
GPT-6.1 Solmax 64.9%
GLM-5.3max 62.2%
GPT-6.1 Terramax 60.2%
Gemini 3.8 Flashhigh 59.9%
Fable 5.1max 59.4%
MiMo-V2.6-Pro 58.6%
Kimi K3max 58.3%
Mistral Large 4high 57.8%
Qwen3.8-Max 56.2%
GPT-6 Lunamax 53.2%
Muse Spark Openxhigh 49.7%
Qwen3.8xhigh 48.2%
Claude Haiku 5.5max 35.4%
Nemotron 3.5 Super VL 14.8%
Inklingxhigh 5%

27 models · 2 drops · Artificial Analysis (2026-11-04)

APEX-Agents

11 models compared.

Claude Opus 5.5max 73.5%
Fable 5.1max 68.6%
Kimi K3.1 Promax 67.2%
GPT-6 Astramax 64.7%
DeepSeek V4.1 Flash 1017max 63.4%
Claude Opus 5 60.6%
5 more
Fable 5max 59.2%
Grok 4.6high 57.5%
GPT-5.6 Solmax 56.7%
GLM-5.3max 56.6%
Grok 4.5high 47.1%

11 models · 2 drops · Kimi K3.1 Pro, Grok 4.6

Agents' Last Exam v1

9 models compared.

Claude Opus 5.5max 63.2%
GPT-6.1 Terramax 60%
GPT-6 Astramax 59.3%
GPT-6.1 Solmax 58.6%
GPT-6 Solmax 56.4%
GPT-5.6 Solmax 52.8%
3 more
Claude Opus 5max 52.7%
GPT-6 Lunamax 50.9%
GPT-5.6 Terramax 50.7%

9 models · 2 drops · GPT-6.1 Terra, GPT-6 Sol & GPT-6 Luna

Chartography (with tools)

9 models compared.

GPT-6 Astramax 91%
Kimi K3.1 Promax 90.2%
Claude Opus 5.5max 89%
Fable 5.1max 88.4%
Claude Opus 5 84%
DeepSeek V4.1 Flash 1017 83%
3 more
GPT-5.6 Sol 79.9%
DeepSeek V4.1 Flash 78.9%
Kimi K3 68.1%

9 models · 2 drops · Kimi K3.1 Pro, DeepSeek V4.1 Flash 1017

PinchBench

9 models compared.

Kimi K2.6 90.2%
Nemotron 3 Ultra 90%
Qwen3.6 35B A3B 88.1%
Qwen3.5 397B 86.6%
Nemotron 3.5 Lightning 85.4%
GLM-5.1 81.2%
3 more
Gemma 4 26B A4B 74.7%
Nemotron 3 Nano 66.1%
gpt-oss-20b 57.2%

9 models · 2 drops · Nemotron 3 Ultra, Nemotron 3.5 Lightning

Agents' Last Exam (CLI)

8 models compared.

Kimi K3.1 Promax 44.2%
Claude Opus 5.5max 40.7%
GPT-6 Astramax 38.8%
DeepSeek V4.1 Flash 1017max 36%
GPT-5.6 Sol 28.6%
GLM-5.3 28.5%
2 more
Kimi K3 27.6%
Fable 5 23.8%

8 models · 2 drops · Kimi K3.1 Pro, GLM-5.3

HealthBench Professional

8 models compared.

Mythos 5 66%
Claude Mythos Preview 64.7%
Fable 5.1max 62.1%
GPT-5.6 Solmax 60.5%
Claude Opus 4.8 56.9%
Grok 4.7xhigh 56.7%
2 more
GPT-5.5 51.8%
Grok 4.6high 48.5%

8 models · 2 drops · Fable 5 / Mythos 5, Grok 4.7

OSWorld

8 models compared.

Claude Opus 5.5max 81.8%
Fable 5.1max 80.7%
Kimi K3.1 Promax 79.6%
GPT-6 Astramax 75.8%
DeepSeek V4.1 Flash 1017max 69.3%
Claude Sonnet 4.5 61.4%
2 more
Claude Opus 4.1 44.4%
Claude Sonnet 4 42.2%

8 models · 2 drops · Kimi K3.1 Pro, Claude Sonnet 4.5

SWE-Marathon

8 models compared.

Muse Wonder Fullxhigh 57.5%
Muse Wondermax 53.1%
Kimi K3.1 Promax 51.7%
Claude Opus 5.5max 46.3%
Fable 5.1max 45.6%
GPT-6 Astramax 42.5%
2 more
GLM-5.3max 42.5%
DeepSeek V4.1 Flash 1017max 40%

8 models · 2 drops · Muse Wonder, Kimi K3.1 Pro

Toolathlon-Verified

8 models compared.

Claude Opus 5 80.6%
MiMo-V2.6-Pro 76.9%
Kimi K3 76.5%
GPT-5.6 Sol 74.9%
Nanbeige4.3-33Bmax 70.4%
Qwen3.8max 68.5%
2 more
Muse Glimmer-30Bmax 47.2%
Gemma 4max 27.8%

8 models · 2 drops · MiMo-V2.6-Pro, Nanbeige4.3-33B model card

Vals Index

8 models compared.

Gemini 4 Argonmax 68.9%
Claude Opus 5.5 67%
Fable 5.1 65.8%
GPT-6 Astra 63.1%
GLM-5.3 53.5%
Kimi K3 50.3%
2 more
Mistral Large 4 48.1%
DeepSeek V4 Pro 0813 47.6%

8 models · 2 drops · Gemini 4 Argon, Mistral Large 4

Video-MMMU

8 models compared.

Gemini 3 Pro 87.6%
Kimi K2.5 86.6%
GPT-5.2xhigh 85.9%
Claude Opus 4.5 84.4%
Gemini 2.5 Pro 83.6%
GPT-5.1 80.4%
2 more
Qwen3-VL-235B-A22B-Thinking 80%
Claude Sonnet 4.5 77.8%

8 models · 2 drops · Kimi K2.5, Gemini 3 Pro

BFCL v3

7 models compared.

Qwen3 70.8%
o1 67.8%
Nemotron Nano 2 66.9%
Qwen3-8B 66.3%
o3-minimedium 64.6%
Gemini 2.5 Pro 62.9%
1 more
DeepSeek R1 56.9%

7 models · 2 drops · Qwen3, Nemotron Nano 2

DeepSearchQA

7 models compared.

GPT-5.6 Solmax 93.1%
Claude Opus 5max 90.4%
Muse Spark 1.3max 90.3%
Muse Spark 74.8%
Claude Opus 4.6max 73.7%
GPT-5.4xhigh 73.6%
1 more
Gemini 3.1 Prohigh 69.7%

7 models · 2 drops · Muse Spark 1.3, Muse Spark

HLE-Verified

7 models compared.

Gemini 3.8 Flashmax 54.9%
GPT-5.6 Solmax 54.5%
Claude Opus 5max 54.4%
Gemini 3.7 Flashhigh 53.6%
Gemini 3.6 Flash 51.2%
GPT-5.6 Terramax 51.1%
1 more
Claude Sonnet 5max 31%

7 models · 2 drops · Gemini 3.8 Flash, Gemini 3.7 Flash

HMMT 2025

7 models compared.

Grok 4 Fast 93.3%
GPT-5high 93.3%
Grok 4 90%
GPT-5 minihigh 87.8%
Gemini 2.5 Pro 82.5%
o3 77.5%
1 more
Claude Opus 4 58.3%

7 models · 2 drops · Grok 4 Fast, Grok 4

SimpleQA Verified

7 models compared.

Gemini 3 Pro 72.1%
Gemini 3 Flashxhigh 68.7%
Gemini 2.5 Pro 54.5%
GPT-5.2xhigh 38%
GPT-5.1 34.9%
Claude Sonnet 4.5xhigh 29.3%
1 more
Grok 4.1 Fast 19.5%

7 models · 2 drops · Gemini 3 Pro, Gemini 3 Flash

Terminal-Bench 2.1 (standard harness)

7 models compared.

Claude Opus 5 89.1%
GPT-5.6 Terra 88%
GPT-5.6 Sol 88%
Gemini 3.8 Flashhigh 87.6%
Muse Spark 1.3xhigh 85.4%
Grok 4.5 81.6%
1 more
Muse Spark 1.2 80.1%

7 models · 2 drops · Muse Spark 1.3, Muse Spark 1.2

AA-Briefcase

6 models compared.

Claude Opus 5 1715 Elo
Grok 4.6high 1577 Elo
Fable 5max 1574 Elo
Ling-3.1-flash-1019 1524 Elo
GPT-5.6 Solmax 1502 Elo
Grok 4.5high 1313 Elo

6 models · 2 drops · Grok 4.6, Ling-3.1-flash-1019 model card

CursorBench

6 models compared.

GPT-5.4high 63.9%
Composer 2 61.3%
GPT-5.3-Codex 59.1%
Claude Opus 4.6high 58.2%
Composer 38%
Kimi K2.5 36%

6 models · 2 drops · Composer 2, Composer

CursorBench 4.0

Long coding tasks inside Cursor.

Claude Opus 5.5 57.8%
Fable 5.1max 51.8%
Claude Opus 5 46.6%
Grok 4.7xhigh 46.3%
GPT-5.6 Solmax 41.7%
Grok 4.6high 40.4%

6 models · 2 drops · Claude Opus 5.5, Grok 4.7

Multi-IF

6 models compared.

LFM3-1.7B 89.1%
Gemini 2.5 Pro 77.8%
Qwen3 71.9%
DeepSeek R1 67.7%
o1 48.8%
o3-minimedium 48.4%

6 models · 2 drops · Liquid AI, "Introducing LFM3", Qwen3

TAU-bench Airline

6 models compared.

Claude Sonnet 4 60%
Claude Opus 4 59.6%
Claude 3.7 Sonnet 58.4%
Claude Opus 4.1 56%
o3 52%
GPT-4.1 49.4%

6 models · 2 drops · Claude Opus 4.1, Claude Opus 4 & Sonnet 4

Artificial Analysis Coding Agent Index

5 models compared.

Claude Opus 5.5 62
Gemini 4 Carbon 59
Grok 4.7 56
Gemini 4 Argon 53
Grok 4.6 47

5 models · 2 drops · Gemini 4 Carbon, Grok 4.7

FrontierMath

5 models compared.

GPT-5.1high 26.7%
GPT-5high 26.3%
o3-minihigh 9.2%
o1-mini 5.8%
o1 5.5%

5 models · 2 drops · GPT-5.1, o3-mini

FrontierSWE v2

Frontier software engineering tasks.

GPT-6 Astramax 65.5%
Claude Opus 5.5max 62.3%
Gemini 4 Carbonmax 61.2%
Fable 5.1max 56.3%
Gemini 4 Argonmax 55%

5 models · 2 drops · Gemini 4 Carbon

Professional Capture-the-Flag Challenges

5 models compared.

GPT-5.2-Codex 88.1%
GPT-5.2 82.4%
GPT-5.1-Codex-Max 75.6%
GPT-5-Codex 50.5%
GPT-5.1xhigh 43%

5 models · 2 drops · GPT-5.2-Codex, GPT-5.1-Codex-Max

RULER @ 1M

5 models compared.

Nemotron 3 Ultra 94.7%
Nemotron 3 Super 91.8%
Qwen3.5-122B-A10B 91.3%
Qwen3.5 397B 90.1%
gpt-oss-120b 22.3%

5 models · 2 drops · Nemotron 3 Ultra, Nemotron 3 Super

SWE-Lancer IC Diamond

5 models compared.

GPT-5.3-Codex 81.4%
GPT-5.1-Codex-Maxxhigh 79.9%
GPT-5.2-Codex 76%
GPT-5.2high 74.6%
GPT-5.1-Codexhigh 66.3%

5 models · 2 drops · GPT-5.3-Codex, GPT-5.1-Codex-Max

MRCR (1M)

4 models compared.

Gemini 1.5 Pro 82.6%
Gemini 1.5 Flash 71.9%
Gemini 2.5 Flash 66.3%
Gemini 2.0 Flash 48.2%

4 models · 2 drops · Gemini 2.0 Flash, Gemini 2.5 Flash

PaperBench-10

4 models compared.

GPT-5.2-Codex 43%
GPT-5.1-Codex-Maxxhigh 40%
GPT-5.2 39%
GPT-5.1xhigh 34%

4 models · 2 drops · GPT-5.2-Codex, GPT-5.1-Codex-Max

CyBench

3 models compared.

Grok 4 43%
Grok 4.1 39%
Grok Code Fast 1 22.5%

3 models · 2 drops · Grok 4.1, Grok Code Fast 1

LiveBench Coding

3 models compared.

o3-minihigh 84.6%
o1 67.4%
o1-preview 52.3%

3 models · 2 drops · o3-mini, o1

MASK dishonesty rate

Lower is better.

Grok 4 43%
Grok 4.1 49%
Grok Code Fast 1 71.9%

3 models · 2 drops · Grok 4.1, Grok Code Fast 1

From a single launch

FrontierCode 2.0

Real coding tasks in open-source repos (Cognition), run by Artificial Analysis.

Claude Opus 5.5max 46.4%
GPT-6 Astramax 42.4%
GPT-6.1 Terramax 42.3%
Claude Sonnet 5.5max 41.7%
Gemini 4 Carbonhigh 40.6%
GPT-6.1 Solmax 40.6%
21 more
Muse Wonder Fullmax 37.8%
Claude Haiku 5.5max 34.9%
Fable 5.1max 31.6%
Hy4 31%
Grok 4.7xhigh 28.7%
DeepSeek V4.1 Flash 1017max 28.5%
Muse Wondermax 27.9%
Gemini 4 Argonhigh 25.7%
Kimi K3.1 Pro 25.5%
GPT-6 Lunamax 25.2%
MiMo-V2.6-Pro 24%
Qwen3.8-Max 23.4%
GLM-5.3max 21.6%
MiniMax M3.1 Flash 20.8%
Gemini 3.8 Flashhigh 18.7%
Ling-3.1-flash-1019 17%
Mistral Large 4high 15%
Qwen3.8xhigh 14.9%
Muse Spark Openxhigh 7.6%
Nemotron 3.5 Super VL 6.1%
Inklingxhigh 3.9%

27 models · 1 drop · Artificial Analysis (2026-11-04), Tencent Hunyuan Hy4 launch post

Artificial Analysis Intelligence Index v5.0

26 models compared.

Gemini 4 Carbonhigh 60
Claude Opus 5.5max 59.1
GPT-6 Astramax 56.9
GPT-6.1 Solmax 54.6
Claude Sonnet 5.5max 54.5
GPT-6.1 Terramax 51.3
20 more
Gemini 4 Argonhigh 50.4
Muse Wonder Fullmax 50.1
Fable 5.1max 49.2
Kimi K3.1 Pro 46.5
Muse Wondermax 43.6
Claude Haiku 5.5max 36.5
Grok 4.7xhigh 35.5
Qwen3.8-Max 33.7
DeepSeek V4.1 Flash 1017max 32.6
GLM-5.3max 31.7
Gemini 3.8 Flashhigh 30.8
MiMo-V2.6-Pro 29.9
Ling-3.1-flash-1019 29.6
MiniMax M3.1 Flash 29.1
GPT-6 Lunamax 27.9
Mistral Large 4high 26.1
Muse Spark Openxhigh 24.9
Qwen3.8xhigh 20.1
Nemotron 3.5 Super VL 13.6
Inklingxhigh 12.5

26 models · 1 drop · Artificial Analysis (2026-11-04)

Vals Index v2.2

24 models compared.

Claude Sonnet 5.5 70.9%
Gemini 4 Carbon 70.5%
Claude Opus 5.5 70.2%
Gemini 4 Argon 69.2%
Muse Wonder Full 66.8%
GPT-6 Astra 65%
18 more
Muse Wonder 64.9%
GPT-6.1 Sol 63.9%
Kimi K3.1 Pro 62.5%
DeepSeek V4.1 Flash 1017 61.3%
GPT-6.1 Terra 60.5%
GLM-5.3 59.7%
Qwen3.8-Max 59.1%
Claude Haiku 5.5 58.7%
Grok 4.7 58.6%
MiMo-V2.6-Pro 57.8%
Mistral Large 4 56.5%
Gemini 3.8 Flash 56%
MiniMax M3.1 Flash 53.8%
Kimi K3 53%
GPT-6 Luna 52.3%
Muse Spark Open 48.6%
Nemotron 3.5 Super VL 29.8%
Inkling 29%

24 models · 1 drop · Vals Index v2.2 (2026-11-04)

Artificial Analysis Cyber Index

14 models compared.

Kimi K3.1 Pro 64.1
DeepSeek V4.1 Flash 1017max 56.8
Grok 4.7xhigh 56.4
MiMo-V2.6-Pro 56.1
GPT-6 Lunamax 52.7
Mistral Large 4high 50.9
8 more
GLM-5.3-Flash 50
Kimi K3max 41.3
GLM-5.3max 36.4
GPT-6 Astramax 33.4
Claude Opus 5.5max 28.7
Gemini 3.8 Flashhigh 25.1
Fable 5.1max 25
Qwen3.8xhigh 12.7

14 models · 1 drop · Artificial Analysis (2026-11-04), Mistral Large 4

Apex Bench v1.1

6 models compared.

DeepSeek V4.1 Flash 1017 72.1%
Claude Opus 5 65.8%
DeepSeek V4.1 Flash 64.6%
GLM-5.3 56.6%
GPT-5.6 Sol 54.3%
Kimi K3 47.8%

6 models · 1 drop · DeepSeek V4.1 Flash 1017

BankerToolBench

6 models compared.

MiniMax M3.1 Flash 76.2
Claude Opus 5 76.2
GPT-5.6 Sol 74.6
GLM-5.3 74.5
DeepSeek V4.1 Flash 73.7
Kimi K3 69.1

6 models · 1 drop · MiniMax M3.1 Flash

Frontier Finance

6 models compared.

Fable 5.1max 60.8%
Kimi K3.1 Promax 58.6%
GLM-5.3max 57.5%
Claude Opus 5.5max 56.7%
DeepSeek V4.1 Flash 1017max 54.5%
GPT-6 Astramax 45%

6 models · 1 drop · Kimi K3.1 Pro

OfficeQA Pro v2

6 models compared.

Kimi K3.1 Promax 64.4%
Claude Opus 5.5max 62.2%
Fable 5.1max 54.5%
GPT-6 Astramax 54.4%
DeepSeek V4.1 Flash 1017max 40.4%
GLM-5.3max 31.1%

6 models · 1 drop · Kimi K3.1 Pro

Tokens per second

6 models compared.

Grok Code Fast 1 160
Gemini 2.5 Pro 92.4
Qwen3-Coder 80
Claude Sonnet 4 78.7
GPT-5 50.1
Grok 4 40.6

6 models · 1 drop · Grok Code Fast 1

Artificial Analysis Coding Agent Index v2.0

5 models compared.

Claude Opus 5.5max 62
Claude Sonnet 5.5max 59
GPT-6.1 Solmax 54
GPT-6.1 Terramax 48
GPT-6 Lunamax 31

5 models · 1 drop · GPT-6.1 Terra

BenchCAD (with tools)

5 models compared.

Claude Opus 5.5max 96.2%
GPT-6 Astramax 95.9%
Kimi K3.1 Promax 92.7%
Fable 5.1max 92.6%
DeepSeek V4.1 Flash 1017max 81.6%

5 models · 1 drop · Kimi K3.1 Pro

CyberGym-E2E

5 models compared.

Mistral Large 4 82%
MiMo-V2.6-Pro 79%
Grok 4.7 74%
Kimi K3 58%
GLM-5.3 29%

5 models · 1 drop · Mistral Large 4

EQ-Bench3

5 models compared.

Grok 4.1 1586
Kimi K2 1561
Gemini 2.5 Pro 1460
Claude Opus 4 1304
Grok 4 1206

5 models · 1 drop · Grok 4.1

Factual error rate

Lower is better.

GPT-6 Astramax 4.2%
GPT-6.1 Terramax 4.3%
GPT-6.1 Solmax 4.4%
GPT-6 Lunamax 18.4%
GPT-5.6 Terramax 22.3%

5 models · 1 drop · GPT-6.1 Terra

IFBench (loose)

5 models compared.

Gemma 4 26B A4B 77.3%
Nemotron 3 Nano 72.2%
Nemotron 3.5 Lightning 71.9%
gpt-oss-20b 68.5%
Qwen3.6 35B A3B 63.7%

5 models · 1 drop · Nemotron 3.5 Lightning

MemoryBench

5 models compared.

Muse Wondermax 85.6%
Muse Wonder Fullmax 85.5%
Claude Opus 5.5max 81.5%
GPT-6 Astramax 81.4%
Fable 5.1max 79%

5 models · 1 drop · Muse Wonder

MMMU-Pro Vision

5 models compared.

GPT-6 Astramax 86.7%
GPT-6.1 Solmax 86.3%
GPT-6.1 Terramax 85.5%
GPT-5.6 Terramax 81.7%
GPT-6 Lunamax 79.9%

5 models · 1 drop · GPT-6.1 Terra

Personal Superintelligence Index

5 models compared.

Muse Wonder Fullmax 59.1
Claude Opus 5.5max 55.2
GPT-6 Astramax 54.5
Muse Wondermax 54.3
Fable 5.1max 49.9

5 models · 1 drop · Muse Wonder

SWE-sweep

5 models compared.

Gemini 4 Carbonmax 9.6%
GPT-6 Astramax 9.5%
Gemini 4 Argonmax 4.3%
Claude Opus 5.5max 3.6%
Fable 5.1max 2.9%

5 models · 1 drop · Gemini 4 Carbon

Vals Finance Agent v2

5 models compared.

Gemini 3.8 Flashmax 61.4%
Claude Opus 5max 58.6%
GPT-5.6 Terramax 54.4%
Claude Sonnet 5max 53.9%
GPT-5.6 Solmax 53.8%

5 models · 1 drop · Gemini 3.8 Flash

τ²-Bench

5 models compared.

Claude Sonnet 4.5 84.7%
GPT-5 80.1%
MiniMax-M2 77.2%
GLM-4.6 75.9%
Kimi K2 0905 70.3%

5 models · 1 drop · MiniMax-M2

Agents' Last Exam v1.1

4 models compared.

Gemini 4 Carbonmax 77.4%
GPT-6 Astramax 73.1%
Claude Opus 5.5max 71.6%
Gemini 4 Argonmax 69%

4 models · 1 drop · Gemini 4 Carbon

ArenaHard

4 models compared.

Mistral Medium 3 97.1%
GPT-4o 95.4%
Claude 3.7 Sonnet 93.2%
Llama 4 Maverick 91.8%

4 models · 1 drop · Mistral Medium 3

Chartography

4 models compared.

Claude Sonnet 5.5 61.6%
Claude Haiku 5.5max 46.4%
GPT-6 Luna 29.1%
Claude Haiku 4.5 6.4%

4 models · 1 drop · Claude Haiku 5.5

ChartQA

4 models compared.

Llama 4 Maverick 90%
Llama 4 Scout 88.8%
Gemini 2.0 Flash 88.3%
GPT-4o 85.7%

4 models · 1 drop · Llama 4 Maverick & Scout

Claw-Eval pass^3

4 models compared.

Nanbeige4.3-33Bmax 78.3%
Qwen3.8max 75.2%
Muse Glimmer-30Bmax 57.3%
Gemma 4max 44.2%

4 models · 1 drop · Nanbeige4.3-33B model card

CursorBench 3.2

4 models compared.

Fable 5max 70.5%
Grok 4.6high 69.9%
GPT-5.6 Solmax 67.2%
Grok 4.5high 66.7%

4 models · 1 drop · Grok 4.6

DeepPlanning

4 models compared.

Nanbeige4.3-33Bmax 63.3%
Qwen3.8max 49.2%
Muse Glimmer-30Bmax 21.4%
Gemma 4max 19.4%

4 models · 1 drop · Nanbeige4.3-33B model card

DocVQA

4 models compared.

Mistral Medium 3 95.3%
Llama 4 Maverick 94.1%
GPT-4o 85.9%
Claude 3.7 Sonnet 84.3%

4 models · 1 drop · Mistral Medium 3

EEBench

4 models compared.

Grok 4.7xhigh 64%
Fable 5.1max 56.4%
Grok 4.6high 53%
GPT-5.6 Solmax 39.4%

4 models · 1 drop · Grok 4.7

Finance Agent v2

4 models compared.

GLM-5.3 55.8%
Mistral Large 4 54.7%
GPT-6 Astra 53.5%
Kimi K3 53.1%

4 models · 1 drop · Mistral Large 4

GDM-MRCR v2

4 models compared.

Gemini 3.6 Flashmax 91.8%
Grok 4.5max 81.4%
GPT-5.6 Lunamax 74.8%
Claude Sonnet 5max 71.6%

4 models · 1 drop · Gemini 3.6 Flash

GDPval rubrics

4 models compared.

Qwen3.8max 86.4%
Nanbeige4.3-33Bmax 84.8%
Muse Glimmer-30Bmax 75.8%
Gemma 4max 71.4%

4 models · 1 drop · Nanbeige4.3-33B model card

HealthBench Hard

4 models compared.

Muse Spark 42.8%
GPT-5.4xhigh 40.1%
Gemini 3.1 Prohigh 20.6%
Claude Opus 4.6max 14.8%

4 models · 1 drop · Muse Spark

HLE-Diamond (no tools)

4 models compared.

Muse Spark Openxhigh 26.1%
Kimi K3max 22.2%
Qwen3.8-2.4T-A95Bxhigh 17.3%
GLM-5.3max 16.4%

4 models · 1 drop · Muse Spark Open

HumanEval

4 models compared.

Mistral Medium 3 92.1%
Claude 3.7 Sonnet 92.1%
GPT-4o 91.5%
Llama 4 Maverick 85.4%

4 models · 1 drop · Mistral Medium 3

MathArena Apex

4 models compared.

Gemini 3 Pro 23.4%
Claude Sonnet 4.5 1.6%
GPT-5.1 1%
Gemini 2.5 Pro 0.5%

4 models · 1 drop · Gemini 3 Pro

MathVista

4 models compared.

Llama 4 Maverick 73.7%
Gemini 2.0 Flash 73.1%
Llama 4 Scout 70.7%
GPT-4o 63.8%

4 models · 1 drop · Llama 4 Maverick & Scout

Meta Internal Coding Bench

4 models compared.

Claude Opus 5max 79.4%
Muse Spark 1.2xhigh 70.6%
Muse Spark 1.1xhigh 68.3%
GPT-5.6 Terramax 65.4%

4 models · 1 drop · Muse Spark 1.2

MLE-Bench

4 models compared.

Claude Sonnet 5max 66.9%
Gemini 3.6 Flashmax 63.9%
GPT-5.6 Lunamax 47.6%
Grok 4.5max 43.2%

4 models · 1 drop · Gemini 3.6 Flash

PaperBench

4 models compared.

Qwen3.8-Max 93%
GPT-5.6 Solmax 90.5%
Fable 5max 88.8%
Claude Opus 4.8max 80.3%

4 models · 1 drop · Qwen3.8-Max

SEC-Bench Pro

4 models compared.

GPT-5.6 Solmax 71.2%
GPT-5.6 Terramax 57.7%
GPT-5.6 Lunamax 48.9%
GPT-5.5xhigh 45.8%

4 models · 1 drop · GPT-5.6 Sol, Terra, & Luna

SimpleQA hallucination rate

Lower is better.

GPT-4.5 37.1%
o1 44%
GPT-4o 61.8%
o3-mini 80.3%

4 models · 1 drop · GPT-4.5

SkillsBench

4 models compared.

Nanbeige4.3-33Bmax 61.1%
Qwen3.8max 56.4%
Muse Glimmer-30Bmax 39.7%
Gemma 4max 28.9%

4 models · 1 drop · Nanbeige4.3-33B model card

SuperGPQA

4 models compared.

Qwen3-Max 65.1%
Qwen3-235B-A22B-Instruct-2507 62.6%
DeepSeek V3.1 59.8%
Claude Opus 4 56.5%

4 models · 1 drop · Qwen3-Max

USAMO 2025

4 models compared.

Gemini Deep Think 49.4%
Grok 4 37.5%
Gemini 2.5 Pro 34.5%
o3 21.7%

4 models · 1 drop · Grok 4

τ²-Bench (Weighted)

4 models compared.

Qwen3-Max 74.8%
Claude Opus 4 67.7%
Qwen3-235B-A22B-Instruct-2507 52.9%
DeepSeek V3.1 46.4%

4 models · 1 drop · Qwen3-Max

AIME 2024 4/4 reliability

3 models compared.

o3-pro 90%
o3medium 80%
o1-pro 80%

3 models · 1 drop · o3-pro

ARC-AGI-3

3 models compared.

GPT-6 Astra 99.9%
Claude Opus 5 30.2%
GPT-5.6 Sol 7.8%

3 models · 1 drop · GPT-6 Astra

Arena AI text score

3 models compared.

Gemma 4 1452
Gemma 4 26B A4B 1441
Gemma 3 27B 1365

3 models · 1 drop · Gemma 4

Arena Hard

3 models compared.

DeepSeek R1 92%
Llama 3.1 Nemotron Ultra 87%
Llama 3.1 405B 66.2%

3 models · 1 drop · Llama 3.1 Nemotron Ultra

BioMysteryBench hard

3 models compared.

Mythos 5 46.1%
Claude Opus 4.8 40%
Claude Mythos Preview 29.6%

3 models · 1 drop · Fable 5 / Mythos 5

BioMysteryBench human-solved

3 models compared.

Mythos 5 83.9%
Claude Mythos Preview 82.6%
Claude Opus 4.8 80.4%

3 models · 1 drop · Fable 5 / Mythos 5

Codeforces 4/4 reliability

3 models compared.

o3-pro 2301
o3medium 2011
o1-pro 1423

3 models · 1 drop · o3-pro

Creative Writing v3

3 models compared.

Grok 4.1 1721.9
o3 1696.4
Claude Sonnet 4.5 1648.7

3 models · 1 drop · Grok 4.1

CVE-Bench

3 models compared.

GPT-5.2-Codex 87%
GPT-5.1-Codex-Max 80%
GPT-5.2 69%

3 models · 1 drop · GPT-5.2-Codex

Cyber Range

3 models compared.

GPT-5.1-Codex-Max 81.8%
GPT-5.2-Codex 72.7%
GPT-5.2 63.6%

3 models · 1 drop · GPT-5.2-Codex

CyberGym Level 1 leaderboard

3 models compared.

Wiz Atlas 90.9%
MAI-Cyber-1-Flash 86.3%
Claude Mythos Preview 83.1%

3 models · 1 drop · MAI-Cyber-1-Flash

Cybersecurity Capture The Flag Challenges

3 models compared.

GPT-5.3-Codex 77.6%
GPT-5.2high 67.7%
GPT-5.2-Codex 67.4%

3 models · 1 drop · GPT-5.3-Codex

Dense200

3 models compared.

Mistral Large 4 42%
GPT-6 Astra 41.5%
Kimi K3 28.9%

3 models · 1 drop · Mistral Large 4

FrontierMath Tier 4

3 models compared.

GPT-6 Astra 97.6%
Fable 5.1 87.8%
GPT-5.6 Sol 83%

3 models · 1 drop · GPT-6 Astra

GPQA Diamond 4/4 reliability

3 models compared.

o3-pro 76%
o1-pro 74%
o3medium 67%

3 models · 1 drop · o3-pro

HMMT Feb25

3 models compared.

Nemotron 3 Super 93.7%
Qwen3.5-122B-A10B 91.4%
gpt-oss-120b 90%

3 models · 1 drop · Nemotron 3 Super

MATH

3 models compared.

Gemini 2.0 Flash 89.7%
Gemini 1.5 Pro 86.5%
Gemini 1.5 Flash 77.9%

3 models · 1 drop · Gemini 2.0 Flash

MLE-Bench-30

3 models compared.

GPT-5.1-Codex-Max 17%
GPT-5.2 16%
GPT-5.2-Codex 10%

3 models · 1 drop · GPT-5.2-Codex

MRCR 512K-1M

3 models compared.

Muse Spark 1.3max 98.1%
GPT-5.6 Solmax 73.8%
Muse Spark 1.2xhigh 55.5%

3 models · 1 drop · Muse Spark 1.3

MultiChallenge

3 models compared.

GPT-4.5 43.8%
GPT-4.1 38.3%
GPT-4o 27.8%

3 models · 1 drop · GPT-4.1

Natural2Code

3 models compared.

Gemini 2.0 Flash 92.9%
Gemini 1.5 Pro 85.4%
Gemini 1.5 Flash 79.8%

3 models · 1 drop · Gemini 2.0 Flash

OpenAI-MRCR

3 models compared.

GPT-4.5 38.5
GPT-4o 31.9
GPT-4.1 2

3 models · 1 drop · GPT-4.1

SWE-Bench (OpenHands)

3 models compared.

Nemotron 3 Nano 38.8%
gpt-oss-20b 34%
Qwen3-30B-A3B-Thinking-2507 22%

3 models · 1 drop · Nemotron 3 Nano

SWE-bench Pro v2 Hard

3 models compared.

Claude Opus 5.5 100%
Fern 0.1 76.5%
Nemotron 3 Ultra 31.4%

3 models · 1 drop · Mirendil, "Introducing Fern"

SWE-Lancer Diamond

3 models compared.

GPT-4.5 32.6%
GPT-4o 23.3%
o3-minihigh 10.8%

3 models · 1 drop · GPT-4.5

τ²-bench Airline

3 models compared.

o3 64.8%
GPT-5high 62.6%
GPT-4.1 56%

3 models · 1 drop · GPT-5

AA-Omniscience

2 models compared.

Gemini 4 Carbon 54
Claude Opus 5.5 48

2 models · 1 drop · Gemini 4 Carbon

Artifacts Bench

2 models compared.

Claude Haiku 4.5 36.6%
MAI-Code-1-Flash 36.4%

2 models · 1 drop · MAI-Code-1-Flash

BFCL V2 Live

2 models compared.

Llama 3.1 Nemotron Ultra 74.1%
Llama 3.1 405B 58.7%

2 models · 1 drop · Llama 3.1 Nemotron Ultra

Code refactoring tasks

2 models compared.

GPT-5-Codex 51.3%
GPT-5high 33.9%

2 models · 1 drop · GPT-5-Codex

Code review comments per PR

2 models compared.

GPT-5high 1.3
GPT-5-Codex 0.9

2 models · 1 drop · GPT-5-Codex

Code review high-impact comments

2 models compared.

GPT-5-Codex 52.4%
GPT-5high 39.4%

2 models · 1 drop · GPT-5-Codex

Code review incorrect comments

Lower is better.

GPT-5-Codex 4.4%
GPT-5high 13.7%

2 models · 1 drop · GPT-5-Codex

FActScore

Lower is better.

Grok 4.1 3%
Grok 4 Fastnon reasoning 9.9%

2 models · 1 drop · Grok 4.1

Hallucination Rate

Lower is better.

Grok 4.1 4.2%
Grok 4 Fastnon reasoning 12.1%

2 models · 1 drop · Grok 4.1

Hermes Bench

2 models compared.

Nemotron 3.5 Super VL 63.4%
Nemotron 3 Ultra 56.3%

2 models · 1 drop · Nemotron 3.5 Super VL

Irregular medium cyber challenges

2 models compared.

GPT-5xhigh 11
GPT-5.1-Codex-Maxxhigh 9

2 models · 1 drop · GPT-5.1-Codex-Max

MGSM

2 models compared.

o1-preview 90.8%
o1 89.3%

2 models · 1 drop · o1

OpenAI PRs

2 models compared.

GPT-5.1-Codex-Maxxhigh 53%
GPT-5.1xhigh 45%

2 models · 1 drop · GPT-5.1-Codex-Max

RULER (128K)

2 models compared.

Nemotron Nano 2reasoning off 78.9%
Qwen3-8B 74.1%

2 models · 1 drop · Nemotron Nano 2

RULER-100@1M

2 models compared.

Nemotron 3 Nano 86.3%
Qwen3-30B-A3B-Thinking-2507 77.5%

2 models · 1 drop · Nemotron 3 Nano

Sycophancy rate

Lower is better.

Grok 4 7%
Grok 4.1 19%

2 models · 1 drop · Grok 4.1

UK AISI "The Last Ones" cyber range

2 models compared.

Claude Mythos Preview 22
Claude Opus 4.6 16

2 models · 1 drop · Claude Mythos Preview

Video-MME

2 models compared.

GPT-4.1 72%
GPT-4o 65.3%

2 models · 1 drop · GPT-4.1

WMDP Cyber

2 models compared.

Grok 4 79%
Grok Code Fast 1 62.1%

2 models · 1 drop · Grok Code Fast 1

CritPt v2 Hard

Unpublished research-level physics problems, hard subset, run by Artificial Analysis.

GPT-6 Astramax 70.5%
GPT-6.1 Solmax 69.5%
Gemini 4 Carbonhigh 69%
GPT-6.1 Terramax 68%
Claude Opus 5.5max 68%
Claude Sonnet 5.5max 59%
21 more
Muse Wonder Fullmax 56.5%
Gemini 4 Argonhigh 52.5%
Fable 5.1max 52.5%
Muse Wondermax 50.5%
Kimi K3.1 Pro 49.5%
Gemini 3.8 Flashhigh 46.5%
Claude Haiku 5.5max 43.5%
GPT-6 Lunamax 42.5%
Kimi K3max 40.5%
Ling-3.1-flash-1019 38.5%
MiMo-V2.6-Pro 34%
GLM-5.3max 33.5%
Grok 4.7xhigh 33.5%
MiniMax M3.1 Flash 32.5%
Qwen3.8-Max 29.5%
Muse Spark Openxhigh 29.5%
Mistral Large 4high 27%
DeepSeek V4.1 Flash 1017max 25%
Inklingxhigh 18%
Nemotron 3.5 Super VL 13.5%
Qwen3.8xhigh 4%

27 models · 0 drops · Artificial Analysis (2026-11-04)

Terminal-Bench Science (standard harness)

Terminal-Bench Science run by a third party on one shared harness.

Gemini 4 Carbonhigh 64.8%
GPT-6 Astramax 63.3%
Claude Opus 5.5max 59%
GPT-6.1 Solmax 58.1%
GPT-6.1 Terramax 57.6%
Claude Sonnet 5.5max 53.3%
21 more
Gemini 4 Argonhigh 52.9%
Fable 5.1max 43.3%
Muse Wonder Fullmax 37.1%
Kimi K3.1 Pro 34.3%
Muse Wondermax 25.7%
Claude Haiku 5.5max 20%
DeepSeek V4.1 Flash 1017max 12.9%
Qwen3.8-Max 11.9%
Grok 4.7xhigh 11.4%
Gemini 3.8 Flashhigh 10%
GLM-5.3max 9.5%
GPT-6 Lunamax 8.6%
Kimi K3max 8.1%
MiMo-V2.6-Pro 5.7%
Mistral Large 4high 4.3%
Ling-3.1-flash-1019 4.3%
MiniMax M3.1 Flash 3.3%
Muse Spark Openxhigh 1.9%
Qwen3.8xhigh 0.5%
Nemotron 3.5 Super VL 0%
Inklingxhigh 0%

27 models · 0 drops · Artificial Analysis (2026-11-04)

Release pace

Every Model Drop by month, marked with its maker.

3 Dec 2024
2 Jan 2025
3 Feb 2025
1 Mar 2025
6 Apr 2025
3 May 2025
1 Jun 2025
3 Jul 2025
6 Aug 2025
4 Sep 2025
3 Oct 2025
6 Nov 2025
5 Dec 2025
1 Jan 2026
6 Feb 2026
4 Mar 2026
9 Apr 2026
5 May 2026
7 Jun 2026
8 Jul 2026
7 Aug 2026
10 Sep 2026
18 Oct 2026
2 Nov 2026

How this board works

Scores come from the benchmark section of each handyai.news Model Drop, mostly the makers' launch tables. When drops disagree, the newest drop wins, since later launches re-run rivals on the current version of a benchmark; the model's own launch number is kept as launch_score. Settings (effort level, harness) vary between makers, so compare within a board with care. Models without their own drop are dated by the first drop that cites them.

The data is open: /api/benchmarks (JSON), /benchmarks.md (Markdown), one benchmark at /api/benchmarks/{benchmark}, the best model overall at /api/benchmarks/weigh-in (or /weigh-in/{type} for a topic), a PNG of any card by adding .png, and the list_benchmarks, get_benchmark and get_weigh_in tools on our MCP server.