The benchmark board

Every score from our Model Drops on one board, since Dec 5, 2024: who leads, how fast the best scores move, and which new models jumped furthest past the last generation.

108benchmarks

81models scored

922scores logged

114Model Drops

Best model overall

Handy AI's weigh-in: every leaderboard on this page folded into one ranking.

Best score over time

Each dot is a model at its release. The line is the record holder at each point. Hollow dots are models without a drop of their own, placed where we first saw them cited.

Open weights only

Artificial Analysis Intelligence Index

Best score rose 46.5, from 8.5 (DeepSeek V3) to 55 (Kimi K3.1 Pro).

Open weights only

GPQA Diamond

Best score rose 35.1 points, from 59.1% (DeepSeek V3) to 94.2% (Kimi K3.1 Pro).

Open weights only

SWE-bench Verified

Best score rose 38.6 points, from 42% (DeepSeek V3) to 80.6% (DeepSeek V4 Pro).

Open weights only

Humanity's Last Exam (no tools)

Best score rose 42.6 points, from 8.5% (DeepSeek R1) to 51.1% (Kimi K3.1 Pro).

Open weights only

LiveCodeBench

Best score rose 53 points, from 40.5% (DeepSeek V3) to 93.5% (DeepSeek V4).

Open weights only

AIME 2025

Best score rose 27.9 points, from 70% (DeepSeek R1) to 97.9% (gpt-oss-120b).

Open weights only

DeepSWE v1.1

Best score rose 74.3 points, from 2.7% (Gemma 4) to 77% (DeepSeek V4.1 Flash 1017).

Open weights only

Terminal-Bench 2.0

Best score rose 44.8 points, from 18.7% (gpt-oss-120b) to 63.5% (GLM-5.1).

Biggest generational jumps

A new model against the model it replaced, on the same chart in its launch post.

Gemma 3 27BGemma 4

τ²-bench Retail: 6.6% to 86.4%

+79.8

DeepSeek V3 0324DeepSeek V3.1

SWE-bench Verified: 45.4% to 66%

+20.6

Qwen3.6-27BQwen3.8

OSWorld-Verified: 63.9% to 84.3%

+20.4

DeepSeek R1DeepSeek R1 0528

AIME 2025: 70% to 87.5%

+17.5

gpt-oss-20bgpt-oss-120b

TAU-bench Retail: 54.8% to 67.8%

+13

DeepSeek V4.1 FlashDeepSeek V4.1 Flash 1017

Terminal-Bench 4.0: 31.2% to 42.3%

+11.1

Leaderboards

The latest reported score for every model on each benchmark. Tap a card's share button for an image of it.

Open weights only

Artificial Analysis Intelligence Index

62 models compared.

Kimi K3.1 Pro 55

MiMo-V2.6-Pro 46.3

DeepSeek V4.1 Flash 1017 45.1

GLM-5.3 44.8

Kimi K3 44

MiniMax M3.1 Flash 42.7

56 more

Qwen3.8-2.4T-A95B 39.9

DeepSeek V4.1 Flash 39.5

Muse Spark Open 38.9

Mistral Large 4 38.4

DeepSeek V4 Pro 0813 36

GLM-5.2 33.7

Qwen3.8 33.7

DeepSeek V4 Pro 30.4

GLM-5 27.9

Kimi K2.6 27

GLM-5.1 26.1

Inkling 25

Kimi K2.5 23.5

Qwen3.5 27B 22.9

Nemotron 3 Ultra 22.9

MiniMax M2.7 22.8

GLM-4.7 22.2

Kimi K2 Thinking 22

DeepSeek V3.2 21.5

Qwen3.6-27B 21.4

MiniMax-M2.1 20.9

MiniMax-M2 18.6

GLM-4.6 18.5

Qwen3.5 18.4

Qwen3.5 397B 18.4

Qwen3.6 35B A3B 18.2

Gemma 4 26B A4B 16.7

DeepSeek V3.2-Exp 16.6

Qwen3.5-122B-A10B 15.6

Kimi K2 0905 15.3

Gemma 4 14.7

DeepSeek V3.1 13.5

DeepSeek R1 0528 13.1

Nemotron 3.5 Lightning 12.9

GLM-4.5 12.8

Nemotron 3 Super 12.8

Kimi K2 12.7

Qwen3-235B-A22B-Thinking-2507 12.7

Qwen3-Coder 11.9

gpt-oss-120b 11.6

DeepSeek R1 11.4

Llama 4 Maverick 10

Qwen3-30B-A3B-Thinking-2507 9.8

DeepSeek V3-0324 9.7

DeepSeek V3 0324 9.7

Qwen3 9.5

Mistral Medium 3 9

gpt-oss-20bhigh 9

Nemotron 3 Nano 8.9

DeepSeek V3 8.5

Qwen3 235B 8.3

Qwen2.5 72B 7.7

Llama 3.1 Nemotron Ultra 7.5

Nemotron Nano 2 7.4

Llama 3.1 405B 7.3

Qwen3-8B 5.2

62 models · 75 drops · Kimi K3.1 Pro, DeepSeek V4.1 Flash 1017, Muse Spark Open, Mistral Large 4, Qwen3.8, Qwen3.5, Inkling, Nemotron 3 Ultra, GLM-5.1, Gemma 4, Kimi K2.5, DeepSeek V3.2, Kimi K2 Thinking, Nemotron 3.5 Lightning, Nemotron 3 Super, Qwen3-Max, DeepSeek V3.1, gpt-oss-120b & gpt-oss-20b, Grok Code Fast 1, DeepSeek R1 0528, Kimi K2, Nemotron 3 Nano, Llama 4 Maverick & Scout, Qwen3, Mistral Medium 3, DeepSeek V3, Llama 3.1 Nemotron Ultra, Nemotron Nano 2

Open weights only

GPQA Diamond

50 models compared.

Kimi K3.1 Promax 94.2%

Qwen3.8-2.4T-A95Bxhigh 93.5%

Kimi K3max 93.5%

Qwen3.8-Flash-Next 92.7%

DeepSeek V4.1 Flash 1017max 92.2%

GLM-5.3max 91.7%

44 more

Kimi K2.6 91.1%

Qwen3.7-Plusmax 90.3%

Nemotron 3.5 Super VL 89.5%

GLM-5.2 89.5%

Nanbeige4.3-33Bmax 89.5%

Qwen3.8max 89.2%

Muse Spark Openxhigh 89%

Qwen3.6-27Bmax 87.8%

Kimi K2.5 87.6%

Inkling 87.2%

Qwen3.5 397B 87.1%

Nemotron 3 Ultra 87%

Qwen3.5-122B-A10B 86.7%

GLM-5.1 86.1%

GLM-5high 86%

Kimi K2 Thinking 84.5%

Gemma 4 84.3%

Clover-35B-A3B-Preview 83.8%

Qwen3.6 35B A3B 83.4%

DeepSeek V3.2 82.4%

DeepSeek R1 0528 81%

gpt-oss-120b 80.1%

DeepSeek V3.1thinking 80.1%

Gemma 4 26B A4B 79.6%

Nemotron 3 Super 79.2%

Llama 3.1 Nemotron Ultra 76%

Nemotron 3.5 Lightning 75.4%

Kimi K2 75.1%

Nemotron 3 Nano 74.1%

DeepSeek R1 71.5%

gpt-oss-20b 71.5%

Qwen3 71.1%

Qwen3 235B 71.1%

DeepSeek V3-0324 68.4%

DeepSeek V3 0324 68.4%

Nemotron Nano 2 64%

LFM3-1.7B 63.4%

Llama 4 Maverick 61.1%

Qwen3-8B 59.6%

DeepSeek V3 59.1%

Llama 4 Scout 57.2%

Mistral Medium 3 57.1%

Llama 3.1 405B 43.4%

Gemma 3 27B 42.4%

50 models · 59 drops · Kimi K3.1 Pro, Muse Spark Open, Nemotron 3.5 Super VL, Inkling, Qwen3.8, Nanbeige4.3-33B model card, GLM-5, Nemotron 3 Ultra, DeepSeek V3.2, Gemma 4, DiffuSpace, "Clover-35B-A3B-Preview", Nemotron 3.5 Lightning, MAI-Thinking-1, DeepSeek V3.1, Nemotron 3 Super, Llama 3.1 Nemotron Ultra, Kimi K2, DeepSeek R1 0528, Qwen3, Llama 4 Maverick & Scout, Nemotron Nano 2, Liquid AI, "Introducing LFM3", Mistral Medium 3, DeepSeek V3

Open weights only

SWE-bench Verified

34 models compared.

DeepSeek V4 Pro 80.6%

Kimi K2.6 80.2%

GLM-5.3-Flash 79.1%

GLM-5high 77.8%

Inkling 77.6%

Kimi K2.5 76.8%

28 more

Qwen3.5 76.4%

Nemotron 3 Ultra 73.9%

Clover-35B-A3B-Preview 73.6%

DeepSeek V3.2 73.1%

Qwen3.5-122B-A10B 71.9%

Kimi K2 Thinkinghigh 71.3%

Nemotron 3.5 Super VL 71.1%

Qwen3.6 35B A3B 70.1%

MiniMax-M2 69.4%

Kimi K2 0905 69.2%

GLM-4.6 68%

DeepSeek V3.2-Exp 67.8%

DeepSeek V3.1 66%

Kimi K2 65.4%

GLM-4.5 64.2%

Nemotron 3 Super 60.5%

Gemma 4 26B A4B 57.4%

gpt-oss-20b 52.4%

Qwen3-235B-A22B-Instruct-2507 52.2%

Nemotron 3.5 Lightning 51.6%

DeepSeek R1 49.2%

DeepSeek V3 0324 45.4%

DeepSeek R1 0528 44.6%

LFM3-1.7B 43.4%

DeepSeek V3 42%

gpt-oss-120b 41.9%

Nemotron 3 Nano 34.1%

Llama 3.1 405B 24.5%

34 models · 48 drops · Inkling, Nemotron 3.5 Super VL, GLM-5, Qwen3.5, DiffuSpace, "Clover-35B-A3B-Preview", MAI-Thinking-1, DeepSeek V3.2, Nemotron 3.5 Lightning, MiniMax-M2, Kimi K2 Thinking, Qwen3-Max, GLM-4.5, Nemotron 3 Super, DeepSeek R1 0528, DeepSeek V3.1, Liquid AI, "Introducing LFM3", DeepSeek V3

Open weights only

Humanity's Last Exam (no tools)

Expert-level questions, no tools.

Kimi K3.1 Promax 51.1%

Qwen3.8max 43.6%

Kimi K3 43.5%

DeepSeek V4.1 Flash 1017 42.6%

GLM-5.2 40.1%

Qwen3.8-Flash-Next 39.6%

28 more

DeepSeek V4.1 Flash 36.8%

Kimi K2.6 35.9%

Nemotron 3.5 Super VL 31.8%

Kimi K2.5 31.5%

GLM-5high 30.5%

Inkling 29.7%

Nemotron 3 Ultra 29.2%

Qwen3.5 28.7%

Qwen3.5 397B 28.5%

GLM-5.1 27.2%

DeepSeek V3.2 25.1%

Qwen3.5-122B-A10B 24.2%

Kimi K2 Thinkinghigh 23.9%

Gemma 4 19.5%

Nemotron 3 Super 18.3%

DeepSeek R1 0528 17.7%

DeepSeek V3.1thinking 15.9%

gpt-oss-120b 14.9%

GLM-4.5 14.4%

Qwen3 235B 11.8%

gpt-oss-20b 10.9%

Nemotron 3 Nano 10.6%

Qwen3-30B-A3B-Thinking-2507 9.8%

Gemma 4 26B A4B 8.7%

DeepSeek R1 8.5%

LFM3-1.7B 7.2%

Nemotron Nano 2 6.5%

Qwen3-8B 4.4%

34 models · 40 drops · Kimi K3.1 Pro, Qwen3.8, DeepSeek V4.1 Flash 1017, Inkling, Nemotron 3.5 Super VL, GLM-5.1, GLM-5, Qwen3.5, Nemotron 3 Ultra, DeepSeek V3.2, Gemma 4, Nemotron 3 Super, DeepSeek V3.1, GLM-4.5, DeepSeek R1 0528, Nemotron 3 Nano, Liquid AI, "Introducing LFM3", Nemotron Nano 2

Open weights only

Sentiment Bench

How people received the model in its first week on X, Hacker News, in the news and among trusted reviewers (Handy AI).

GLM 5.2 88

DeepSeek V4.1 Flash 1017 83.9

Kimi K2.5 83.7

Qwen3.8 83.6

Kimi K2.7 Code 82.5

Kimi K2.6 82.3

21 more

Gemma 4 82.3

Kimi K3 80.3

Kimi K2 Thinking 79.6

GLM-5.3 78.4

Nemotron 3.5 Lightning 77.3

MiniMax-M2 77.3

DeepSeek V3.2 74.8

GLM-5.1 74.3

DeepSeek V4 74

Nemotron 3 Super 71.6

MiMo-V2.6-Pro 71.3

Nemotron 3 Ultra 70.7

GLM-5 68.5

Nemotron 3.5 Super VL 68.2

Nemotron 3 Nano 68

Qwen3.5 67.9

Kimi K3.1 Pro 67.5

MiniMax M3.1 Flash 61.1

Muse Spark Open 52.7

Mistral Large 4 49.4

Inkling 48.1

27 models · 27 drops · Every model's sources, quotes and method

Open weights only

LiveCodeBench

34 models compared.

DeepSeek V4 93.5%

Kimi K2.6 89.6%

Nemotron 3 Ultra 89%

GLM-5.1 85.7%

Kimi K2.5 85%

Qwen3.5 83.6%

28 more

DeepSeek V3.2 83.3%

Kimi K2 Thinking 82.6%

Gemma 4 80%

Qwen3.5 397B 79.3%

Gemma 4 26B A4B 77.1%

DeepSeek V3.2-Exp 74.1%

LFM3-1.7B 73.5%

DeepSeek R1 0528 73.3%

Nemotron Nano 2 71.1%

Qwen3 70.7%

Nemotron 3 Nano 68.3%

Qwen3 235B 66.5%

Llama 3.1 Nemotron Ultra 66.3%

Qwen3-30B-A3B-Thinking-2507 66%

DeepSeek R1 63.5%

gpt-oss-20b 61%

Qwen3-8B 59.5%

Kimi K2 53.7%

DeepSeek V3.1 52.3%

Qwen3-235B-A22B-Instruct-2507 51.8%

DeepSeek V3 0324 46.9%

DeepSeek V3-0324 45.8%

DeepSeek V3 40.5%

Llama 4 Scout 32.8%

Mistral Medium 3 30.3%

Gemma 3 27B 29.1%

Llama 4 Maverick 28.7%

Llama 3.1 405B 28.4%

34 models · 24 drops · MAI-Thinking-1, Nemotron 3 Ultra, Qwen3.5, DeepSeek V3.2, Gemma 4, Kimi K2 Thinking, Liquid AI, "Introducing LFM3", DeepSeek R1 0528, Nemotron Nano 2, Qwen3, Nemotron 3 Nano, Llama 3.1 Nemotron Ultra, Kimi K2, Qwen3-Max, Llama 4 Maverick & Scout, DeepSeek V3, Mistral Medium 3

Open weights only

AIME 2025

18 models compared.

gpt-oss-120b 97.9%

Kimi K2 Thinkinghigh 94.5%

DeepSeek V3.2 93.1%

gpt-oss-20b 91.7%

DeepSeek V3.2-Exp 89.3%

Nemotron 3 Nano 89.1%

12 more

DeepSeek R1 0528 87.5%

Qwen3-30B-A3B-Thinking-2507 85%

Qwen3 81.5%

Qwen3 235B 81.5%

Llama 3.1 Nemotron Ultra 72.5%

Nemotron Nano 2 72.1%

Qwen3-235B-A22B-Instruct-2507 70.3%

DeepSeek R1 70%

Qwen3-8B 69.3%

DeepSeek V3.1 49.8%

Kimi K2 49.5%

DeepSeek V3 0324 46.7%

18 models · 24 drops · gpt-oss-120b & gpt-oss-20b, DeepSeek V3.2, MAI-Thinking-1, Nemotron 3 Nano, Kimi K2 Thinking, DeepSeek V3.1, Qwen3, DeepSeek R1 0528, Llama 3.1 Nemotron Ultra, Nemotron Nano 2, Qwen3-Max, Kimi K2

Open weights only

DeepSWE v1.1

Fixing issues in real repositories.

DeepSeek V4.1 Flash 1017max 77%

Kimi K3.1 Promax 75.5%

DeepSeek V4.1 Flash 74.2%

MiniMax M3.1 Flash 73.7%

MiMo-V2.6-Pro 71.9%

Kimi K3 67.5%

9 more

GLM-5.3max 66.9%

Nanbeige4.3-33Bmax 65.5%

Mistral Large 4 62%

Muse Spark Openxhigh 60%

DeepSeek V4 Pro 0813 57%

Qwen3.8-2.4T-A95Bxhigh 57%

Qwen3.8max 42.5%

Nemotron 3 Ultra 3.5%

Gemma 4max 2.7%

15 models · 24 drops · Kimi K3.1 Pro, MiniMax M3.1 Flash, MiMo-V2.6-Pro, Nanbeige4.3-33B model card, Muse Spark Open, Mistral Large 4, Mirendil, "Introducing Fern"

Open weights only

Terminal-Bench 2.0

Hard tasks in a real terminal (2025 edition).

GLM-5.1 63.5%

GLM-5high 56.2%

Qwen3.5 52.5%

Kimi K2.5 50.8%

DeepSeek V3.2 46.4%

Qwen3.5-122B-A10B 37.5%

3 more

Kimi K2 Thinking 35.7%

Nemotron 3 Super 31%

gpt-oss-120b 18.7%

9 models · 22 drops · GLM-5.1, GLM-5, Qwen3.5, MAI-Thinking-1, Nemotron 3 Super, DeepSeek V3.2

Open weights only

BrowseComp

22 models compared.

Kimi K3.1 Promax 92.1%

LongCat-2.5 91.9%

Qwen3.5 69%

GLM-5high 62%

Kimi K2.6 61.3%

Kimi K2.5 60.6%

16 more

Kimi K2 Thinkinghigh 60.2%

GLM-5.1 59.4%

DeepSeek V3.2 51.4%

Qwen3.6 35B A3B 48.7%

GLM-4.6 45.1%

Nemotron 3 Ultra 44.4%

MiniMax-M2 44%

Qwen3.5 397B 40.5%

DeepSeek V3.2-Exp 40.1%

Nemotron 3.5 Lightning 37%

DeepSeek V3.1thinking 30%

GLM-4.5 26.4%

Gemma 4 26B A4B 26.3%

Nemotron 3 Nano 13.7%

DeepSeek R1 0528 8.9%

Kimi K2 7.9%

22 models · 21 drops · Kimi K3.1 Pro, Meituan LongCat-2.5 launch post, Qwen3.5, GLM-5, Nemotron 3 Ultra, GLM-5.1, DeepSeek V3.2, Nemotron 3.5 Lightning, MiniMax-M2, Kimi K2 Thinking, DeepSeek V3.1, GLM-4.5

Open weights only

Terminal-Bench 2.1

25 models compared.

MiMo-V2.6-Pro 89.9%

LongCat-2.5 88.4%

Kimi K3max 85%

GLM-5.3max 83.9%

GLM-5.2 82.7%

Qwen3.8-2.4T-A95Bxhigh 82%

19 more

Nanbeige4.3-33Bmax 80.9%

Muse Spark Openxhigh 76.4%

Qwen3.8max 73%

Kimi K2.6 71.3%

Qwen3.7-Plusmax 64%

Inkling 63.8%

Qwen3.6-27Bmax 63.4%

GLM-5.1 59.3%

Nemotron 3 Ultra 56.4%

Clover-35B-A3B-Preview 51.7%

Qwen3.5 397B 49.9%

Gemma 4max 47.2%

Qwen3.6 35B A3B 44.4%

SenseNova-Loop-9B-Preview 40.5%

Gemma 4 26B A4B 37.2%

Nemotron 3.5 Lightning 24.6%

gpt-oss-20b 15.2%

LFM3-1.7B 12.9%

Nemotron 3 Nano 8.3%

25 models · 20 drops · MiMo-V2.6-Pro, Meituan LongCat-2.5 launch post, Muse Spark Open, Inkling, Nanbeige4.3-33B model card, Qwen3.8, Nemotron 3 Ultra, DiffuSpace, "Clover-35B-A3B-Preview", Nemotron 3.5 Lightning, SenseNova-Loop-9B-Preview model card, Liquid AI, "Introducing LFM3"

Open weights only

Terminal-Bench 4.0

Multi-hour work in a real terminal.

Kimi K3.1 Promax 65.7%

Hy4 44.4%

DeepSeek V4.1 Flash 1017max 42.3%

GLM-5.3max 41.8%

LongCat-2.5 38.5%

MiniMax M3.1 Flash 38.1%

9 more

MiMo-V2.6-Pro 34.9%

DeepSeek V4.1 Flash 31.2%

Mistral Large 4 28.3%

Nanbeige4.3-33Bmax 23.8%

Kimi K3 12.6%

DeepSeek V4 Pro 0813 10%

Qwen3.8max 6.3%

Gemma 4max 0%

Nemotron 3 Ultra 0%

15 models · 19 drops · Kimi K3.1 Pro, Tencent Hunyuan Hy4 launch post, Meituan LongCat-2.5 launch post, MiniMax M3.1 Flash, MiMo-V2.6-Pro, Mistral Large 4, Nanbeige4.3-33B model card, Mirendil, "Introducing Fern"

Open weights only

GDPval-AA

Professional work products, graded head to head (Elo).

Kimi K3.1 Promax 1846 Elo

GLM-5.3 1769 Elo

DeepSeek V4.1 Flash 1017max 1746 Elo

MiMo-V2.6-Pro 1673 Elo

Qwen3.8-2.4T-A95Bxhigh 1613 Elo

Kimi K3max 1537 Elo

2 more

Mistral Large 4 1424 Elo

Muse Spark Openxhigh 1401 Elo

8 models · 19 drops · Kimi K3.1 Pro, GLM-5.3, MiMo-V2.6-Pro, Muse Spark Open

Open weights only

Aider Polyglot

10 models compared.

DeepSeek V3.1thinking 76.3%

DeepSeek R1 0528 71.6%

Qwen3 235B 65%

Kimi K2 60%

Qwen3no thinking 59.6%

DeepSeek R1 57%

4 more

DeepSeek V3-0324 55.1%

DeepSeek V3 0324 55.1%

DeepSeek V3 48.4%

Llama 4 Maverick 15.6%

10 models · 17 drops · DeepSeek V3.1, DeepSeek R1 0528, Kimi K2, Qwen3, Llama 4 Maverick & Scout, GPT-4.5

Open weights only

MMMU

5 models compared.

Qwen3.5 85%

Kimi K2.5 84.3%

Llama 4 Maverick 71.8%

Llama 4 Scout 69.4%

Mistral Medium 3 66.1%

5 models · 17 drops · Qwen3.5, Mistral Medium 3, Llama 4 Maverick & Scout

Open weights only

AutomationBench

Real business automations (Zapier).

MiniMax M3.1 Flash 62.2%

DeepSeek V4.1 Flash 1017 60.5%

Mistral Large 4 59.9%

DeepSeek V4 Pro 0813 56.7%

Hy4 56.2%

DeepSeek V4.1 Flash 54.8%

8 more

Nanbeige4.3-33Bmax 54.2%

MiMo-V2.6-Pro 53.1%

LongCat-2.5 51.8%

GLM-5.3 48.8%

Kimi K3 46.7%

Qwen3.8max 38.3%

SenseNova-Loop-9B-Preview 13.7%

Gemma 4max 9.7%

14 models · 16 drops · MiniMax M3.1 Flash, Mistral Large 4, Tencent Hunyuan Hy4 launch post, Nanbeige4.3-33B model card, MiMo-V2.6-Pro, Meituan LongCat-2.5 launch post, SenseNova-Loop-9B-Preview model card

Open weights only

SWE-bench Pro

14 models compared.

GLM-5.2 62.1%

Qwen3.8max 61.7%

Qwen3.8-Flash-Next 60.4%

Nanbeige4.3-33Bmax 60.3%

GLM-5.3-Flash 59.7%

Kimi K2.6 58.6%

8 more

GLM-5.1 58.4%

Qwen3.7-Plusmax 57.6%

Inkling 54.3%

Kimi K2.5 53.8%

Qwen3.6-27Bmax 53.5%

Nemotron 3.5 Super VL 51.8%

Gemma 4max 51.2%

Nemotron 3 Ultra 49.7%

14 models · 15 drops · Inkling, Qwen3.8, Nemotron 3.5 Super VL, Nanbeige4.3-33B model card, MAI-Thinking-1, GLM-5.1

Open weights only

Humanity's Last Exam (with tools)

Expert-level questions, with search and code.

Kimi K3.1 Promax 70%

GLM-5.3 62.5%

Kimi K3 59.8%

DeepSeek V4.1 Flash 1017max 58.8%

GLM-5.2 54.7%

Kimi K2.6 54%

14 more

Kimi K2.5 51.8%

GLM-5high 50.4%

Inkling 46%

Kimi K2 Thinkinghigh 44.9%

DeepSeek V3.2 40.8%

Nemotron 3 Ultra 37.4%

MiniMax-M2 31.8%

GLM-4.6 30.4%

DeepSeek V3.1thinking 29.8%

DeepSeek R1 0528 24.8%

DeepSeek V3.2-Exp 20.3%

gpt-oss-120b 19%

gpt-oss-20b 17.3%

LFM3-1.7B 13.9%

20 models · 14 drops · Kimi K3.1 Pro, GLM-5.3, Inkling, GLM-5, Kimi K2 Thinking, MiniMax-M2, DeepSeek V3.1, gpt-oss-120b & gpt-oss-20b, Liquid AI, "Introducing LFM3"

Open weights only

AIME 2024

8 models compared.

GLM-4.5 91%

DeepSeek R1 0528 89.3%

Qwen3 85.7%

Qwen3 235B 85.7%

Llama 3.1 Nemotron Ultra 80.8%

DeepSeek R1 79.8%

2 more

DeepSeek V3 39.2%

Llama 3.1 405B 23.3%

8 models · 12 drops · GLM-4.5, Qwen3, DeepSeek R1 0528, Llama 3.1 Nemotron Ultra, DeepSeek V3

Open weights only

τ²-bench Telecom

14 models compared.

GLM-5.3-Flash 98.5%

Kimi K2 Thinking 93%

Qwen3.5-122B-A10B 87.2%

MiniMax-M2 86.8%

Nemotron 3 Ultra 83%

Nemotron 3.5 Super VL 73.2%

8 more

gpt-oss-120b 66%

Kimi K2 65.8%

Nemotron 3 Super 64.4%

gpt-oss-20b 49.7%

Nemotron 3 Nano 42.2%

DeepSeek V3.2-Exp 33.9%

DeepSeek V3 0324 32.5%

Qwen3-30B-A3B-Thinking-2507 26.3%

14 models · 11 drops · Nemotron 3.5 Super VL, Kimi K2 Thinking, Nemotron 3 Super, Kimi K2, Nemotron 3 Nano

Open weights only

SimpleQA

6 models compared.

Kimi K2 31%

DeepSeek R1 30.1%

DeepSeek R1 0528 27.8%

DeepSeek V3 0324 27.7%

DeepSeek V3 24.9%

Llama 3.1 405B 17.1%

6 models · 11 drops · Kimi K2, DeepSeek R1 0528, Grok 3, DeepSeek V3

Open weights only

OSWorld-Verified

Using a computer: apps, files and the web (2025 edition).

Kimi K3 84.8%

Qwen3.8max 84.3%

MiMo-V2.6-Pro 82%

Qwen3.7-Plusmax 73.3%

Qwen3.6-27Bmax 63.9%

5 models · 10 drops · MiMo-V2.6-Pro, Qwen3.8

Open weights only

MMLU-Pro

12 models compared.

Gemma 4 85.2%

DeepSeek R1 0528 84.9%

Clover-35B-A3B-Preview 84.8%

GLM-4.5 84.6%

Gemma 4 26B A4B 82.6%

DeepSeek V3-0324 81.2%

6 more

Llama 4 Maverick 80.4%

Mistral Medium 3 77.2%

DeepSeek V3 75.9%

Llama 4 Scout 74.3%

Llama 3.1 405B 73.3%

Gemma 3 27B 67.6%

12 models · 9 drops · Gemma 4, GLM-4.5, DiffuSpace, "Clover-35B-A3B-Preview", Llama 4 Maverick & Scout, Mistral Medium 3, Grok 3, DeepSeek V3

Open weights only

IFBench

17 models compared.

Nemotron 3 Ultra 82.9%

Inkling 79.8%

Qwen3.5 397B 78.2%

GLM-5.1 76.6%

Qwen3.5 76.5%

Kimi K2.6 76%

11 more

GLM-5.2 73.3%

LFM3-1.7B 73%

Nemotron 3 Super 72.6%

Nemotron 3 Nano 71.5%

Nemotron 3.5 Super VL 70.9%

Kimi K2.5 70.2%

gpt-oss-120b 68.3%

Qwen3.5-122B-A10B 67.6%

gpt-oss-20b 65%

GLM-5.3-Flash 57.7%

Qwen3-30B-A3B-Thinking-2507 51%

17 models · 8 drops · Nemotron 3.5 Super VL, Inkling, Nemotron 3 Ultra, Qwen3.5, Liquid AI, "Introducing LFM3", Nemotron 3 Super, Nemotron 3 Nano

Open weights only

Terminal-Bench

Tasks in a real terminal (first edition).

Kimi K2 Thinkinghigh 47.1%

MiniMax-M2 46.3%

Kimi K2 0905 44.5%

GLM-4.6 40.5%

DeepSeek V3.2-Exp 37.7%

GLM-4.5 37.5%

4 more

DeepSeek V3.1 31.3%

Kimi K2 25%

DeepSeek V3 0324 13.3%

DeepSeek R1 0528 5.7%

10 models · 8 drops · Kimi K2 Thinking, MiniMax-M2, GLM-4.5, DeepSeek V3.1

Open weights only

OSWorld 2

Using a computer: apps, files and the web.

Hy4 78.9%

MiniMax M3.1 Flash 61.7%

Kimi K3 58.3%

DeepSeek V4.1 Flash 54.3%

4 models · 8 drops · Tencent Hunyuan Hy4 launch post, MiniMax M3.1 Flash

Open weights only

Harvey Legal Agent Benchmark

Legal work end to end.

Kimi K3.1 Promax 23.7%

DeepSeek V4.1 Flash 1017max 21%

Mistral Large 4 15.8%

Kimi K3 12.9%

GLM-5.3 8.3%

5 models · 7 drops · Kimi K3.1 Pro, Mistral Large 4

Open weights only

MCP Atlas

7 models compared.

Muse Spark Openxhigh 89.2%

Qwen3.8-2.4T-A95Bxhigh 84.5%

GLM-5.3max 84.2%

Kimi K3max 82.3%

GLM-5high 67.8%

Kimi K2.5 63.8%

1 more

LFM3-1.7B 32.6%

7 models · 6 drops · Muse Spark Open, GLM-5, Liquid AI, "Introducing LFM3"

Open weights only

TAU-bench Retail

4 models compared.

GLM-4.5 79.7%

Kimi K2 73.9%

gpt-oss-120b 67.8%

gpt-oss-20b 54.8%

4 models · 6 drops · GLM-4.5, gpt-oss-120b & gpt-oss-20b

Open weights only

Terminal-Bench Science

Scientific computing tasks in a terminal.

Kimi K3.1 Promax 38.1%

Hy4 19.8%

DeepSeek V4.1 Flashmax 15.7%

GLM-5.3max 8.1%

4 models · 6 drops · Kimi K3.1 Pro, Tencent Hunyuan Hy4 launch post

Open weights only

Epoch Capabilities Index

27 models compared.

DeepSeek V4.1 Flash 1017 158.8

Kimi K3 158.3

GLM-5.3 156.2

Muse Spark Open 155.1

GLM 5.2 152.1

Qwen3.8 151.8

21 more

Kimi K2.6 151.1

Mistral Large 4 150.6

Kimi K2.7 Code 150.2

GLM-5.1 149.9

Inkling 148.5

Kimi K2.5 148.1

DeepSeek V3.2 146.1

Nemotron 3 Ultra 146

Kimi K2 Thinking 145.9

GLM-5 145.8

Gemma 4 26B A4B 142

Kimi K2 139.9

gpt-oss-120b 139.8

DeepSeek V3.1 139.8

DeepSeek R1 138.8

gpt-oss-20b 137.8

Mistral Medium 3 134

Llama 4 Maverick 132.3

DeepSeek V3 132.3

DeepSeek V3-0324 132

Llama 4 Scout 129.6

27 models · 5 drops · Epoch Capabilities Index (2026-11-04), Epoch Capabilities Index (Nov 3), Llama 4 Maverick & Scout

Open weights only

AA-LCR

21 models compared.

Kimi K3max 88.7%

MiMo-V2.6-Pro 86.3%

Kimi K3.1 Pro 86.3%

Nanbeige4.3-33Bmax 84%

DeepSeek V4.1 Flash 1017max 82.7%

Ling-3.1-flash-1019 82.3%

15 more

Qwen3.8xhigh 82%

MiniMax M3.1 Flash 79.7%

GLM-5.3max 79.7%

Muse Spark Openxhigh 78.3%

Inklingxhigh 77.3%

Gemma 4max 72%

Clover-35B-A3B-Preview 72%

Nemotron 3.5 Super VL 69%

Qwen3.6 35B A3B 61.1%

Qwen3-30B-A3B-Thinking-2507 59%

Gemma 4 26B A4B 57.6%

LFM3-1.7B 52.8%

Nemotron 3.5 Lightning 52%

gpt-oss-20b 32.9%

Nemotron 3 Nano 32.8%

21 models · 5 drops · Artificial Analysis (2026-11-04), Nanbeige4.3-33B model card, DiffuSpace, "Clover-35B-A3B-Preview", Nemotron 3.5 Lightning, Nemotron 3 Nano, Liquid AI, "Introducing LFM3"

Open weights only

MATH-500

8 models compared.

Nemotron Nano 2 97.8%

DeepSeek R1 97.3%

Llama 3.1 Nemotron Ultra 97%

Qwen3-8B 96.3%

Mistral Medium 3 91%

DeepSeek V3 90.2%

2 more

Llama 4 Maverick 90%

Llama 3.1 405B 66.2%

8 models · 5 drops · Nemotron Nano 2, Llama 3.1 Nemotron Ultra, Mistral Medium 3, DeepSeek V3

Open weights only

ProgramBench

7 models compared.

Kimi K3.1 Promax 31.5%

Hy4 30%

DeepSeek V4.1 Flash 1017 29.1%

LongCat-2.5 25.5%

DeepSeek V4.1 Flash 20.3%

GLM-5.3max 19%

1 more

Kimi K3 17.5%

7 models · 5 drops · Kimi K3.1 Pro, Tencent Hunyuan Hy4 launch post, DeepSeek V4.1 Flash 1017, Meituan LongCat-2.5 launch post

Open weights only

CyberGym

6 models compared.

GLM-5.3 84.5%

MiniMax M3.1 Flash 82.2%

Kimi K3 80%

GLM-5.1 68.7%

Kimi K2.5 41.3%

DeepSeek V3.2 17.3%

6 models · 5 drops · MiniMax M3.1 Flash, GLM-5.1

Open weights only

SWE-bench Multilingual

6 models compared.

Kimi K2.5 65.1%

DeepSeek V3.2-Exp 57.9%

MiniMax-M2 56.5%

Kimi K2 0905 55.9%

GLM-4.6 53.8%

LFM3-1.7B 23%

6 models · 5 drops · Composer 2, MiniMax-M2, Liquid AI, "Introducing LFM3"

Open weights only

Codeforces

4 models compared.

gpt-oss-120b 2622

gpt-oss-20b 2516

Qwen3 2056

DeepSeek R1 2029

4 models · 5 drops · gpt-oss-120b & gpt-oss-20b, Qwen3

Open weights only

AIME 2026

9 models compared.

GLM-5.2 99.2%

Inkling 97.1%

Kimi K2.6 96.4%

Nanbeige4.3-33Bmax 95.2%

Qwen3.8max 94.5%

Nemotron 3 Ultra 94.2%

3 more

Gemma 4 89.2%

Gemma 4 26B A4B 88.3%

Gemma 3 27B 20.8%

9 models · 4 drops · Inkling, Nanbeige4.3-33B model card, Gemma 4

Open weights only

NL2Repo

9 models compared.

DeepSeek V4.1 Flash 1017 68.2%

DeepSeek V4.1 Flash 65.4%

MiniMax M3.1 Flash 61.2%

Kimi K3 58%

GLM-5.3 58%

Nanbeige4.3-33Bmax 53%

3 more

Qwen3.8max 48.3%

GLM-5.1 42.7%

Kimi K2.5 32%

9 models · 4 drops · MiniMax M3.1 Flash, Nanbeige4.3-33B model card, GLM-5.1

Open weights only

JobBench

8 models compared.

MiMo-V2.6-Pro 62%

Muse Spark Openxhigh 61.8%

GLM-5.3max 61.4%

Kimi K3max 54.3%

Qwen3.8-2.4T-A95Bxhigh 52.7%

Nanbeige4.3-33Bmax 44.8%

2 more

Qwen3.8max 34.5%

Gemma 4max 9%

8 models · 4 drops · MiMo-V2.6-Pro, Muse Spark Open, Nanbeige4.3-33B model card

Open weights only

Agents' Last Exam

7 models compared.

Kimi K3.1 Promax 43.2%

Hy4 41%

DeepSeek V4.1 Flash 1017 35.4%

DeepSeek V4.1 Flash 31.8%

MiniMax M3.1 Flash 29.6%

Kimi K3 28.3%

1 more

GLM-5.3max 26.2%

7 models · 4 drops · Kimi K3.1 Pro, Tencent Hunyuan Hy4 launch post, DeepSeek V4.1 Flash 1017, MiniMax M3.1 Flash

Open weights only

MMMU-Pro

7 models compared.

Kimi K3.1 Promax 84.1%

DeepSeek V4.1 Flash 1017max 80.6%

Kimi K2.5 78.5%

Gemma 4 76.9%

Gemma 4 26B A4B 73.8%

Qwen3-VL-235B-A22B-Thinking 69.3%

1 more

Gemma 3 27B 49.7%

7 models · 4 drops · Kimi K3.1 Pro, Kimi K2.5, Gemma 4

Open weights only

τ²-bench Retail

5 models compared.

Gemma 4 86.4%

Gemma 4 26B A4B 85.5%

Kimi K2 70.6%

DeepSeek V3 0324 69.1%

Gemma 3 27B 6.6%

5 models · 4 drops · Gemma 4, Kimi K2

Open weights only

CharXiv Reasoning

4 models compared.

Qwen3.5-122B-A10B 83.5%

GLM-5.3-Flash 81.9%

Nemotron 3.5 Super VL 80.8%

Inkling Small 77.8%

4 models · 4 drops · Nemotron 3.5 Super VL

Open weights only

IFEval

4 models compared.

LFM3-1.7B 96.7%

Nemotron Nano 2 90.3%

Qwen3-8B 89.4%

DeepSeek R1 83.3%

4 models · 4 drops · Liquid AI, "Introducing LFM3", Nemotron Nano 2, Claude 3.7 Sonnet

Open weights only

AA-Omniscience hallucination rate

Lower is better.

Kimi K2.5 64%

Kimi K2 Thinking 74%

2 models · 4 drops · Kimi K2.5

Open weights only

MMLU

2 models compared.

gpt-oss-120b 90%

gpt-oss-20b 85.3%

2 models · 4 drops · gpt-oss-120b & gpt-oss-20b

Open weights only

Terminal-Bench 4.0 (standard harness)

Terminal-Bench run by a third party on one shared harness.

Kimi K3.1 Pro 67.2%

GLM-5.3max 41.9%

MiMo-V2.6-Pro 34.8%

DeepSeek V4.1 Flash 1017max 33.8%

Ling-3.1-flash-1019 32.8%

DeepSeek V4.1 Flash 27%

8 more

MiniMax M3.1 Flash 25.8%

Mistral Large 4high 24.7%

DeepSeek V4 Pro 0813 14.1%

Kimi K3max 12.6%

Muse Spark Openxhigh 8.1%

Qwen3.8xhigh 5.6%

Inklingxhigh 1%

Nemotron 3.5 Super VL 0%

14 models · 3 drops · Artificial Analysis (2026-11-04), Grok 4.7, Mistral Large 4

Open weights only

ZeroBench (with tools)

6 models compared.

Kimi K3.1 Promax 63%

DeepSeek V4.1 Flash 1017 51%

Muse Spark Openxhigh 50%

DeepSeek V4.1 Flash 49%

Kimi K3max 41%

Inkling 14%

6 models · 3 drops · Kimi K3.1 Pro, DeepSeek V4.1 Flash 1017, Muse Spark Open

Open weights only

SciCode

5 models compared.

Ling-3.1-flash-1019 57%

Nanbeige4.3-33Bmax 52.6%

Qwen3.8max 46.7%

Gemma 4max 45.5%

LFM3-1.7B 28.5%

5 models · 3 drops · Ling-3.1-flash-1019 model card, Nanbeige4.3-33B model card, Liquid AI, "Introducing LFM3"

Open weights only

Tau3-Bench

5 models compared.

LFM3-1.7B 68%

Clover-35B-A3B-Preview 65.3%

Nanbeige4.3-33Bmax 48.8%

Qwen3.8max 48.5%

Gemma 4max 18.6%

5 models · 3 drops · Liquid AI, "Introducing LFM3", DiffuSpace, "Clover-35B-A3B-Preview", Nanbeige4.3-33B model card

Open weights only

BabyVision (with tools)

4 models compared.

Kimi K3.1 Promax 95.4%

DeepSeek V4.1 Flash 1017 92.5%

DeepSeek V4.1 Flash 89.6%

Kimi K3 85.7%

4 models · 3 drops · Kimi K3.1 Pro, DeepSeek V4.1 Flash 1017

Open weights only

GPQA

4 models compared.

Qwen3.5 88.4%

Kimi K2.5 87.6%

DeepSeek R1 0528 81.3%

GLM-4.5 79.1%

4 models · 3 drops · Qwen3.5, GLM-4.5

Open weights only

ExploitBench

3 models compared.

Kimi K3.1 Promax 74.7%

GLM-5.3 54.4%

Kimi K3 32.2%

3 models · 3 drops · Kimi K3.1 Pro, GLM-5.3

Open weights only

GDP.pdf

Work products built from PDFs (Surge AI).

Kimi K3.1 Promax 34.6%

DeepSeek V4.1 Flash 1017max 17.2%

GLM-5.3max 11.2%

3 models · 3 drops · Kimi K3.1 Pro

Open weights only

Terminal-Bench 3.0

2 models compared.

GLM-5.3 28.3%

Kimi K3 17.4%

2 models · 3 drops · GLM-5.3

Open weights only

AA-Briefcase v1.1

12 models compared.

Kimi K3.1 Pro 1675 Elo

DeepSeek V4.1 Flash 1017max 1570 Elo

Ling-3.1-flash-1019 1522 Elo

MiMo-V2.6-Pro 1512 Elo

GLM-5.3max 1503 Elo

Kimi K3max 1497 Elo

6 more

MiniMax M3.1 Flash 1475 Elo

Qwen3.8xhigh 1393 Elo

Mistral Large 4high 1348 Elo

Muse Spark Openxhigh 1278 Elo

Nemotron 3.5 Super VL 999 Elo

Inklingxhigh 833 Elo

12 models · 2 drops · Artificial Analysis (2026-11-04)

Open weights only

AutomationBench-AA

Business automations, run by Artificial Analysis.

Kimi K3.1 Pro 73.3%

MiniMax M3.1 Flash 72.5%

Ling-3.1-flash-1019 67.5%

DeepSeek V4.1 Flash 1017max 66.6%

GLM-5.3max 62.2%

MiMo-V2.6-Pro 58.6%

6 more

Kimi K3max 58.3%

Mistral Large 4high 57.8%

Muse Spark Openxhigh 49.7%

Qwen3.8xhigh 48.2%

Nemotron 3.5 Super VL 14.8%

Inklingxhigh 5%

12 models · 2 drops · Artificial Analysis (2026-11-04)

Open weights only

PinchBench

9 models compared.

Kimi K2.6 90.2%

Nemotron 3 Ultra 90%

Qwen3.6 35B A3B 88.1%

Qwen3.5 397B 86.6%

Nemotron 3.5 Lightning 85.4%

GLM-5.1 81.2%

3 more

Gemma 4 26B A4B 74.7%

Nemotron 3 Nano 66.1%

gpt-oss-20b 57.2%

9 models · 2 drops · Nemotron 3 Ultra, Nemotron 3.5 Lightning

Open weights only

RULER @ 1M

5 models compared.

Nemotron 3 Ultra 94.7%

Nemotron 3 Super 91.8%

Qwen3.5-122B-A10B 91.3%

Qwen3.5 397B 90.1%

gpt-oss-120b 22.3%

5 models · 2 drops · Nemotron 3 Ultra, Nemotron 3 Super

Open weights only

Toolathlon-Verified

5 models compared.

MiMo-V2.6-Pro 76.9%

Kimi K3 76.5%

Nanbeige4.3-33Bmax 70.4%

Qwen3.8max 68.5%

Gemma 4max 27.8%

5 models · 2 drops · MiMo-V2.6-Pro, Nanbeige4.3-33B model card

Open weights only

Agents' Last Exam (CLI)

4 models compared.

Kimi K3.1 Promax 44.2%

DeepSeek V4.1 Flash 1017max 36%

GLM-5.3 28.5%

Kimi K3 27.6%

4 models · 2 drops · Kimi K3.1 Pro, GLM-5.3

Open weights only

BFCL v3

4 models compared.

Qwen3 70.8%

Nemotron Nano 2 66.9%

Qwen3-8B 66.3%

DeepSeek R1 56.9%

4 models · 2 drops · Qwen3, Nemotron Nano 2

Open weights only

Chartography (with tools)

4 models compared.

Kimi K3.1 Promax 90.2%

DeepSeek V4.1 Flash 1017 83%

DeepSeek V4.1 Flash 78.9%

Kimi K3 68.1%

4 models · 2 drops · Kimi K3.1 Pro, DeepSeek V4.1 Flash 1017

Open weights only

EnterpriseOps-Gym

4 models compared.

Nanbeige4.3-33Bmax 41%

Qwen3.8max 40.2%

LFM3-1.7B 28.2%

Gemma 4max 27.3%

4 models · 2 drops · Nanbeige4.3-33B model card, Liquid AI, "Introducing LFM3"

Open weights only

SWE-Atlas-QnA

4 models compared.

Nanbeige4.3-33Bmax 50.8%

Qwen3.8max 41.9%

Nemotron 3 Ultra 16.1%

Gemma 4max 3.3%

4 models · 2 drops · Nanbeige4.3-33B model card, Mirendil, "Introducing Fern"

Open weights only

Vals Index

4 models compared.

GLM-5.3 53.5%

Kimi K3 50.3%

Mistral Large 4 48.1%

DeepSeek V4 Pro 0813 47.6%

4 models · 2 drops · Mistral Large 4

Open weights only

APEX-Agents

3 models compared.

Kimi K3.1 Promax 67.2%

DeepSeek V4.1 Flash 1017max 63.4%

GLM-5.3max 56.6%

3 models · 2 drops · Kimi K3.1 Pro

Open weights only

Multi-IF

3 models compared.

LFM3-1.7B 89.1%

Qwen3 71.9%

DeepSeek R1 67.7%

3 models · 2 drops · Liquid AI, "Introducing LFM3", Qwen3

Open weights only

SWE-Marathon

3 models compared.

Kimi K3.1 Promax 51.7%

GLM-5.3max 42.5%

DeepSeek V4.1 Flash 1017max 40%

3 models · 2 drops · Kimi K3.1 Pro

Open weights only

OSWorld

2 models compared.

Kimi K3.1 Promax 79.6%

DeepSeek V4.1 Flash 1017max 69.3%

2 models · 2 drops · Kimi K3.1 Pro

Open weights only

Video-MMMU

2 models compared.

Kimi K2.5 86.6%

Qwen3-VL-235B-A22B-Thinking 80%

2 models · 2 drops · Kimi K2.5

From a single launch

Open weights only

FrontierCode 2.0

Real coding tasks in open-source repos (Cognition), run by Artificial Analysis.

Hy4 31%

DeepSeek V4.1 Flash 1017max 28.5%

Kimi K3.1 Pro 25.5%

MiMo-V2.6-Pro 24%

GLM-5.3max 21.6%

MiniMax M3.1 Flash 20.8%

6 more

Ling-3.1-flash-1019 17%

Mistral Large 4high 15%

Qwen3.8xhigh 14.9%

Muse Spark Openxhigh 7.6%

Nemotron 3.5 Super VL 6.1%

Inklingxhigh 3.9%

12 models · 1 drop · Tencent Hunyuan Hy4 launch post, Artificial Analysis (2026-11-04)

Open weights only

Artificial Analysis Intelligence Index v5.0

11 models compared.

Kimi K3.1 Pro 46.5

DeepSeek V4.1 Flash 1017max 32.6

GLM-5.3max 31.7

MiMo-V2.6-Pro 29.9

Ling-3.1-flash-1019 29.6

MiniMax M3.1 Flash 29.1

5 more

Mistral Large 4high 26.1

Muse Spark Openxhigh 24.9

Qwen3.8xhigh 20.1

Nemotron 3.5 Super VL 13.6

Inklingxhigh 12.5

11 models · 1 drop · Artificial Analysis (2026-11-04)

Open weights only

Vals Index v2.2

10 models compared.

Kimi K3.1 Pro 62.5%

DeepSeek V4.1 Flash 1017 61.3%

GLM-5.3 59.7%

MiMo-V2.6-Pro 57.8%

Mistral Large 4 56.5%

MiniMax M3.1 Flash 53.8%

4 more

Kimi K3 53%

Muse Spark Open 48.6%

Nemotron 3.5 Super VL 29.8%

Inkling 29%

10 models · 1 drop · Vals Index v2.2 (2026-11-04)

Open weights only

Artificial Analysis Cyber Index

8 models compared.

Kimi K3.1 Pro 64.1

DeepSeek V4.1 Flash 1017max 56.8

MiMo-V2.6-Pro 56.1

Mistral Large 4high 50.9

GLM-5.3-Flash 50

Kimi K3max 41.3

2 more

GLM-5.3max 36.4

Qwen3.8xhigh 12.7

8 models · 1 drop · Artificial Analysis (2026-11-04), Mistral Large 4

Open weights only

IFBench (loose)

5 models compared.

Gemma 4 26B A4B 77.3%

Nemotron 3 Nano 72.2%

Nemotron 3.5 Lightning 71.9%

gpt-oss-20b 68.5%

Qwen3.6 35B A3B 63.7%

5 models · 1 drop · Nemotron 3.5 Lightning

Open weights only

Apex Bench v1.1

4 models compared.

DeepSeek V4.1 Flash 1017 72.1%

DeepSeek V4.1 Flash 64.6%

GLM-5.3 56.6%

Kimi K3 47.8%

4 models · 1 drop · DeepSeek V4.1 Flash 1017

Open weights only

BankerToolBench

4 models compared.

MiniMax M3.1 Flash 76.2

GLM-5.3 74.5

DeepSeek V4.1 Flash 73.7

Kimi K3 69.1

4 models · 1 drop · MiniMax M3.1 Flash

Open weights only

CyberGym-E2E

4 models compared.

Mistral Large 4 82%

MiMo-V2.6-Pro 79%

Kimi K3 58%

GLM-5.3 29%

4 models · 1 drop · Mistral Large 4

Open weights only

HLE-Diamond (no tools)

4 models compared.

Muse Spark Openxhigh 26.1%

Kimi K3max 22.2%

Qwen3.8-2.4T-A95Bxhigh 17.3%

GLM-5.3max 16.4%

4 models · 1 drop · Muse Spark Open

Open weights only

Arena AI text score

3 models compared.

Gemma 4 1452

Gemma 4 26B A4B 1441

Gemma 3 27B 1365

3 models · 1 drop · Gemma 4

Open weights only

Arena Hard

3 models compared.

DeepSeek R1 92%

Llama 3.1 Nemotron Ultra 87%

Llama 3.1 405B 66.2%

3 models · 1 drop · Llama 3.1 Nemotron Ultra

Open weights only

Claw-Eval pass^3

3 models compared.

Nanbeige4.3-33Bmax 78.3%

Qwen3.8max 75.2%

Gemma 4max 44.2%

3 models · 1 drop · Nanbeige4.3-33B model card

Open weights only

DeepPlanning

3 models compared.

Nanbeige4.3-33Bmax 63.3%

Qwen3.8max 49.2%

Gemma 4max 19.4%

3 models · 1 drop · Nanbeige4.3-33B model card

Open weights only

Finance Agent v2

3 models compared.

GLM-5.3 55.8%

Mistral Large 4 54.7%

Kimi K3 53.1%

3 models · 1 drop · Mistral Large 4

Open weights only

Frontier Finance

3 models compared.

Kimi K3.1 Promax 58.6%

GLM-5.3max 57.5%

DeepSeek V4.1 Flash 1017max 54.5%

3 models · 1 drop · Kimi K3.1 Pro

Open weights only

GDPval rubrics

3 models compared.

Qwen3.8max 86.4%

Nanbeige4.3-33Bmax 84.8%

Gemma 4max 71.4%

3 models · 1 drop · Nanbeige4.3-33B model card

Open weights only

HMMT Feb25

3 models compared.

Nemotron 3 Super 93.7%

Qwen3.5-122B-A10B 91.4%

gpt-oss-120b 90%

3 models · 1 drop · Nemotron 3 Super

Open weights only

OfficeQA Pro v2

3 models compared.

Kimi K3.1 Promax 64.4%

DeepSeek V4.1 Flash 1017max 40.4%

GLM-5.3max 31.1%

3 models · 1 drop · Kimi K3.1 Pro

Open weights only

SkillsBench

3 models compared.

Nanbeige4.3-33Bmax 61.1%

Qwen3.8max 56.4%

Gemma 4max 28.9%

3 models · 1 drop · Nanbeige4.3-33B model card

Open weights only

SWE-Bench (OpenHands)

3 models compared.

Nemotron 3 Nano 38.8%

gpt-oss-20b 34%

Qwen3-30B-A3B-Thinking-2507 22%

3 models · 1 drop · Nemotron 3 Nano

Open weights only

τ²-Bench

3 models compared.

MiniMax-M2 77.2%

GLM-4.6 75.9%

Kimi K2 0905 70.3%

3 models · 1 drop · MiniMax-M2

Open weights only

ArenaHard

2 models compared.

Mistral Medium 3 97.1%

Llama 4 Maverick 91.8%

2 models · 1 drop · Mistral Medium 3

Open weights only

BenchCAD (with tools)

2 models compared.

Kimi K3.1 Promax 92.7%

DeepSeek V4.1 Flash 1017max 81.6%

2 models · 1 drop · Kimi K3.1 Pro

Open weights only

BFCL V2 Live

2 models compared.

Llama 3.1 Nemotron Ultra 74.1%

Llama 3.1 405B 58.7%

2 models · 1 drop · Llama 3.1 Nemotron Ultra

Open weights only

ChartQA

2 models compared.

Llama 4 Maverick 90%

Llama 4 Scout 88.8%

2 models · 1 drop · Llama 4 Maverick & Scout

Open weights only

Dense200

2 models compared.

Mistral Large 4 42%

Kimi K3 28.9%

2 models · 1 drop · Mistral Large 4

Open weights only

DocVQA

2 models compared.

Mistral Medium 3 95.3%

Llama 4 Maverick 94.1%

2 models · 1 drop · Mistral Medium 3

Open weights only

Hermes Bench

2 models compared.

Nemotron 3.5 Super VL 63.4%

Nemotron 3 Ultra 56.3%

2 models · 1 drop · Nemotron 3.5 Super VL

Open weights only

HumanEval

2 models compared.

Mistral Medium 3 92.1%

Llama 4 Maverick 85.4%

2 models · 1 drop · Mistral Medium 3

Open weights only

MathVista

2 models compared.

Llama 4 Maverick 73.7%

Llama 4 Scout 70.7%

2 models · 1 drop · Llama 4 Maverick & Scout

Open weights only

RULER (128K)

2 models compared.

Nemotron Nano 2reasoning off 78.9%

Qwen3-8B 74.1%

2 models · 1 drop · Nemotron Nano 2

Open weights only

RULER-100@1M

2 models compared.

Nemotron 3 Nano 86.3%

Qwen3-30B-A3B-Thinking-2507 77.5%

2 models · 1 drop · Nemotron 3 Nano

Open weights only

SuperGPQA

2 models compared.

Qwen3-235B-A22B-Instruct-2507 62.6%

DeepSeek V3.1 59.8%

2 models · 1 drop · Qwen3-Max

Open weights only

τ²-Bench (Weighted)

2 models compared.

Qwen3-235B-A22B-Instruct-2507 52.9%

DeepSeek V3.1 46.4%

2 models · 1 drop · Qwen3-Max

Open weights only

CritPt v2 Hard

Unpublished research-level physics problems, hard subset, run by Artificial Analysis.

Kimi K3.1 Pro 49.5%

Kimi K3max 40.5%

Ling-3.1-flash-1019 38.5%

MiMo-V2.6-Pro 34%

GLM-5.3max 33.5%

MiniMax M3.1 Flash 32.5%

6 more

Muse Spark Openxhigh 29.5%

Mistral Large 4high 27%

DeepSeek V4.1 Flash 1017max 25%

Inklingxhigh 18%

Nemotron 3.5 Super VL 13.5%

Qwen3.8xhigh 4%

12 models · 0 drops · Artificial Analysis (2026-11-04)

Open weights only

Terminal-Bench Science (standard harness)

Terminal-Bench Science run by a third party on one shared harness.

Kimi K3.1 Pro 34.3%

DeepSeek V4.1 Flash 1017max 12.9%

GLM-5.3max 9.5%

Kimi K3max 8.1%

MiMo-V2.6-Pro 5.7%

Mistral Large 4high 4.3%

6 more

Ling-3.1-flash-1019 4.3%

MiniMax M3.1 Flash 3.3%

Muse Spark Openxhigh 1.9%

Qwen3.8xhigh 0.5%

Nemotron 3.5 Super VL 0%

Inklingxhigh 0%

12 models · 0 drops · Artificial Analysis (2026-11-04)

Release pace

Every Model Drop by month, marked with its maker.

1 Dec 2024

1 Jan 2025

0 Feb 2025

0 Mar 2025

3 Apr 2025

2 May 2025

0 Jun 2025

2 Jul 2025

3 Aug 2025

0 Sep 2025

1 Oct 2025

1 Nov 2025

2 Dec 2025

1 Jan 2026

2 Feb 2026

2 Mar 2026

3 Apr 2026

0 May 2026

3 Jun 2026

2 Jul 2026

3 Aug 2026

1 Sep 2026

11 Oct 2026

2 Nov 2026

How this board works

Scores come from the benchmark section of each handyai.news Model Drop, mostly the makers' launch tables. When drops disagree, the newest drop wins, since later launches re-run rivals on the current version of a benchmark; the model's own launch number is kept as launch_score. Settings (effort level, harness) vary between makers, so compare within a board with care. Models without their own drop are dated by the first drop that cites them.

The data is open: /api/benchmarks (JSON), /benchmarks.md (Markdown), one benchmark at /api/benchmarks/{benchmark}, the best model overall at /api/benchmarks/weigh-in (or /weigh-in/{type} for a topic), a PNG of any card by adding .png, and the list_benchmarks, get_benchmark and get_weigh_in tools on our MCP server.