GSM8K
AGI-Eval’s leaderboard hasn’t scored a model released after Kimi K2.5 (January 2026). Labs report their own numbers for 9 newer models, such as DiffusionGemma 26B A4B at 96.3%.
8.5K grade-school math word problems. Final numeric answer scored by exact match.
Saturated.
- Made by
- OpenAI
- Released
- October 2021
- Kind
- Word problems
- Level
- School
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- All public
- Measures
- Capability
- Subject
- Mathematics
- Language
- English
- Interaction
- Single turn
- Refresh
- New versions replace it
- Run by
- Anyone
- Strongest model tested
-
DeepSeek V4.1 Flash #34 on the capability index
- Last in a launch post
- September 2026, dropped after it saturated
- Last model added
- October 2026
- Scores refreshed
- November 20th, 2026
In the authors’ words
State-of-the-art language models can match human performance on many tasks, but they still struggle to robustly perform multi-step mathematical reasoning. To diagnose the failures of current models and support research, we introduce GSM8K, a dataset of 8.5K high quality linguistically diverse grade school math word problems. We find that even the largest transformer models fail to achieve high test performance, despite the conceptual simplicity of this problem distribution. To increase performance, we propose training verifiers to judge the correctness of model completions. At test time, we generate many candidate solutions and select the one ranked highest by the verifier. We demonstrate that verification significantly improves performance on GSM8K, and we provide strong empirical evidence that verification scales more effectively with increased data than a finetuning baseline.
Known issues
A published audit found a problem in GSM8K. We summarize each finding in our words and link its source. The scores on this page don't include any fixes. See all audits
- Contamination Still open
Scale AI reports that many models score lower on GSM1k, a new set written to mirror GSM8K, than on GSM8K itself. It reads the gap as evidence that some models partially memorized GSM8K, while frontier models show minimal signs of overfitting.
Accuracy drops by up to 8% on GSM1k. A model's chance of generating GSM8K examples tracks its GSM8K-to-GSM1k gap (Spearman's r² = 0.36).
Our analysis confirms the widespread suspicion in the field that many models are contaminated by benchmark data, with the worst models performing 8% worse on GSM1k compared to GSM8k.
A Careful Examination of Large Language Model Performance on Grade School Arithmetic by Scale AI, May 1st, 2024
- GSM1k in the almanac
Llama 3.1-405B leads with 96.8%
Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.
Claude Opus 4.5 leads with 98.1% across all independent runs
Best score per model across AGI-Eval’s leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.
Claude Opus 4.5 leads with 98.1%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
- 96.8
Llama 3.1-405B
- 96.6
ERNIE-4.5-300B-A47B
- 96.4
Cogito v1 Preview 70B
- 96.4 Claude 3.5 Sonnet
- 96.3
DiffusionGemma 26B A4B
- 96.2
Cogito v2 Preview 671B MoE
- 95.9
Qwen2.5-32B
- 95.9
Gemma 3 27B
- 95.8
Qwen2.5-72B
- 95.5
Tulu 3 405B
- 95.3
Qwen2.5-VL-72B
- 95.1
Llama 3.1-70B
- 95.0 Claude 3 Opus
- 94.8
Qwen2.5-14B
- 94.8
MiniMax-Text-01
- 94.8
Qwen2.5-14B-Instruct-1M
- 94.4
Gemini 1.0 Ultra
- 94.4
Gemma 3 12B
- 94.2 Granite 4.1 30B
- 94.0 Sarvam-M
- 93.8
Qwen-Turbo
- 93.3
Code World Model (CWM)
- 93.2
Qwen2-72B
- 93.0
Llama 3-70B
- 93.0
Mistral Large 2
- 93.0
DeepSeek V4.1 Flash
- 93.0
DeepSeek-V4.1-Flash Base
- 92.6
DeepSeek-V4-Pro
- 92.3
Nemotron 3 Nano 30B-A3B Base
- 92.3 Claude 3 Sonnet
- 92.2
DeepSeek-V2 (MoE-236B, May 2024)
- 92.0 GPT-4
- 91.7
Nemotron 3.5 Lightning 30B-A3B Base
- 91.7
Nemotron 3.5 Super 120B-A12B Base
- 91.7
Gemini 1.5 Pro
- 91.6
Qwen2.5-7B
- 91.6
InternVL3.5-241B-A28B
- 91.2
Mistral Large
- 91.0 PaLM 2-L
- 91.0
phi-3-medium 14B
- 90.8
Gemini 1.5 Pro
- 90.0 Grok-1.5
- 89.6
phi-3-small 7.4B
- 89.3
DeepSeek-V3
- 89.2
Gemma 3 4B
- 88.9 Claude 3 Haiku
- 88.7
Phi-3.5-MoE
- 88.7
Qwen2.5-Omni-7B
- 88.6
Phi-4 Mini
- 88.1
Nemotron 3 Ultra 550B-A55B Base
- 88.0 Claude 2
- 87.6 GLM-4
- 87.6
OLMo 2 32B Instruct
- 87.3 Granite 4.0 H Small
- 86.2
Phi-3.5-mini
- 85.6 LFM2.5-1.2B-Thinking
- 84.5
Llama 3.1-8B
- 84.4 LFM2-8B-A1B
- 84.1
DeepSeek LLM 67B
- 83.6
Falcon-H1-34B
- 82.5
phi-3-mini 3.8B
- 81.7 Granite 3.2 8B Instruct
- 80.9 Granite 3.3 8B Instruct
- 80.7
Mistral Small 3
- 79.6
Llama 3-8B
- 79.5
Qwen1.5-72B
- 79.4 GLM-4.5
- 77.6
Apertus 70B
- 77.4 Moonlight-16B-A3B
- 76.3 granite-4.0-1b
- 74.0
Gemma 2 27B
- 72.3 ChatGLM3-6B
- 68.8 GLM-5
- 68.6
Gemma 2 9B
- 68.3
OLMo 2 1B Instruct
- 67.7
Falcon-Arabic-7B
- 62.9 Grok-1
- 62.8
Gemma 3 1B
- 61.1
Phi-2
- 60.1
Qwen-14B
- 58.4
Mixtral 8x7B
- 58.4
BitNet b1.58 2B4T
- 58.3 LFM2-1.2B
- 56.8
Llama 2-70B
- 54.1
Qwen-7B
- 52.2
Mistral 7B v0.1
- 46.4
Gemma 7B
- 42.2
Llama 2-34B
- 28.7
Llama 2-13B
- 27.8 Qwen-1_8B
- 17.7
Gemma 2B
- 14.6
Llama 2-7B
- 1
Llama 3.1-405B 96.8%
- 2
ERNIE-4.5-300B-A47B 96.6%
- 3
Cogito v1 Preview 70B 96.4%
- 4 Claude 3.5 Sonnet 96.4%
- 5
DiffusionGemma 26B A4B 96.3%
- 6
Cogito v2 Preview 671B MoE 96.2%
- 7
Qwen2.5-32B 95.9%
- 8
Gemma 3 27B 95.9%
- 9
Qwen2.5-72B 95.8%
- 10
Tulu 3 405B 95.5%
- 11
Qwen2.5-VL-72B 95.3%
- 12
Llama 3.1-70B 95.1%
- 13 Claude 3 Opus 95%
- 14
Qwen2.5-14B 94.8%
- 15
MiniMax-Text-01 94.8%
- 16
Qwen2.5-14B-Instruct-1M 94.8%
- 17
Gemini 1.0 Ultra 94.4%
- 18
Gemma 3 12B 94.4%
- 19 Granite 4.1 30B 94.2%
- 20 Sarvam-M 94%
- 21
Qwen-Turbo 93.8%
- 22
Code World Model (CWM) 93.3%
- 23
Qwen2-72B 93.2%
- 24
Llama 3-70B 93%
- 25
Mistral Large 2 Jul 2024 93%
- 26
DeepSeek V4.1 Flash 93%
- 27
DeepSeek-V4.1-Flash Base 93%
- 28
DeepSeek-V4-Pro 92.6%
- 29
Nemotron 3 Nano 30B-A3B Base 92.3%
- 30 Claude 3 Sonnet 92.3%
- 31
DeepSeek-V2 (MoE-236B, May 2024) 92.2%
- 32 GPT-4 Mar 2023 92%
- 33
Nemotron 3.5 Lightning 30B-A3B Base 91.7%
- 34
Nemotron 3.5 Super 120B-A12B Base 91.7%
- 35
Gemini 1.5 Pro Feb 2024 91.7%
- 36
Qwen2.5-7B 91.6%
- 37
InternVL3.5-241B-A28B 91.6%
- 38
Mistral Large 91.2%
- 39 PaLM 2-L 91%
- 40
phi-3-medium 14B 91%
- 41
Gemini 1.5 Pro May 2024 90.8%
- 42 Grok-1.5 90%
- 43
phi-3-small 7.4B 89.6%
- 44
DeepSeek-V3 89.3%
- 45
Gemma 3 4B 89.2%
- 46 Claude 3 Haiku 88.9%
- 47
Phi-3.5-MoE 88.7%
- 48
Qwen2.5-Omni-7B 88.7%
- 49
Phi-4 Mini 88.6%
- 50
Nemotron 3 Ultra 550B-A55B Base 88.1%
- 51 Claude 2 88%
- 52 GLM-4 87.6%
- 53
OLMo 2 32B Instruct 87.6%
- 54 Granite 4.0 H Small 87.3%
- 55
Phi-3.5-mini 86.2%
- 56 LFM2.5-1.2B-Thinking 85.6%
- 57
Llama 3.1-8B 84.5%
- 58 LFM2-8B-A1B 84.4%
- 59
DeepSeek LLM 67B 84.1%
- 60
Falcon-H1-34B 83.6%
- 61
phi-3-mini 3.8B 82.5%
- 62 Granite 3.2 8B Instruct 81.7%
- 63 Granite 3.3 8B Instruct 80.9%
- 64
Mistral Small 3 80.7%
- 65
Llama 3-8B 79.6%
- 66
Qwen1.5-72B 79.5%
- 67 GLM-4.5 79.4%
- 68
Apertus 70B 77.6%
- 69 Moonlight-16B-A3B 77.4%
- 70 granite-4.0-1b 76.3%
- 71
Gemma 2 27B 74%
- 72 ChatGLM3-6B 72.3%
- 73 GLM-5 68.8%
- 74
Gemma 2 9B 68.6%
- 75
OLMo 2 1B Instruct 68.3%
- 76
Falcon-Arabic-7B 67.7%
- 77 Grok-1 62.9%
- 78
Gemma 3 1B 62.8%
- 79
Phi-2 61.1%
- 80
Qwen-14B 60.1%
- 81
Mixtral 8x7B 58.4%
- 82
BitNet b1.58 2B4T 58.4%
- 83 LFM2-1.2B 58.3%
- 84
Llama 2-70B 56.8%
- 85
Qwen-7B 54.1%
- 86
Mistral 7B v0.1 52.2%
- 87
Gemma 7B 46.4%
- 88
Llama 2-34B 42.2%
- 89
Llama 2-13B 28.7%
- 90 Qwen-1_8B 27.8%
- 91
Gemma 2B 17.7%
- 92
Llama 2-7B 14.6%
Source: Labs’ model cards
- 98.1 Claude Opus 4.5
- 97.7 Claude Sonnet 4
- 97.7
Gemini 3 Flash
- 97.5
Gemini 3 Pro
- 97.5 Kimi K2.5
- 97.4 Claude Sonnet 4.5
- 97.4 GPT-5.1
- 97.3
Doubao-Seed-1.6
- 97.3
Qwen3-Next-80B-A3B
- 97.3 Kimi K2 Thinking
- 97.2
QwQ-32B
- 97.2 o3
- 97.2 o4-mini
- 97.2 GPT-5
- 97.2 GPT-5.2
- 97.1 Claude 3.7 Sonnet
- 97.1
DeepSeek-V3
- 97.1
Step 3
- 97.1
DeepSeek-V3.2-Speciale
- 97.0 o3-mini
- 97.0
Qwen3-Max-Preview
- 97.0 GLM-4.6
- 97.0
Ring-1T
- 97.0 Kimi K2
- 97.0
Qwen3-235B-A22B-Thinking
- 97.0
Qwen3-Next-80B-A3B-Instruct
- 96.9 GPT-4.5
- 96.9 Claude Opus 4
- 96.9
Gemini 2.5 Pro
- 96.9
DeepSeek-V3.1
- 96.8
DeepSeek-V3
- 96.8
ERNIE-4.5-300B-A47B
- 96.8 gpt-oss-120b
- 96.7 Claude 3.5 Sonnet (October 2024)
- 96.7
DeepSeek-V3.2-Exp
- 96.5
Llama 4 Maverick
- 96.5
DeepSeek-R1
- 96.4
Llama 3.1-405B
- 96.4
Hunyuan-T1
- 96.4
Qwen3-235B-A22B-Instruct
- 96.4 GLM-4.5
- 96.4
DeepSeek-V3.1-Terminus
- 96.4 GPT-5.1 Instant
- 96.4
DeepSeek-V3.2
- 96.3
Gemini 2.5 Flash
- 96.1
Llama 4 Scout
- 96.1 GPT-4.1
- 96.1
Qwen3-235B-A22B
- 96.1 GPT-4o
- 96.1
Gemma 3 27B
- 95.9
Qwen2.5-72B
- 95.8
MiniMax-M2
- 95.8
Qwen3-32B
- 95.7
Mistral Large 2
- 95.6 GPT-4o
- 95.6 Grok 4.1
- 95.5 Grok 3
- 95.5 gpt-oss-20b
- 95.4
MiniMax-M1-80k
- 95.3
Qwen3-14B
- 95.3
Qwen3-30B-A3B-Thinking
- 95.2
Llama 3.1-70B
- 95.2
Qwen2.5-32B
- 95.0
Qwen2.5-14B
- 94.8
Qwen3-30B-A3B
- 94.5 Grok 4
- 94.5
Gemma 3 12B
- 93.6 GPT-4o mini
- 93.0
Qwen3-8B
- 91.1
Qwen2.5-7B
- 89.6
Gemma 3 4B
- 87.3
Gemma 2 9B
- 85.3
Llama 3.1-8B
- 85.1
Qwen2.5-3B
- 82.9 GLM-4-9B
- 72.1
Qwen2.5-1.5B
- 58.9
Gemma 3 1B
- 44.3
Qwen2.5-0.5B
- 1 Claude Opus 4.5 98.1%
- 2 Claude Sonnet 4 97.7%
- 3
Gemini 3 Flash 97.7%
- 4
Gemini 3 Pro 97.5%
- 5 Kimi K2.5 Chat 97.5%
- 6 Claude Sonnet 4.5 Thinking 97.4%
- 7 GPT-5.1 97.4%
- 8
Doubao-Seed-1.6 Auto 97.3%
- 9
Qwen3-Next-80B-A3B Thinking 97.3%
- 10 Kimi K2 Thinking Thinking 97.3%
- 11
QwQ-32B 97.2%
- 12 o3 97.2%
- 13 o4-mini 97.2%
- 14 GPT-5 Thinking (High) 97.2%
- 15 GPT-5.2 97.2%
- 16 Claude 3.7 Sonnet 97.1%
- 17
DeepSeek-V3 Mar 2025 97.1%
- 18
Step 3 97.1%
- 19
DeepSeek-V3.2-Speciale Thinking 97.1%
- 20 o3-mini high 97%
- 21
Qwen3-Max-Preview 97%
- 22 GLM-4.6 97%
- 23
Ring-1T 97%
- 24 Kimi K2 Jul 2025 97%
- 25
Qwen3-235B-A22B-Thinking Jul 2025, Thinking 97%
- 26
Qwen3-Next-80B-A3B-Instruct 97%
- 27 GPT-4.5 96.9%
- 28 Claude Opus 4 96.9%
- 29
Gemini 2.5 Pro Jun 2025 96.9%
- 30
DeepSeek-V3.1 Thinking 96.9%
- 31
DeepSeek-V3 96.8%
- 32
ERNIE-4.5-300B-A47B 96.8%
- 33 gpt-oss-120b high 96.8%
- 34 Claude 3.5 Sonnet (October 2024) 96.7%
- 35
DeepSeek-V3.2-Exp Thinking 96.7%
- 36
Llama 4 Maverick 96.5%
- 37
DeepSeek-R1 May 2025 96.5%
- 38
Llama 3.1-405B 96.4%
- 39
Hunyuan-T1 96.4%
- 40
Qwen3-235B-A22B-Instruct Jul 2025 96.4%
- 41 GLM-4.5 96.4%
- 42
DeepSeek-V3.1-Terminus Thinking 96.4%
- 43 GPT-5.1 Instant 96.4%
- 44
DeepSeek-V3.2 Thinking 96.4%
- 45
Gemini 2.5 Flash Sep 2025 96.3%
- 46
Llama 4 Scout 96.1%
- 47 GPT-4.1 96.1%
- 48
Qwen3-235B-A22B 96.1%
- 49 GPT-4o Nov 2024 96.1%
- 50
Gemma 3 27B 96.1%
- 51
Qwen2.5-72B 95.9%
- 52
MiniMax-M2 95.8%
- 53
Qwen3-32B 95.8%
- 54
Mistral Large 2 Nov 2024 95.7%
- 55 GPT-4o May 2024 95.6%
- 56 Grok 4.1 Thinking 95.6%
- 57 Grok 3 95.5%
- 58 gpt-oss-20b high 95.5%
- 59
MiniMax-M1-80k 95.4%
- 60
Qwen3-14B Thinking 95.3%
- 61
Qwen3-30B-A3B-Thinking Jul 2025, Thinking 95.3%
- 62
Llama 3.1-70B 95.2%
- 63
Qwen2.5-32B 95.2%
- 64
Qwen2.5-14B 95%
- 65
Qwen3-30B-A3B 94.8%
- 66 Grok 4 94.5%
- 67
Gemma 3 12B 94.5%
- 68 GPT-4o mini 93.6%
- 69
Qwen3-8B 93%
- 70
Qwen2.5-7B 91.1%
- 71
Gemma 3 4B 89.6%
- 72
Gemma 2 9B 87.3%
- 73
Llama 3.1-8B 85.3%
- 74
Qwen2.5-3B 85.1%
- 75 GLM-4-9B 82.9%
- 76
Qwen2.5-1.5B 72.1%
- 77
Gemma 3 1B 58.9%
- 78
Qwen2.5-0.5B 44.3%
Source: AGI-Eval’s leaderboard
- 98.1 Claude Opus 4.5
- 97.7 Claude Sonnet 4
- 97.7
Gemini 3 Flash
- 97.5
Gemini 3 Pro
- 97.5 Kimi K2.5
- 97.4 Claude Sonnet 4.5
- 97.4 GPT-5.1
- 97.3
Doubao-Seed-1.6
- 97.3
Qwen3-Next-80B-A3B
- 97.3 Kimi K2 Thinking
- 97.2
QwQ-32B
- 97.2 o3
- 97.2 o4-mini
- 97.2 GPT-5
- 97.2 GPT-5.2
- 97.1 Claude 3.7 Sonnet
- 97.1
DeepSeek-V3
- 97.1
Step 3
- 97.1
DeepSeek-V3.2-Speciale
- 97.0 o3-mini
- 97.0
Qwen3-Max-Preview
- 97.0 GLM-4.6
- 97.0
Ring-1T
- 97.0 Kimi K2
- 97.0
Qwen3-235B-A22B-Thinking
- 97.0
Qwen3-Next-80B-A3B-Instruct
- 96.9 GPT-4.5
- 96.9 Claude Opus 4
- 96.9
Gemini 2.5 Pro
- 96.9
DeepSeek-V3.1
- 96.8
DeepSeek-V3
- 96.8
ERNIE-4.5-300B-A47B
- 96.8 gpt-oss-120b
- 96.8
Llama 3.1-405B
- 96.7 Claude 3.5 Sonnet (October 2024)
- 96.7
DeepSeek-V3.2-Exp
- 96.5
Llama 4 Maverick
- 96.5
DeepSeek-R1
- 96.4
Hunyuan-T1
- 96.4
Cogito v1 Preview 70B
- 96.4
Qwen3-235B-A22B-Instruct
- 96.4 GLM-4.5
- 96.4 Claude 3.5 Sonnet
- 96.4
DeepSeek-V3.1-Terminus
- 96.4 GPT-5.1 Instant
- 96.4
DeepSeek-V3.2
- 96.3
DiffusionGemma 26B A4B
- 96.3
Gemini 2.5 Flash
- 96.2
Cogito v2 Preview 671B MoE
- 96.1
Llama 4 Scout
- 96.1 GPT-4.1
- 96.1
Qwen3-235B-A22B
- 96.1 GPT-4o
- 96.1
Gemma 3 27B
- 95.9
Qwen2.5-72B
- 95.9
Qwen2.5-32B
- 95.8
MiniMax-M2
- 95.8
Qwen3-32B
- 95.7
Mistral Large 2
- 95.6 GPT-4o
- 95.6 Grok 4.1
- 95.5 Grok 3
- 95.5 gpt-oss-20b
- 95.5
Tulu 3 405B
- 95.4
MiniMax-M1-80k
- 95.3
Qwen2.5-VL-72B
- 95.3
Qwen3-14B
- 95.3
Qwen3-30B-A3B-Thinking
- 95.2
Llama 3.1-70B
- 95.0 Claude 3 Opus
- 95.0
Qwen2.5-14B
- 94.8
MiniMax-Text-01
- 94.8
Qwen2.5-14B-Instruct-1M
- 94.8
Qwen3-30B-A3B
- 94.5 Grok 4
- 94.5
DeepSeek-Coder-V2 236B
- 94.5
Gemma 3 12B
- 94.4
Gemini 1.0 Ultra
- 94.2 Qwen2.5-Coder-14B-Instruct
- 94.2 Granite 4.1 30B
- 94.0 Sarvam-M
- 93.8
Qwen-Turbo
- 93.6 GPT-4o mini
- 93.3
Code World Model (CWM)
- 93.2
Qwen2-72B
- 93.0
Qwen3-8B
- 93.0
Llama 3-70B
- 93.0
Mistral Large 2
- 93.0
Qwen2.5-Coder-32B-Instruct
- 93.0
DeepSeek V4.1 Flash
- 93.0
DeepSeek-V4.1-Flash Base
- 92.6
DeepSeek-V4-Pro
- 92.3
Nemotron 3 Nano 30B-A3B Base
- 92.3 Claude 3 Sonnet
- 92.2
DeepSeek-V2 (MoE-236B, May 2024)
- 92.0 GPT-4
- 91.7
Nemotron 3.5 Lightning 30B-A3B Base
- 91.7
Nemotron 3.5 Super 120B-A12B Base
- 91.7
Gemini 1.5 Pro
- 91.6
Qwen2.5-7B
- 91.6
InternVL3.5-241B-A28B
- 91.2
Mistral Large
- 91.1
Qwen2.5-Coder-32B
- 91.0 PaLM 2-L
- 91.0
phi-3-medium 14B
- 90.8
Gemini 1.5 Pro
- 90.0 Grok-1.5
- 90.0 GPT-4
- 89.6
Gemma 3 4B
- 89.6
phi-3-small 7.4B
- 88.9 Claude 3 Haiku
- 88.7
Phi-3.5-MoE
- 88.7 Qwen2.5-Coder-14B
- 88.7
Qwen2.5-Omni-7B
- 88.6
Phi-4 Mini
- 88.1
Nemotron 3 Ultra 550B-A55B Base
- 88.0 Claude 2
- 87.6 GLM-4
- 87.6 DeepSeek-Coder-V2-Lite-Instruct
- 87.6
OLMo 2 32B Instruct
- 87.3
Gemma 2 9B
- 87.3 Granite 4.0 H Small
- 86.7 Claude Instant
- 86.7
Qwen2.5-Coder (7B)
- 86.2
Phi-3.5-mini
- 85.6 LFM2.5-1.2B-Thinking
- 85.3
Llama 3.1-8B
- 85.1
Qwen2.5-3B
- 84.4 LFM2-8B-A1B
- 84.2
Mistral NeMo
- 84.1
DeepSeek LLM 67B
- 83.6
Falcon-H1-34B
- 82.9 GLM-4-9B
- 82.5
phi-3-mini 3.8B
- 82.4
Gemini 1.5 Flash
- 81.7 Granite 3.2 8B Instruct
- 80.9 Granite 3.3 8B Instruct
- 80.7
Mistral Small 3
- 80.7 Qwen2.5-Coder-3B-Instruct
- 79.6
Llama 3-8B
- 79.5
Qwen1.5-72B
- 78.2 davinci-003
- 77.6
Apertus 70B
- 77.4 Moonlight-16B-A3B
- 76.3 granite-4.0-1b
- 76.0
Yi-34B
- 75.7 Qwen2.5-Coder-3B
- 74.4
Mixtral 8x7B
- 74.0
Gemma 2 27B
- 72.3 ChatGLM3-6B
- 72.1
Qwen2.5-1.5B
- 69.6
Llama 2-70B
- 69.6
Stable Beluga 2
- 68.8 GLM-5
- 68.3
OLMo 2 1B Instruct
- 67.7
Falcon-Arabic-7B
- 67.1 DeepSeek-Coder-V2-Lite-Base
- 65.8
Qwen2.5-Coder (1.5B)
- 62.9 internlm-20b
- 62.9 Grok-1
- 62.8
Gemma 3 1B
- 61.3
Qwen-14B
- 61.1
Phi-2
- 58.4
BitNet b1.58 2B4T
- 58.3 LFM2-1.2B
- 57.8 GPT-3.5 Turbo
- 57.7
StarCoder 2 15B
- 56.5
PaLM (540B)
- 54.4
LLaMA-65B
- 54.4
Falcon-180B
- 54.4
Mistral 7B v0.1
- 54.1
Qwen-7B
- 53.8
Falcon 2 11B
- 52.8
Baichuan2-13B
- 46.4
Gemma 7B
- 46.0
Nemotron-4 15B
- 45.7 Baichuan2-13B-Chat
- 44.9
Yi 6B
- 44.3
Qwen2.5-0.5B
- 44.1
LLaMA-33B
- 42.2
Llama 2-34B
- 38.6 INTELLECT-1
- 37.7 CodeQwen1.5-7B
- 36.9
Llama 2-13B
- 35.4
DeepSeek Coder 33B
- 35.4
Mistral 7B v0.2
- 34.5 Qwen2.5-Coder-0.5B
- 33.0 PaLM 62B
- 32.7
StarCoder 2 7B
- 32.4 chatglm2-6b
- 31.2 internlm-7b
- 28.1 vicuna-13b-v1.1
- 27.8 Qwen-1_8B
- 26.8
Baichuan 1-13B
- 24.6
Baichuan 2-7B
- 21.6
StarCoder 2 3B
- 21.5
Falcon-40B
- 21.3
DeepSeek Coder 6.7B
- 20.6
LLaMA-13B
- 17.7
Gemma 2B
- 16.7
Llama 2-7B
- 15.7 internlm-chat-20b
- 15.2
MPT-30B
- 11.0
LLaMA-7B
- 9.20
Baichuan1-7B
- 9.10
MPT-7B
- 6.80
Falcon-7B
- 4.40
DeepSeek Coder 1.3B
- 1 Claude Opus 4.5 98.1%
- 2 Claude Sonnet 4 97.7%
- 3
Gemini 3 Flash 97.7%
- 4
Gemini 3 Pro 97.5%
- 5 Kimi K2.5 Chat 97.5%
- 6 Claude Sonnet 4.5 Thinking 97.4%
- 7 GPT-5.1 97.4%
- 8
Doubao-Seed-1.6 Auto 97.3%
- 9
Qwen3-Next-80B-A3B Thinking 97.3%
- 10 Kimi K2 Thinking Thinking 97.3%
- 11
QwQ-32B 97.2%
- 12 o3 97.2%
- 13 o4-mini 97.2%
- 14 GPT-5 Thinking (High) 97.2%
- 15 GPT-5.2 97.2%
- 16 Claude 3.7 Sonnet 97.1%
- 17
DeepSeek-V3 Mar 2025 97.1%
- 18
Step 3 97.1%
- 19
DeepSeek-V3.2-Speciale Thinking 97.1%
- 20 o3-mini high 97%
- 21
Qwen3-Max-Preview 97%
- 22 GLM-4.6 97%
- 23
Ring-1T 97%
- 24 Kimi K2 Jul 2025 97%
- 25
Qwen3-235B-A22B-Thinking Jul 2025, Thinking 97%
- 26
Qwen3-Next-80B-A3B-Instruct 97%
- 27 GPT-4.5 96.9%
- 28 Claude Opus 4 96.9%
- 29
Gemini 2.5 Pro Jun 2025 96.9%
- 30
DeepSeek-V3.1 Thinking 96.9%
- 31
DeepSeek-V3 96.8%
- 32
ERNIE-4.5-300B-A47B 96.8%
- 33 gpt-oss-120b high 96.8%
- 34
Llama 3.1-405B 96.8%
- 35 Claude 3.5 Sonnet (October 2024) 96.7%
- 36
DeepSeek-V3.2-Exp Thinking 96.7%
- 37
Llama 4 Maverick 96.5%
- 38
DeepSeek-R1 May 2025 96.5%
- 39
Hunyuan-T1 96.4%
- 40
Cogito v1 Preview 70B 96.4%
- 41
Qwen3-235B-A22B-Instruct Jul 2025 96.4%
- 42 GLM-4.5 96.4%
- 43 Claude 3.5 Sonnet 96.4%
- 44
DeepSeek-V3.1-Terminus Thinking 96.4%
- 45 GPT-5.1 Instant 96.4%
- 46
DeepSeek-V3.2 Thinking 96.4%
- 47
DiffusionGemma 26B A4B 96.3%
- 48
Gemini 2.5 Flash Sep 2025 96.3%
- 49
Cogito v2 Preview 671B MoE 96.2%
- 50
Llama 4 Scout 96.1%
- 51 GPT-4.1 96.1%
- 52
Qwen3-235B-A22B 96.1%
- 53 GPT-4o Nov 2024 96.1%
- 54
Gemma 3 27B 96.1%
- 55
Qwen2.5-72B 95.9%
- 56
Qwen2.5-32B 95.9%
- 57
MiniMax-M2 95.8%
- 58
Qwen3-32B 95.8%
- 59
Mistral Large 2 Nov 2024 95.7%
- 60 GPT-4o May 2024 95.6%
- 61 Grok 4.1 Thinking 95.6%
- 62 Grok 3 95.5%
- 63 gpt-oss-20b high 95.5%
- 64
Tulu 3 405B 95.5%
- 65
MiniMax-M1-80k 95.4%
- 66
Qwen2.5-VL-72B 95.3%
- 67
Qwen3-14B Thinking 95.3%
- 68
Qwen3-30B-A3B-Thinking Jul 2025, Thinking 95.3%
- 69
Llama 3.1-70B 95.2%
- 70 Claude 3 Opus 95%
- 71
Qwen2.5-14B 95%
- 72
MiniMax-Text-01 94.8%
- 73
Qwen2.5-14B-Instruct-1M 94.8%
- 74
Qwen3-30B-A3B 94.8%
- 75 Grok 4 94.5%
- 76
DeepSeek-Coder-V2 236B 94.5%
- 77
Gemma 3 12B 94.5%
- 78
Gemini 1.0 Ultra 94.4%
- 79 Qwen2.5-Coder-14B-Instruct 94.2%
- 80 Granite 4.1 30B 94.2%
- 81 Sarvam-M 94%
- 82
Qwen-Turbo 93.8%
- 83 GPT-4o mini 93.6%
- 84
Code World Model (CWM) 93.3%
- 85
Qwen2-72B 93.2%
- 86
Qwen3-8B 93%
- 87
Llama 3-70B 93%
- 88
Mistral Large 2 Jul 2024 93%
- 89
Qwen2.5-Coder-32B-Instruct 93%
- 90
DeepSeek V4.1 Flash 93%
- 91
DeepSeek-V4.1-Flash Base 93%
- 92
DeepSeek-V4-Pro 92.6%
- 93
Nemotron 3 Nano 30B-A3B Base 92.3%
- 94 Claude 3 Sonnet 92.3%
- 95
DeepSeek-V2 (MoE-236B, May 2024) 92.2%
- 96 GPT-4 Mar 2023 92%
- 97
Nemotron 3.5 Lightning 30B-A3B Base 91.7%
- 98
Nemotron 3.5 Super 120B-A12B Base 91.7%
- 99
Gemini 1.5 Pro Feb 2024 91.7%
- 100
Qwen2.5-7B 91.6%
- 101
InternVL3.5-241B-A28B 91.6%
- 102
Mistral Large 91.2%
- 103
Qwen2.5-Coder-32B 91.1%
- 104 PaLM 2-L 91%
- 105
phi-3-medium 14B 91%
- 106
Gemini 1.5 Pro May 2024 90.8%
- 107 Grok-1.5 90%
- 108 GPT-4 Jun 2023 90%
- 109
Gemma 3 4B 89.6%
- 110
phi-3-small 7.4B 89.6%
- 111 Claude 3 Haiku 88.9%
- 112
Phi-3.5-MoE 88.7%
- 113 Qwen2.5-Coder-14B 88.7%
- 114
Qwen2.5-Omni-7B 88.7%
- 115
Phi-4 Mini 88.6%
- 116
Nemotron 3 Ultra 550B-A55B Base 88.1%
- 117 Claude 2 88%
- 118 GLM-4 87.6%
- 119 DeepSeek-Coder-V2-Lite-Instruct 87.6%
- 120
OLMo 2 32B Instruct 87.6%
- 121
Gemma 2 9B 87.3%
- 122 Granite 4.0 H Small 87.3%
- 123 Claude Instant 86.7%
- 124
Qwen2.5-Coder (7B) 86.7%
- 125
Phi-3.5-mini 86.2%
- 126 LFM2.5-1.2B-Thinking 85.6%
- 127
Llama 3.1-8B 85.3%
- 128
Qwen2.5-3B 85.1%
- 129 LFM2-8B-A1B 84.4%
- 130
Mistral NeMo 84.2%
- 131
DeepSeek LLM 67B 84.1%
- 132
Falcon-H1-34B 83.6%
- 133 GLM-4-9B 82.9%
- 134
phi-3-mini 3.8B 82.5%
- 135
Gemini 1.5 Flash May 2024 82.4%
- 136 Granite 3.2 8B Instruct 81.7%
- 137 Granite 3.3 8B Instruct 80.9%
- 138
Mistral Small 3 80.7%
- 139 Qwen2.5-Coder-3B-Instruct 80.7%
- 140
Llama 3-8B 79.6%
- 141
Qwen1.5-72B 79.5%
- 142 davinci-003 78.2%
- 143
Apertus 70B 77.6%
- 144 Moonlight-16B-A3B 77.4%
- 145 granite-4.0-1b 76.3%
- 146
Yi-34B 76%
- 147 Qwen2.5-Coder-3B 75.7%
- 148
Mixtral 8x7B 74.4%
- 149
Gemma 2 27B 74%
- 150 ChatGLM3-6B 72.3%
- 151
Qwen2.5-1.5B 72.1%
- 152
Llama 2-70B 69.6%
- 153
Stable Beluga 2 69.6%
- 154 GLM-5 68.8%
- 155
OLMo 2 1B Instruct 68.3%
- 156
Falcon-Arabic-7B 67.7%
- 157 DeepSeek-Coder-V2-Lite-Base 67.1%
- 158
Qwen2.5-Coder (1.5B) 65.8%
- 159 internlm-20b 62.9%
- 160 Grok-1 62.9%
- 161
Gemma 3 1B 62.8%
- 162
Qwen-14B 61.3%
- 163
Phi-2 61.1%
- 164
BitNet b1.58 2B4T 58.4%
- 165 LFM2-1.2B 58.3%
- 166 GPT-3.5 Turbo Jun 2023 57.8%
- 167
StarCoder 2 15B 57.7%
- 168
PaLM (540B) 56.5%
- 169
LLaMA-65B 54.4%
- 170
Falcon-180B 54.4%
- 171
Mistral 7B v0.1 54.4%
- 172
Qwen-7B 54.1%
- 173
Falcon 2 11B 53.8%
- 174
Baichuan2-13B 52.8%
- 175
Gemma 7B 46.4%
- 176
Nemotron-4 15B 46%
- 177 Baichuan2-13B-Chat 45.7%
- 178
Yi 6B 44.9%
- 179
Qwen2.5-0.5B 44.3%
- 180
LLaMA-33B 44.1%
- 181
Llama 2-34B 42.2%
- 182 INTELLECT-1 38.6%
- 183 CodeQwen1.5-7B 37.7%
- 184
Llama 2-13B 36.9%
- 185
DeepSeek Coder 33B 35.4%
- 186
Mistral 7B v0.2 35.4%
- 187 Qwen2.5-Coder-0.5B 34.5%
- 188 PaLM 62B 33%
- 189
StarCoder 2 7B 32.7%
- 190 chatglm2-6b 32.4%
- 191 internlm-7b 31.2%
- 192 vicuna-13b-v1.1 28.1%
- 193 Qwen-1_8B 27.8%
- 194
Baichuan 1-13B 26.8%
- 195
Baichuan 2-7B 24.6%
- 196
StarCoder 2 3B 21.6%
- 197
Falcon-40B 21.5%
- 198
DeepSeek Coder 6.7B 21.3%
- 199
LLaMA-13B 20.6%
- 200
Gemma 2B 17.7%
- 201
Llama 2-7B 16.7%
- 202 internlm-chat-20b 15.7%
- 203
MPT-30B 15.2%
- 204
LLaMA-7B 11%
- 205
Baichuan1-7B 9.20%
- 206
MPT-7B 9.10%
- 207
Falcon-7B 6.80%
- 208
DeepSeek Coder 1.3B 4.40%
Sources: AGI-Eval’s leaderboard, labs’ model cards and another board, via Epoch AI (CC BY 4.0)
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 96.8% | Jul 2024 | Lab model card | |
| 2 | | 96.6% | Jun 2025 | Lab model card | |
| 3 | | 96.4% | Apr 2025 | Lab model card | |
| 4 | Claude 3.5 Sonnet | 96.4% | Jun 2024 | Lab model card | |
| 5 | | 96.3% | Jul 2026 | Lab model card | |
| 6 | | 96.2% | Jul 2025 | Lab model card | |
| 7 | | 95.9% | Dec 2024 | Lab model card | |
| 8 | | 95.9% | Mar 2025 | Lab model card | |
| 9 | | 95.8% | Sep 2024 | Lab model card | |
| 10 | | 95.5% | Jan 2025 | Lab model card | |
| 11 | | 95.3% | Feb 2025 | Lab model card | |
| 12 | | 95.1% | Jul 2024 | Lab model card | |
| 13 | Claude 3 Opus | 95% | Mar 2024 | Lab model card | |
| 14 | | 94.8% | Dec 2024 | Lab model card | |
| 15 | | 94.8% | Jan 2025 | Lab model card | |
| 16 | | 94.8% | Jan 2025 | Lab model card | |
| 17 | | 94.4% | Dec 2023 | Lab model card | |
| 18 | | 94.4% | Mar 2025 | Lab model card | |
| 19 | Granite 4.1 30B | 94.2% | Apr 2026 | Lab model card | |
| 20 | Sarvam-M | 94% | May 2025 | Lab model card | |
| 21 | | 93.8% | Dec 2024 | Lab model card | |
| 22 | | 93.3% | Sep 2025 | Lab model card | |
| 23 | | 93.2% | Jul 2024 | Lab model card | |
| 24 | | 93% | Apr 2024 | Lab model card | |
| 25 | | 93% | Jul 2024 | Lab model card | |
| 26 | | 93% | Sep 2026 | Lab model card | |
| 27 | | 93% | Sep 2026 | Lab model card | |
| 28 | | 92.6% | Apr 2026 | Lab model card | |
| 29 | | 92.3% | Dec 2025 | Lab model card | |
| 30 | Claude 3 Sonnet | 92.3% | Mar 2024 | Lab model card | |
| 31 | | 92.2% | May 2024 | Lab model card | |
| 32 | GPT-4 Mar 2023 | 92% | Mar 2023 | Lab model card | |
| 33 | | 91.7% | Oct 2026 | Lab model card | |
| 34 | | 91.7% | Oct 2026 | Lab model card | |
| 35 | | 91.7% | Feb 2024 | Lab model card | |
| 36 | | 91.6% | Dec 2024 | Lab model card | |
| 37 | | 91.6% | Aug 2025 | Lab model card | |
| 38 | | 91.2% | Feb 2024 | Lab model card | |
| 39 | PaLM 2-L | 91% | May 2023 | Lab model card | |
| 40 | | 91% | Aug 2024 | Lab model card | |
| 41 | | 90.8% | May 2024 | Lab model card | |
| 42 | Grok-1.5 | 90% | Mar 2024 | Lab model card | |
| 43 | | 89.6% | Aug 2024 | Lab model card | |
| 44 | | 89.3% | Dec 2024 | Lab model card | |
| 45 | | 89.2% | Mar 2025 | Lab model card | |
| 46 | Claude 3 Haiku | 88.9% | Mar 2024 | Lab model card | |
| 47 | | 88.7% | Aug 2024 | Lab model card | |
| 48 | | 88.7% | Mar 2025 | Lab model card | |
| 49 | | 88.6% | Feb 2025 | Lab model card | |
| 50 | | 88.1% | Jun 2026 | Lab model card | |
| 51 | Claude 2 | 88% | Jul 2023 | Lab model card | |
| 52 | GLM-4 | 87.6% | Jan 2024 | Lab model card | |
| 53 | | 87.6% | Mar 2025 | Lab model card | |
| 54 | Granite 4.0 H Small | 87.3% | Oct 2025 | Lab model card | |
| 55 | | 86.2% | Aug 2024 | Lab model card | |
| 56 | LFM2.5-1.2B-Thinking | 85.6% | Jan 2026 | Lab model card | |
| 57 | | 84.5% | Jul 2024 | Lab model card | |
| 58 | LFM2-8B-A1B | 84.4% | Oct 2025 | Lab model card | |
| 59 | | 84.1% | Nov 2023 | Lab model card | |
| 60 | | 83.6% | May 2025 | Lab model card | |
| 61 | | 82.5% | Aug 2024 | Lab model card | |
| 62 | Granite 3.2 8B Instruct | 81.7% | Feb 2025 | Lab model card | |
| 63 | Granite 3.3 8B Instruct | 80.9% | Apr 2025 | Lab model card | |
| 64 | | 80.7% | Jan 2025 | Lab model card | |
| 65 | | 79.6% | Apr 2024 | Lab model card | |
| 66 | | 79.5% | Feb 2024 | Lab model card | |
| 67 | GLM-4.5 | 79.4% | Aug 2025 | Lab model card | |
| 68 | | 77.6% | Sep 2025 | Lab model card | |
| 69 | Moonlight-16B-A3B | 77.4% | Feb 2025 | Lab model card | |
| 70 | granite-4.0-1b | 76.3% | Oct 2025 | Lab model card | |
| 71 | | 74% | Jun 2024 | Lab model card | |
| 72 | ChatGLM3-6B | 72.3% | Oct 2023 | Lab model card | |
| 73 | GLM-5 | 68.8% | Feb 2026 | Lab model card | |
| 74 | | 68.6% | Jul 2024 | Lab model card | |
| 75 | | 68.3% | Apr 2025 | Lab model card | |
| 76 | | 67.7% | Nov 2025 | Lab model card | |
| 77 | Grok-1 | 62.9% | Nov 2023 | Lab model card | |
| 78 | | 62.8% | Mar 2025 | Lab model card | |
| 79 | | 61.1% | Dec 2023 | Lab model card | |
| 80 | | 60.1% | Sep 2023 | Lab model card | |
| 81 | | 58.4% | Dec 2023 | Lab model card | |
| 82 | | 58.4% | Apr 2025 | Lab model card | |
| 83 | LFM2-1.2B | 58.3% | Jul 2025 | Lab model card | |
| 84 | | 56.8% | Jul 2023 | Lab model card | |
| 85 | | 54.1% | Sep 2023 | Lab model card | |
| 86 | | 52.2% | Oct 2023 | Lab model card | |
| 87 | | 46.4% | Feb 2024 | Lab model card | |
| 88 | | 42.2% | Jul 2023 | Lab model card | |
| 89 | | 28.7% | Jul 2023 | Lab model card | |
| 90 | Qwen-1_8B | 27.8% | Sep 2023 | Lab model card | |
| 91 | | 17.7% | Mar 2024 | Lab model card | |
| 92 | | 14.6% | Jul 2023 | Lab model card |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Claude Opus 4.5 | 98.1% | Nov 2025 | Official leaderboard | |
| 2 | Claude Sonnet 4 | 97.7% | May 2025 | Official leaderboard | |
| 3 | 97.7% | Dec 2025 | Official leaderboard | ||
| 4 | 97.5% | Nov 2025 | Official leaderboard | ||
| 5 | Kimi K2.5 | 97.5% | Chat | Jan 2026 | Official leaderboard |
| 6 | Claude Sonnet 4.5 | 97.4% | Thinking | Sep 2025 | Official leaderboard |
| 7 | GPT-5.1 | 97.4% | Nov 2025 | Official leaderboard | |
| 8 | 97.3% | Auto | Jun 2025 | Official leaderboard | |
| 9 | 97.3% | Thinking | Sep 2025 | Official leaderboard | |
| 10 | Kimi K2 Thinking | 97.3% | Thinking | Nov 2025 | Official leaderboard |
| 11 | 97.2% | Mar 2025 | Official leaderboard | ||
| 12 | o3 | 97.2% | Apr 2025 | Official leaderboard | |
| 13 | o4-mini | 97.2% | Apr 2025 | Official leaderboard | |
| 14 | GPT-5 | 97.2% | Thinking (High) | Aug 2025 | Official leaderboard |
| 15 | GPT-5.2 | 97.2% | Dec 2025 | Official leaderboard | |
| 16 | Claude 3.7 Sonnet | 97.1% | Feb 2025 | Official leaderboard | |
| 17 | 97.1% | Mar 2025 | Official leaderboard | ||
| 18 | 97.1% | Jul 2025 | Official leaderboard | ||
| 19 | 97.1% | Thinking | Dec 2025 | Official leaderboard | |
| 20 | o3-mini | 97% | high | Jan 2025 | Official leaderboard |
| 21 | 97% | Sep 2025 | Official leaderboard | ||
| 22 | GLM-4.6 | 97% | Sep 2025 | Official leaderboard | |
| 23 | 97% | Oct 2025 | Official leaderboard | ||
| 24 | Kimi K2 Jul 2025 | 97% | Jul 2025 | Official leaderboard | |
| 25 | 97% | Thinking | Jul 2025 | Official leaderboard | |
| 26 | 97% | Sep 2025 | Official leaderboard | ||
| 27 | GPT-4.5 | 96.9% | Feb 2025 | Official leaderboard | |
| 28 | Claude Opus 4 | 96.9% | May 2025 | Official leaderboard | |
| 29 | 96.9% | Jun 2025 | Official leaderboard | ||
| 30 | 96.9% | Thinking | Aug 2025 | Official leaderboard | |
| 31 | 96.8% | Dec 2024 | Official leaderboard | ||
| 32 | 96.8% | Jun 2025 | Official leaderboard | ||
| 33 | gpt-oss-120b | 96.8% | high | Aug 2025 | Official leaderboard |
| 34 | Claude 3.5 Sonnet (October 2024) | 96.7% | Oct 2024 | Official leaderboard | |
| 35 | 96.7% | Thinking | Sep 2025 | Official leaderboard | |
| 36 | 96.5% | Apr 2025 | Official leaderboard | ||
| 37 | 96.5% | May 2025 | Official leaderboard | ||
| 38 | 96.4% | Jul 2024 | Official leaderboard | ||
| 39 | 96.4% | Mar 2025 | Official leaderboard | ||
| 40 | 96.4% | Jul 2025 | Official leaderboard | ||
| 41 | GLM-4.5 | 96.4% | Aug 2025 | Official leaderboard | |
| 42 | 96.4% | Thinking | Sep 2025 | Official leaderboard | |
| 43 | GPT-5.1 Instant | 96.4% | Nov 2025 | Official leaderboard | |
| 44 | 96.4% | Thinking | Dec 2025 | Official leaderboard | |
| 45 | 96.3% | Sep 2025 | Official leaderboard | ||
| 46 | 96.1% | Apr 2025 | Official leaderboard | ||
| 47 | GPT-4.1 | 96.1% | Apr 2025 | Official leaderboard | |
| 48 | 96.1% | Apr 2025 | Official leaderboard | ||
| 49 | GPT-4o Nov 2024 | 96.1% | Nov 2024 | Official leaderboard | |
| 50 | 96.1% | Mar 2025 | Official leaderboard | ||
| 51 | 95.9% | Sep 2024 | Official leaderboard | ||
| 52 | 95.8% | Oct 2025 | Official leaderboard | ||
| 53 | 95.8% | Apr 2025 | Official leaderboard | ||
| 54 | 95.7% | Nov 2024 | Official leaderboard | ||
| 55 | GPT-4o May 2024 | 95.6% | May 2024 | Official leaderboard | |
| 56 | Grok 4.1 | 95.6% | Thinking | Nov 2025 | Official leaderboard |
| 57 | Grok 3 | 95.5% | Apr 2025 | Official leaderboard | |
| 58 | gpt-oss-20b | 95.5% | high | Aug 2025 | Official leaderboard |
| 59 | 95.4% | Jun 2025 | Official leaderboard | ||
| 60 | 95.3% | Thinking | Apr 2025 | Official leaderboard | |
| 61 | 95.3% | Thinking | Jul 2025 | Official leaderboard | |
| 62 | 95.2% | Jul 2024 | Official leaderboard | ||
| 63 | 95.2% | Sep 2024 | Official leaderboard | ||
| 64 | 95% | Sep 2024 | Official leaderboard | ||
| 65 | 94.8% | Apr 2025 | Official leaderboard | ||
| 66 | Grok 4 | 94.5% | Jul 2025 | Official leaderboard | |
| 67 | 94.5% | Mar 2025 | Official leaderboard | ||
| 68 | GPT-4o mini | 93.6% | Jul 2024 | Official leaderboard | |
| 69 | 93% | Apr 2025 | Official leaderboard | ||
| 70 | 91.1% | Sep 2024 | Official leaderboard | ||
| 71 | 89.6% | Mar 2025 | Official leaderboard | ||
| 72 | 87.3% | Jun 2024 | Official leaderboard | ||
| 73 | 85.3% | Jul 2024 | Official leaderboard | ||
| 74 | 85.1% | Sep 2024 | Official leaderboard | ||
| 75 | GLM-4-9B | 82.9% | Jun 2024 | Official leaderboard | |
| 76 | 72.1% | Sep 2024 | Official leaderboard | ||
| 77 | 58.9% | Mar 2025 | Official leaderboard | ||
| 78 | 44.3% | Sep 2024 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Claude Opus 4.5 | 98.1% | Nov 2025 | Official leaderboard | |
| 2 | Claude Sonnet 4 | 97.7% | May 2025 | Official leaderboard | |
| 3 | 97.7% | Dec 2025 | Official leaderboard | ||
| 4 | 97.5% | Nov 2025 | Official leaderboard | ||
| 5 | Kimi K2.5 | 97.5% | Chat | Jan 2026 | Official leaderboard |
| 6 | Claude Sonnet 4.5 | 97.4% | Thinking | Sep 2025 | Official leaderboard |
| 7 | GPT-5.1 | 97.4% | Nov 2025 | Official leaderboard | |
| 8 | 97.3% | Auto | Jun 2025 | Official leaderboard | |
| 9 | 97.3% | Thinking | Sep 2025 | Official leaderboard | |
| 10 | Kimi K2 Thinking | 97.3% | Thinking | Nov 2025 | Official leaderboard |
| 11 | 97.2% | Mar 2025 | Official leaderboard | ||
| 12 | o3 | 97.2% | Apr 2025 | Official leaderboard | |
| 13 | o4-mini | 97.2% | Apr 2025 | Official leaderboard | |
| 14 | GPT-5 | 97.2% | Thinking (High) | Aug 2025 | Official leaderboard |
| 15 | GPT-5.2 | 97.2% | Dec 2025 | Official leaderboard | |
| 16 | Claude 3.7 Sonnet | 97.1% | Feb 2025 | Official leaderboard | |
| 17 | 97.1% | Mar 2025 | Official leaderboard | ||
| 18 | 97.1% | Jul 2025 | Official leaderboard | ||
| 19 | 97.1% | Thinking | Dec 2025 | Official leaderboard | |
| 20 | o3-mini | 97% | high | Jan 2025 | Official leaderboard |
| 21 | 97% | Sep 2025 | Official leaderboard | ||
| 22 | GLM-4.6 | 97% | Sep 2025 | Official leaderboard | |
| 23 | 97% | Oct 2025 | Official leaderboard | ||
| 24 | Kimi K2 Jul 2025 | 97% | Jul 2025 | Official leaderboard | |
| 25 | 97% | Thinking | Jul 2025 | Official leaderboard | |
| 26 | 97% | Sep 2025 | Official leaderboard | ||
| 27 | GPT-4.5 | 96.9% | Feb 2025 | Official leaderboard | |
| 28 | Claude Opus 4 | 96.9% | May 2025 | Official leaderboard | |
| 29 | 96.9% | Jun 2025 | Official leaderboard | ||
| 30 | 96.9% | Thinking | Aug 2025 | Official leaderboard | |
| 31 | 96.8% | Dec 2024 | Official leaderboard | ||
| 32 | 96.8% | Jun 2025 | Official leaderboard | ||
| 33 | gpt-oss-120b | 96.8% | high | Aug 2025 | Official leaderboard |
| 34 | 96.8% | Jul 2024 | Lab model card | ||
| 35 | Claude 3.5 Sonnet (October 2024) | 96.7% | Oct 2024 | Official leaderboard | |
| 36 | 96.7% | Thinking | Sep 2025 | Official leaderboard | |
| 37 | 96.5% | Apr 2025 | Official leaderboard | ||
| 38 | 96.5% | May 2025 | Official leaderboard | ||
| 39 | 96.4% | Mar 2025 | Official leaderboard | ||
| 40 | 96.4% | Apr 2025 | Lab model card | ||
| 41 | 96.4% | Jul 2025 | Official leaderboard | ||
| 42 | GLM-4.5 | 96.4% | Aug 2025 | Official leaderboard | |
| 43 | Claude 3.5 Sonnet | 96.4% | Jun 2024 | Lab model card | |
| 44 | 96.4% | Thinking | Sep 2025 | Official leaderboard | |
| 45 | GPT-5.1 Instant | 96.4% | Nov 2025 | Official leaderboard | |
| 46 | 96.4% | Thinking | Dec 2025 | Official leaderboard | |
| 47 | 96.3% | Jul 2026 | Lab model card | ||
| 48 | 96.3% | Sep 2025 | Official leaderboard | ||
| 49 | 96.2% | Jul 2025 | Lab model card | ||
| 50 | 96.1% | Apr 2025 | Official leaderboard | ||
| 51 | GPT-4.1 | 96.1% | Apr 2025 | Official leaderboard | |
| 52 | 96.1% | Apr 2025 | Official leaderboard | ||
| 53 | GPT-4o Nov 2024 | 96.1% | Nov 2024 | Official leaderboard | |
| 54 | 96.1% | Mar 2025 | Official leaderboard | ||
| 55 | 95.9% | Sep 2024 | Official leaderboard | ||
| 56 | 95.9% | Dec 2024 | Lab model card | ||
| 57 | 95.8% | Oct 2025 | Official leaderboard | ||
| 58 | 95.8% | Apr 2025 | Official leaderboard | ||
| 59 | 95.7% | Nov 2024 | Official leaderboard | ||
| 60 | GPT-4o May 2024 | 95.6% | May 2024 | Official leaderboard | |
| 61 | Grok 4.1 | 95.6% | Thinking | Nov 2025 | Official leaderboard |
| 62 | Grok 3 | 95.5% | Apr 2025 | Official leaderboard | |
| 63 | gpt-oss-20b | 95.5% | high | Aug 2025 | Official leaderboard |
| 64 | 95.5% | Jan 2025 | Lab model card | ||
| 65 | 95.4% | Jun 2025 | Official leaderboard | ||
| 66 | 95.3% | Feb 2025 | Lab model card | ||
| 67 | 95.3% | Thinking | Apr 2025 | Official leaderboard | |
| 68 | 95.3% | Thinking | Jul 2025 | Official leaderboard | |
| 69 | 95.2% | Jul 2024 | Official leaderboard | ||
| 70 | Claude 3 Opus | 95% | Mar 2024 | Lab model card | |
| 71 | 95% | Sep 2024 | Official leaderboard | ||
| 72 | 94.8% | Jan 2025 | Lab model card | ||
| 73 | 94.8% | Jan 2025 | Lab model card | ||
| 74 | 94.8% | Apr 2025 | Official leaderboard | ||
| 75 | Grok 4 | 94.5% | Jul 2025 | Official leaderboard | |
| 76 | 94.5% | Jun 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | ||
| 77 | 94.5% | Mar 2025 | Official leaderboard | ||
| 78 | 94.4% | Dec 2023 | Lab model card | ||
| 79 | Qwen2.5-Coder-14B-Instruct | 94.2% | Nov 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| 80 | Granite 4.1 30B | 94.2% | Apr 2026 | Lab model card | |
| 81 | Sarvam-M | 94% | May 2025 | Lab model card | |
| 82 | 93.8% | Dec 2024 | Lab model card | ||
| 83 | GPT-4o mini | 93.6% | Jul 2024 | Official leaderboard | |
| 84 | 93.3% | Sep 2025 | Lab model card | ||
| 85 | 93.2% | Jul 2024 | Lab model card | ||
| 86 | 93% | Apr 2025 | Official leaderboard | ||
| 87 | 93% | Apr 2024 | Lab model card | ||
| 88 | 93% | Jul 2024 | Lab model card | ||
| 89 | 93% | Nov 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | ||
| 90 | 93% | Sep 2026 | Lab model card | ||
| 91 | 93% | Sep 2026 | Lab model card | ||
| 92 | 92.6% | Apr 2026 | Lab model card | ||
| 93 | 92.3% | Dec 2025 | Lab model card | ||
| 94 | Claude 3 Sonnet | 92.3% | Mar 2024 | Lab model card | |
| 95 | 92.2% | May 2024 | Lab model card | ||
| 96 | GPT-4 Mar 2023 | 92% | Mar 2023 | Lab model card | |
| 97 | 91.7% | Oct 2026 | Lab model card | ||
| 98 | 91.7% | Oct 2026 | Lab model card | ||
| 99 | 91.7% | Feb 2024 | Lab model card | ||
| 100 | 91.6% | Dec 2024 | Lab model card | ||
| 101 | 91.6% | Aug 2025 | Lab model card | ||
| 102 | 91.2% | Feb 2024 | Lab model card | ||
| 103 | 91.1% | Sep 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | ||
| 104 | PaLM 2-L | 91% | May 2023 | Lab model card | |
| 105 | 91% | Aug 2024 | Lab model card | ||
| 106 | 90.8% | May 2024 | Lab model card | ||
| 107 | Grok-1.5 | 90% | Mar 2024 | Lab model card | |
| 108 | GPT-4 Jun 2023 | 90% | Jun 2023 | Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI | |
| 109 | 89.6% | Mar 2025 | Official leaderboard | ||
| 110 | 89.6% | Aug 2024 | Lab model card | ||
| 111 | Claude 3 Haiku | 88.9% | Mar 2024 | Lab model card | |
| 112 | 88.7% | Aug 2024 | Lab model card | ||
| 113 | Qwen2.5-Coder-14B | 88.7% | Sep 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| 114 | 88.7% | Mar 2025 | Lab model card | ||
| 115 | 88.6% | Feb 2025 | Lab model card | ||
| 116 | 88.1% | Jun 2026 | Lab model card | ||
| 117 | Claude 2 | 88% | Jul 2023 | Lab model card | |
| 118 | GLM-4 | 87.6% | Jan 2024 | Lab model card | |
| 119 | DeepSeek-Coder-V2-Lite-Instruct | 87.6% | Jun 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| 120 | 87.6% | Mar 2025 | Lab model card | ||
| 121 | 87.3% | Jun 2024 | Official leaderboard | ||
| 122 | Granite 4.0 H Small | 87.3% | Oct 2025 | Lab model card | |
| 123 | Claude Instant | 86.7% | Aug 2023 | Claim in Releasing Claude Instant 1.2, via Epoch AI | |
| 124 | 86.7% | Sep 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | ||
| 125 | 86.2% | Aug 2024 | Lab model card | ||
| 126 | LFM2.5-1.2B-Thinking | 85.6% | Jan 2026 | Lab model card | |
| 127 | 85.3% | Jul 2024 | Official leaderboard | ||
| 128 | 85.1% | Sep 2024 | Official leaderboard | ||
| 129 | LFM2-8B-A1B | 84.4% | Oct 2025 | Lab model card | |
| 130 | 84.2% | Jul 2024 | Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI | ||
| 131 | 84.1% | Nov 2023 | Lab model card | ||
| 132 | 83.6% | May 2025 | Lab model card | ||
| 133 | GLM-4-9B | 82.9% | Jun 2024 | Official leaderboard | |
| 134 | 82.5% | Aug 2024 | Lab model card | ||
| 135 | 82.4% | May 2024 | Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI | ||
| 136 | Granite 3.2 8B Instruct | 81.7% | Feb 2025 | Lab model card | |
| 137 | Granite 3.3 8B Instruct | 80.9% | Apr 2025 | Lab model card | |
| 138 | 80.7% | Jan 2025 | Lab model card | ||
| 139 | Qwen2.5-Coder-3B-Instruct | 80.7% | Nov 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| 140 | 79.6% | Apr 2024 | Lab model card | ||
| 141 | 79.5% | Feb 2024 | Lab model card | ||
| 142 | davinci-003 | 78.2% | Nov 2022 | Claim in Qwen Technical Report, via Epoch AI | |
| 143 | 77.6% | Sep 2025 | Lab model card | ||
| 144 | Moonlight-16B-A3B | 77.4% | Feb 2025 | Lab model card | |
| 145 | granite-4.0-1b | 76.3% | Oct 2025 | Lab model card | |
| 146 | 76% | Nov 2023 | Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI | ||
| 147 | Qwen2.5-Coder-3B | 75.7% | Sep 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| 148 | 74.4% | Dec 2023 | Claim in Mixtral of Experts, via Epoch AI | ||
| 149 | 74% | Jun 2024 | Lab model card | ||
| 150 | ChatGLM3-6B | 72.3% | Oct 2023 | Lab model card | |
| 151 | 72.1% | Sep 2024 | Official leaderboard | ||
| 152 | 69.6% | Jul 2023 | Claim in Mixtral of Experts, via Epoch AI | ||
| 153 | 69.6% | Jul 2023 | Claim in Qwen Technical Report, via Epoch AI | ||
| 154 | GLM-5 | 68.8% | Feb 2026 | Lab model card | |
| 155 | 68.3% | Apr 2025 | Lab model card | ||
| 156 | 67.7% | Nov 2025 | Lab model card | ||
| 157 | DeepSeek-Coder-V2-Lite-Base | 67.1% | Jun 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| 158 | 65.8% | Sep 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | ||
| 159 | internlm-20b | 62.9% | Sep 2023 | Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI | |
| 160 | Grok-1 | 62.9% | Nov 2023 | Lab model card | |
| 161 | 62.8% | Mar 2025 | Lab model card | ||
| 162 | 61.3% | Sep 2023 | Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI | ||
| 163 | 61.1% | Dec 2023 | Lab model card | ||
| 164 | 58.4% | Apr 2025 | Lab model card | ||
| 165 | LFM2-1.2B | 58.3% | Jul 2025 | Lab model card | |
| 166 | GPT-3.5 Turbo Jun 2023 | 57.8% | Jun 2023 | Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI | |
| 167 | 57.7% | Feb 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | ||
| 168 | 56.5% | Apr 2022 | Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI | ||
| 169 | 54.4% | Feb 2023 | Claim in Qwen Technical Report, via Epoch AI | ||
| 170 | 54.4% | Sep 2023 | Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI | ||
| 171 | 54.4% | Sep 2023 | Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI | ||
| 172 | 54.1% | Sep 2023 | Lab model card | ||
| 173 | 53.8% | May 2024 | Claim in Falcon2-11B Technical Report, via Epoch AI | ||
| 174 | 52.8% | Sep 2023 | Claim in Nemotron-4 15B Technical Report, via Epoch AI | ||
| 175 | 46.4% | Feb 2024 | Lab model card | ||
| 176 | 46% | Feb 2024 | Claim in Nemotron-4 15B Technical Report, via Epoch AI | ||
| 177 | Baichuan2-13B-Chat | 45.7% | Sep 2023 | Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI | |
| 178 | 44.9% | Nov 2023 | Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI | ||
| 179 | 44.3% | Sep 2024 | Official leaderboard | ||
| 180 | 44.1% | Feb 2023 | Claim in Mixtral of Experts, via Epoch AI | ||
| 181 | 42.2% | Jul 2023 | Lab model card | ||
| 182 | INTELLECT-1 | 38.6% | Nov 2024 | Claim in INTELLECT-1 Technical Report, via Epoch AI | |
| 183 | CodeQwen1.5-7B | 37.7% | Apr 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| 184 | 36.9% | Jul 2023 | Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI | ||
| 185 | 35.4% | Nov 2023 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | ||
| 186 | 35.4% | Dec 2023 | Claim in Gemma: Open Models Based on Gemini Research and Technology, via Epoch AI | ||
| 187 | Qwen2.5-Coder-0.5B | 34.5% | Sep 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| 188 | PaLM 62B | 33% | Apr 2022 | Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI | |
| 189 | 32.7% | Feb 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | ||
| 190 | chatglm2-6b | 32.4% | Jun 2023 | Claim in Qwen Technical Report, via Epoch AI | |
| 191 | internlm-7b | 31.2% | Jul 2023 | Claim in Qwen Technical Report, via Epoch AI | |
| 192 | vicuna-13b-v1.1 | 28.1% | Apr 2023 | Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI | |
| 193 | Qwen-1_8B | 27.8% | Sep 2023 | Lab model card | |
| 194 | 26.8% | Jul 2023 | Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI | ||
| 195 | 24.6% | Sep 2023 | Claim in Qwen Technical Report, via Epoch AI | ||
| 196 | 21.6% | Feb 2024 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | ||
| 197 | 21.5% | Mar 2023 | Claim in Falcon2-11B Technical Report, via Epoch AI | ||
| 198 | 21.3% | Nov 2023 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | ||
| 199 | 20.6% | Feb 2023 | Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI | ||
| 200 | 17.7% | Mar 2024 | Lab model card | ||
| 201 | 16.7% | Jul 2023 | Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI | ||
| 202 | internlm-chat-20b | 15.7% | Sep 2023 | Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI | |
| 203 | 15.2% | Jun 2023 | Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI | ||
| 204 | 11% | Feb 2023 | Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI | ||
| 205 | 9.20% | Jun 2023 | Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI | ||
| 206 | 9.10% | May 2023 | Claim in Qwen Technical Report, via Epoch AI | ||
| 207 | 6.80% | Apr 2023 | Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI | ||
| 208 | 4.40% | Nov 2023 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI |
AGI-Eval’s leaderboard has 92 entries, 78 of them models we track. Epoch AI collected claims for 70 models from labs’ reports and papers. Switch the scores above to rank them together.
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: AGI-Eval’s leaderboard
5 models have held number one in labs’ own numbers since 2023 9 models have held number one across all independent runs since 2023 13 models have held number one with claims included since 2023
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for AGI-Eval’s leaderboard and what labs claim in their cards.
Sources: AGI-Eval’s leaderboard and labs’ model cards
OpenAI’s own number for GPT-4 passed 85% in March 2023
That was 17 months after its release. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: AGI-Eval’s leaderboard, labs’ model cards and claims collected by Epoch AI (CC BY 4.0)
Labs headline it less often now, last in September 2026
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Sep 10, 2026 | 93 | GSM8K (EM) | |
| Apr 29, 2026 | Granite 4.1 30B | 94.2 | GSM8K (8-shot) |
| Apr 24, 2026 | 92.6 | GSM8K (EM) | |
| Jan 20, 2026 | LFM2.5-1.2B-Thinking | 85.6 | GSM8K |
| Oct 28, 2025 | granite-4.0-1b | 76.3 | GSM8K (8-shot) |
| Oct 7, 2025 | LFM2-8B-A1B | 84.4 | GSM8K |
| Oct 2, 2025 | Granite 4.0 H Small | 87.3 | GSM8K (8-shot) |
| Jul 31, 2025 | 96.2 | GSM8K (thinking) | |
| Jul 10, 2025 | LFM2-1.2B | 58.3 | GSM8K |
| Jun 30, 2025 | 96.6 | GSM8K | |
| May 23, 2025 | Sarvam-M | 94 | GSM-8K (0.94) |
| May 20, 2025 | 83.6 | GSM8k | |
| Apr 29, 2025 | 68.3 | GSM8K | |
| Apr 16, 2025 | Granite 3.3 8B Instruct | 80.9 | GSM8K |
| Apr 15, 2025 | 58.4 | GSM8K | |
| Apr 8, 2025 | 96.4 | GSM8K (thinking) | |
| Mar 27, 2025 | 88.7 | GSM8K | |
| Mar 16, 2025 | GSM8K | ||
| Mar 13, 2025 | 87.6 | GSM8k | |
| Mar 12, 2025 | 95.9 | GSM8K 0-shot (IT); PT 8-shot = 82.6 | |
| Feb 26, 2025 | 88.6 | GSM8K (8-shot, CoT) | |
| Feb 26, 2025 | Granite 3.2 8B Instruct | 81.7 | GSM8K |
| Feb 22, 2025 | Moonlight-16B-A3B | 77.4 | GSM8K |
| Jan 30, 2025 | 95.5 | GSM8K (8 shot, CoT) | |
| Jan 30, 2025 | 80.7 | GSM8K (5-shot, maj@1) [base] | |
| Jan 28, 2025 | GSM8K | ||
| Jan 15, 2025 | 94.8 | GSM8k* | |
| Dec 26, 2024 | 89.3 | GSM8K (EM) | |
| Sep 19, 2024 | 95.8 | GSM8K | |
| Aug 20, 2024 | 88.7 | GSM8K (8-shot, CoT) | |
| Jul 24, 2024 | 93 | GSM8K (8-shot) | |
| Jul 23, 2024 | 96.8 | GSM8K (8-shot, CoT) | |
| Jun 27, 2024 | 74 | GSM8K (5-shot, maj@1) | |
| Jun 20, 2024 | Claude 3.5 Sonnet | 96.4 | GSM8K (0-shot CoT) |
| Jun 7, 2024 | 91.1 | GSM8K | |
| May 14, 2024 | 90.8 | GSM8K, 11-shot | |
| May 7, 2024 | 92.2 | GSM8K | |
| Apr 22, 2024 | 90.3 | GSM-8K (0-Shot; CoT) | |
| Apr 18, 2024 | 93 | GSM-8K (8-shot, CoT) [instruct] | |
| Mar 28, 2024 | Grok-1.5 | 90 | GSM8K (8-shot, maj@1) |
| Mar 4, 2024 | Claude 3 Opus | 95 | GSM8K (0-shot CoT) |
| Feb 26, 2024 | 91.2 | GSM8K maj@8 (8-shot) [maj@1 (5-shot) = 81.0] | |
| Feb 21, 2024 | 46.4 | GSM8K (maj@1) | |
| Feb 15, 2024 | 91.7 | GSM8K, 11-shot | |
| Feb 5, 2024 | 79.5 | GSM8K | |
| Jan 16, 2024 | GLM-4 | 87.6 | GSM8K (5-shot) |
| Dec 12, 2023 | 61.1 | Math (GSM8k, 8-shot) | |
| Dec 11, 2023 | 58.4 | GSM-8K (5-shot) [detailed table: 74.4] | |
| Dec 6, 2023 | 94.4 | GSM8K, Maj1@32 | |
| Nov 29, 2023 | 84.1 | GSM8K | |
| Nov 3, 2023 | Grok-1 | 62.9 | GSM8k (8-shot) |
| Oct 27, 2023 | ChatGLM3-6B | 72.3 | GSM8K |
| Sep 27, 2023 | 52.1 | GSM8K (8-shot, maj@8) | |
| Sep 25, 2023 | 60.1 | GSM8K (0-shot) | |
| Jul 11, 2023 | Claude 2 | 88 | GSM8k (0-shot CoT) |
| May 10, 2023 | 91 | GSM8K, CoT + self-consistency (greedy CoT = 80.7) | |
| Mar 14, 2023 | GPT-4 | 92 | GSM-8K (5-shot chain-of-thought) |
72 system cards report it, from 17 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Oct 15, 2026 | | 91.7 | GSM8K, 8-shot CoT exact match, Table 6 | Capabilities |
| Oct 15, 2026 | | 91.7 | GSM8K, 8-shot CoT exact match, Table 6 | Capabilities |
| Sep 10, 2026 | | 93 | GSM8K, EM, 8-shot | Capabilities |
| Jul 31, 2026 | | 96.3 | GSM8K, TD, 95.8 no-think (Table 3) | Capabilities |
| Jun 4, 2026 | | 88.1 | GSM8K, 8-shot CoT EM | Capabilities |
| Feb 17, 2026 | GLM-5 technical report | 68.8 | GLM-5-Base, GSM8K (EM) (Table 11) | Capabilities |
| Dec 23, 2025 | | 92.3 | GSM8K, 8-shot accuracy, base model Table 2 | Capabilities |
| Nov 28, 2025 | LFM2-8B-A1B technical report | 84.4 | GSM8K, LFM2-8B-A1B (Table 7) | Capabilities |
| Nov 28, 2025 | LFM2-1.2B technical report | 58.3 | GSM8K, LFM2-1.2B (Table 6) | Capabilities |
| Nov 8, 2025 | | 67.7 | GSM8K 5-shot, English (Table 8). Arabic-GSM8K exact match 54.89 (Table 7) | Capabilities |
| Oct 8, 2025 | | 68.3 | GSM8K (Table 7, Table 23) | Capabilities |
| Sep 30, 2025 | | 93.3 | GSM8k, without reasoning (Table 13) | Capabilities |
| Sep 17, 2025 | | 77.6 | GSM8K, 8-shot CoT (Table 18) | Capabilities |
| Aug 25, 2025 | | 91.6 | GSM8K, OpenCompass (Table 14) | Capabilities |
| Aug 8, 2025 | GLM-4.5 technical report | 79.4 | GLM-4.5-Base, GSM8K (EM) (Table 2) | Capabilities |
| Jul 30, 2025 | | 83.6 | GSM8k, strict match 5-shot (Table 24) | Capabilities |
| Jul 17, 2025 | | GSM8K 8-shot, text-only, chart without printed numbers (Figure 6). Speech-synthesized GSM8k* 89.7 (Table 8) | Capabilities | |
| Jun 29, 2025 | | 96.6 | GSM8K (Table 5) | Capabilities |
| Apr 16, 2025 | | 58.4 | GSM8K (4-shot; EM) (Table 1, Table 2) | Capabilities |
| Mar 26, 2025 | | 88.7 | GSM8K (Table 1) | Capabilities |
| Mar 25, 2025 | | 89.2 | GSM8K, IT, 0-shot CoT (Table 18) | Capabilities |
| Mar 25, 2025 | | 95.9 | GSM8K, IT, 0-shot CoT (Table 18) | Capabilities |
| Mar 25, 2025 | | 62.8 | GSM8K, IT, 0-shot CoT (Table 18) | Capabilities |
| Mar 25, 2025 | | 94.4 | GSM8K, IT, 0-shot CoT (Table 18) | Capabilities |
| Mar 3, 2025 | | 88.6 | GSM-8K (8-Shot, CoT) (Table 7) | Capabilities |
| Feb 19, 2025 | | 95.3 | GSM8K (Table 4) | Capabilities |
| Jan 26, 2025 | | 94.8 | GSM8K (Table 6) | Capabilities |
| Jan 14, 2025 | | 94.8 | GSM8k, 0-shot CoT (Table 8) | Capabilities |
| Dec 31, 2024 | | 87.6 | GSM8K, 8-shot CoT (Tables 7 and 16) | Capabilities |
| Dec 19, 2024 | | 91.6 | GSM8K, Qwen2.5-7B-Instruct (Table 8) | Capabilities |
| Dec 19, 2024 | | 95.8 | GSM8K, Qwen2.5-72B-Instruct (Table 6) | Capabilities |
| Dec 19, 2024 | | 95.9 | GSM8K, Qwen2.5-32B-Instruct (Table 7) | Capabilities |
| Dec 19, 2024 | | 94.8 | GSM8K, Qwen2.5-14B-Instruct (Table 7) | Capabilities |
| Dec 19, 2024 | | 93.8 | GSM8K, Qwen2.5-Turbo (Table 7) | Capabilities |
| Nov 23, 2024 | | 84.5 | GSM8K (8-shot, CoT), post-trained (Table 2). Pretrained many-shot GSM8K (16-shot) = 60.0 (Table 14) | Capabilities |
| Nov 23, 2024 | | 95.1 | GSM8K (8-shot, CoT), post-trained (Table 2). Pretrained many-shot GSM8K (16-shot) = 83.0 (Table 14) | Capabilities |
| Nov 23, 2024 | | 96.8 | GSM8K (8-shot, CoT), post-trained (Table 2). Pretrained many-shot GSM8K (16-shot) = 90.0 (Table 14) | Capabilities |
| Nov 22, 2024 | | 95.5 | GSM8K (8 shot, CoT), RLVR (Table 4) | Capabilities |
| Aug 30, 2024 | | 89.6 | GSM-8K (8-Shot, CoT) (Section 3 table) | Capabilities |
| Aug 30, 2024 | | 82.5 | GSM-8K (8-Shot, CoT) (Section 3 table) | Capabilities |
| Aug 30, 2024 | | 91 | GSM-8K (8-Shot, CoT) (Section 3 table) | Capabilities |
| Aug 30, 2024 | | 88.7 | GSM8K (8-shot, CoT) (Table 3) | Capabilities |
| Aug 30, 2024 | | 86.2 | GSM8K (8-shot, CoT) (Table 3) | Capabilities |
| Jul 31, 2024 | | 68.6 | GSM8K, PT, 5-shot (Table 13) | Capabilities |
| Jul 31, 2024 | | 74 | GSM8K, PT, 5-shot (Table 13) | Capabilities |
| Jul 23, 2024 | | 84.5 | GSM-8K (CoT), 8-shot, em_maj1@1, instruction tuned (Instruction tuned models table) | Capabilities |
| Jul 23, 2024 | | 95.1 | GSM-8K (CoT), 8-shot, em_maj1@1, instruction tuned (Instruction tuned models table) | Capabilities |
| Jul 23, 2024 | | 96.8 | GSM-8K (CoT), 8-shot, em_maj1@1, instruction tuned (Instruction tuned models table) | Capabilities |
| Jul 15, 2024 | | 93.2 | GSM8K, Qwen2-72B-Instruct (Table 6) | Capabilities |
| Jun 20, 2024 | Claude 3.5 Sonnet model card | 96.4 | GSM8K, 0-shot CoT (Table 1) | Capabilities |
| Jun 18, 2024 | GLM-4 technical report | 87.6 | GLM-4 (0116), GSM8K, chain-of-thought (Table 2) | Capabilities |
| Jun 18, 2024 | ChatGLM3-6B technical report | 72.3 | ChatGLM3-6B-Base, GSM8K (Table 1) | Capabilities |
| May 7, 2024 | | 92.2 | GSM8K 8-shot, DeepSeek-V2 Chat (RL) (Table 3) | Capabilities |
| Apr 18, 2024 | | 79.6 | GSM-8K (8-shot, CoT), instruction tuned (Instruction tuned models table) | Capabilities |
| Apr 18, 2024 | | 93 | GSM-8K (8-shot, CoT), instruction tuned (Instruction tuned models table) | Capabilities |
| Mar 13, 2024 | | 46.4 | GSM8K, PT, maj@1 (Table 6). HuggingFace H6 run (Table 7): 50.9 | Capabilities |
| Mar 13, 2024 | | 17.7 | GSM8K, PT, maj@1 (Table 6) | Capabilities |
| Mar 4, 2024 | Claude 3 Sonnet model card | 92.3 | GSM8K, 0-shot CoT (Table 1) | Capabilities |
| Mar 4, 2024 | Claude 3 Opus model card | 95 | GSM8K, 0-shot CoT (Table 1) | Capabilities |
| Mar 4, 2024 | Claude 3 Haiku model card | 88.9 | GSM8K, 0-shot CoT (Table 1) | Capabilities |
| Jan 8, 2024 | | 58.4 | GSM-8K, 5-shot (Table 3). 8-shot maj@8 74.4 (Table 2) | Capabilities |
| Jan 5, 2024 | | 84.1 | GSM8K 0-shot, chain of thought, DeepSeek 67B Chat (Table 6). 86.7 with tool-integrated reasoning (Table 16) | Capabilities |
| Oct 10, 2023 | | 52.2 | GSM8K, 8-shot maj@8 (Table 2) | Capabilities |
| Sep 28, 2023 | | 54.1 | GSM8K 8-shot, higher of the two settings, 0-shot 50.3 (Table 5) | Capabilities |
| Sep 28, 2023 | | 60.1 | GSM8K 0-shot, higher of the two settings, 8-shot 59.3 (Table 5) | Capabilities |
| Sep 28, 2023 | | 27.8 | GSM8K 0-shot, higher of the two settings, 8-shot 19.5 (Table 5) | Capabilities |
| Jul 19, 2023 | | 14.6 | GSM8k, 8-shot, maj1@1, pretrained (Table 25) | Capabilities |
| Jul 19, 2023 | | 56.8 | GSM8k, 8-shot, maj1@1, pretrained (Table 25) | Capabilities |
| Jul 19, 2023 | | 42.2 | GSM8k, 8-shot, maj1@1, pretrained (Table 25) | Capabilities |
| Jul 19, 2023 | | 28.7 | GSM8k, 8-shot, maj1@1, pretrained (Table 25) | Capabilities |
| Jul 11, 2023 | Claude 2 model card | 88 | GSM8k (0-shot CoT) (4.3) | Capabilities |
| Mar 14, 2023 | GPT-4 technical report | 92 | GSM-8K, 5-shot chain-of-thought, part of the training set in pre-training mix (Table 2, base model) | Capabilities |
The lab's number comes out ahead in 4 of 12 pairs that differ
Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 3 of them match, which usually means one side copied the other. Every benchmark.
| Model | Lab says | Board says | Gap |
|---|---|---|---|
| | 68.6% GSM8K, PT, 5-shot (Table 13) | 87.3% Leaderboard | −19 pts |
| GLM-4.5 | 79.4% GLM-4.5-Base, GSM8K (EM) (Table 2) | 96.4% Leaderboard | −17 pts |
| | 89.3% GSM8K (EM) | 96.8% Leaderboard | −7.5 pts |
| | 62.8% GSM8K, IT, 0-shot CoT (Table 18) | 58.9% Leaderboard | +3.9 pts |
| | 84.5% GSM-8K (CoT), 8-shot, em_maj1@1, instruction tuned (Instruction tuned models table) | 85.3% Leaderboard | −0.8 pts |
| | 95.9% GSM8K, Qwen2.5-32B-Instruct (Table 7) | 95.2% Leaderboard | +0.7 pts |
| | 91.6% GSM8K, Qwen2.5-7B-Instruct (Table 8) | 91.1% Leaderboard | +0.5 pts |
| | 89.2% GSM8K, IT, 0-shot CoT (Table 18) | 89.6% Leaderboard | −0.4 pts |
| | 96.8% GSM8K (8-shot, CoT) | 96.4% Leaderboard | +0.4 pts |
| | 96.6% GSM8K | 96.8% Leaderboard | −0.2 pts |
| | 94.8% GSM8K, Qwen2.5-14B-Instruct (Table 7) | 95% Leaderboard | −0.2 pts |
| | 95.9% GSM8K 0-shot (IT); PT 8-shot = 82.6 | 96.1% Leaderboard | −0.2 pts |
| | 95.1% GSM-8K (CoT), 8-shot, em_maj1@1, instruction tuned (Instruction tuned models table) | 95.2% Leaderboard | Same |
| | 95.8% GSM8K | 95.9% Leaderboard | Same |
| | 94.4% GSM8K, IT, 0-shot CoT (Table 18) | 94.5% Leaderboard | Same |
Versions
More in Math
- AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
- IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
- HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
- PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked.
Qwen3.7-Max 86.5%
- MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
- MathArena Apex Shortlist In use Models solve 47 hard final-answer problems that MathArena selected from 2025 competitions, where the best models scored about 50% at selection. Scored by accuracy over repeated attempts. GPT-5.5 98.4%
Data audit
8 of 10 checks pass, 1 warning, 1 problem. Look first at the Known issues check. 1 open finding: Contamination. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (92 entries), launch posts (55 scores), system cards (71 scores) and claims Epoch AI collected (70 models).
- Pass Models scored 208 models we track, plus 14 other entries on its leaderboard.
- Pass Newest score The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base.
- Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34).
- Note Headroom Saturated. The best is 98.1%, past 85% since March 2023.
Leaderboard import 3 pass
- Pass Importer We read the board on AGI-Eval: 98 rows for models we track and 14 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 problem 1 warning
- Warning Lab claims 2 of 15 pairs differ by more than 10 points, the largest for Gemma 2 9B (−19 pts).
- Problem Known issues 1 open finding: Contamination.