MMLU-Pro
The official leaderboard hasn’t scored a model released after GPT-5.4 (March 2026). Labs report their own numbers for 42 newer models, such as Intern-S2-397B at 89.8%.
Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects.
Launch posts report it widely. The makers fixed answers and formatting many times from 2024-07 to 2026-01, including a leading-space shortcut in STEM options on January 18th, 2026, and kept one dataset.
- Made by
- TIGER-Lab (Waterloo)
- Released
- June 2024
- Kind
- Exam questions
- Level
- University
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- All public
- Measures
- Capability
- Subject
- Mathematics, Physical sciences, Chemical sciences, Biological sciences, Computer and information sciences
- Language
- English
- Interaction
- Single turn
- Items from
- Another benchmark
- Refresh
- One fixed set
- Run by
- Anyone
- Based on
- MMLU
- In views
- Biology and medicine, Chemistry and materials
- Strongest model tested
- GPT-5.4 #28 on the capability index
- Last in a launch post
- October 2026
- Last model added
- October 2026
- Scores refreshed
- November 19th, 2026
In the authors’ words
In the age of large-scale language models, benchmarks like the Massive Multitask Language Understanding (MMLU) have been pivotal in pushing the boundaries of what AI can achieve in language comprehension and reasoning across diverse domains. However, as models continue to improve, their performance on these benchmarks has begun to plateau, making it increasingly difficult to discern differences in model capabilities. This paper introduces MMLU-Pro, an enhanced dataset designed to extend the mostly knowledge-driven MMLU benchmark by integrating more challenging, reasoning-focused questions and expanding the choice set from four to ten options. Additionally, MMLU-Pro eliminates the trivial and noisy questions in MMLU. Our experimental results show that MMLU-Pro not only raises the challenge, causing a significant drop in accuracy by 16% to 33% compared to MMLU but also demonstrates greater stability under varying prompts. With 24 different prompt styles tested, the sensitivity of model scores to prompt variations decreased from 4-5% in MMLU to just 2% in MMLU-Pro. Additionally, we found that models utilizing Chain of Thought (CoT) reasoning achieved better performance on MMLU-Pro compared to direct answering, which is in stark contrast to the findings on the original MMLU, indicating that MMLU-Pro includes more complex reasoning questions. Our assessments confirm that MMLU-Pro is a more discriminative benchmark to better track progress in the field.
Intern-S2-397B leads with 89.8%
Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.
Gemini 3.1 Pro leads with 91.2% across all independent runs
Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.
Gemini 3.1 Pro leads with 91.2%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
- 89.8
Intern-S2-397B
- 89.6
Qwen3.7-Max
- 88.5
Qwen 3.6 Plus
- 88.5
Qwen3.7-Plus
- 88.0
MiniMax-M2.1
- 87.8
Qwen3.5 397B-A17B
- 87.5
DeepSeek-V4-Pro
- 87.2
Hunyuan-T1
- 87.1 Kimi K2.5
- 87.0
Seed-Thinking-v1.5
- 87.0
Seed2.0 Pro
- 86.8
Nemotron 3 Ultra
- 86.7
Qwen3.5-122B-A10B
- 86.6
Intern-S1-Pro
- 86.3
Solar Pro 4
- 86.2
Qwen3.6 27B
- 86.2
DeepSeek-V4-Flash
- 86.2
Solar Open 2
- 86.1
Qwen3.5-27B
- 85.9
Qwen3.5-Omni-Plus
- 85.7
Qwen3-Max-2026-01-23
- 85.4
Cogito v2 Preview 671B MoE
- 85.3
Qwen3.5-35B-A3B
- 85.2
Gemma 4 31B IT
- 85.2
Qwen 3.6 35B-A3B
- 85.2
MiniMax-M2.5
- 85.0
DeepSeek-R1
- 85.0
DeepSeek-V3.1-Terminus
- 85.0
DeepSeek-V3.2-Exp
- 85.0
DeepSeek-V3.2
- 85.0
MAI-Thinking-1
- 84.9 MiMo-V2-Flash
- 84.9
Seed1.8
- 84.8
DeepSeek-V3.1
- 84.7
Cogito v2.1
- 84.6 GLM-4.5
- 84.6 Kimi K2 Thinking
- 84.4
Qwen3-235B-A22B-Thinking
- 84.4
Step 3.5 Flash
- 84.3 GLM-4.7
- 84.3
ERNIE 5.1
- 84.0
DeepSeek-R1
- 83.8
Qwen3-VL-235B-A22B
- 83.8
K-EXAONE
- 83.8
ERNIE 5.0
- 83.7
Nemotron 3 Super
- 83.5
Intern-S1
- 83.5
K-EXAONE 2.0
- 83.5
MiniCPM-V-4.7-35B-A3B
- 83.4
Trinity Large Thinking
- 83.3
EXAONE 4.5 33B
- 82.7
LongCat-Flash-Omni
- 82.7
seed-oss-36b-instruct
- 82.7
Qwen3-Next-80B-A3B
- 82.7
LongCat-Flash-Chat
- 82.6
LongCat-Flash-Thinking
- 82.6
Gemma 4 26B A4B
- 82.5
Qwen3.5-9B
- 82.2
Llama 4 Behemoth
- 82.0
MiniMax-M2
- 81.9
Nemotron 3.5 Lightning 30B-A3B
- 81.9 INTELLECT-3
- 81.8
EXAONE 4.0 32B
- 81.8
MiniMax-M2.7
- 81.7 Sarvam 105B
- 81.6
Nova 2.0 Pro Preview
- 81.4 GLM-4.5-Air
- 81.3
InternVL3.5-241B-A28B
- 81.2
Solar Pro 3 (March 2026 update)
- 81.2
DeepSeek-V3
- 81.1
MiniMax-M1-80k
- 81.1 Kimi K2
- 80.9
Nova 2 Lite
- 80.7
Nova 2 Omni Preview
- 80.7 Hermes 4.3 36B
- 80.6 Hermes 4 405B
- 80.5
Ring-1T
- 80.5
Qwen3-Coder-Next
- 80.5
Llama 4 Maverick
- 80.4
Solar Open 100B
- 80.0 Sarvam 30B
- 79.9 Grok 3
- 79.8
Nemotron-Cascade 2 30B-A3B
- 79.5
Llama-3.3-Nemotron-Super-49B-v1.5
- 79.1
Gemini 2.0 Pro
- 79.1
Qwen3.5-4B
- 79.1
Nemotron 3 Ultra 550B-A55B Base
- 79.0
Hunyuan-TurboS
- 78.9 Grok-3 mini
- 78.5
Cogito v1 Preview 70B
- 78.4
ERNIE-4.5-300B-A47B
- 78.3
Nemotron 3 Nano 30B-A3B
- 78.0 Claude 3.5 Sonnet (October 2024)
- 78.0
Mistral Small 4
- 77.6
Gemini 2.0 Flash
- 77.6
DiffusionGemma 26B A4B
- 77.6 Granite 4.2 30B
- 77.3
Nemotron 3 Nano Omni 30B-A3B
- 77.2
Mistral Medium 3
- 77.2
Gemma 4 12B
- 77.1
Nemotron Nano 12B v2 VL
- 77.0
Solar Pro 3
- 76.5
Solar Pro 2
- 76.4
Gemini 2.0 Flash
- 76.3
Nemotron 3.5 Super 120B-A12B Base
- 76.0
STEP3-VL-10B
- 76.0
Phi-4-Reasoning-plus
- 75.9
DeepSeek-V3
- 75.7
MiniMax-Text-01
- 75.5 Grok-2
- 75.3
Trinity Large Preview
- 74.3
Llama 4 Scout
- 74.3
Phi-4-reasoning
- 74.1
NVIDIA Nemotron 3.5 Transcribe 30B-A4B
- 74.1
DeepSeek V4.1 Flash
- 74.1
DeepSeek-V4.1-Flash Base
- 74.0
EXAONE Deep 32B
- 73.3
Llama 3.1-405B
- 72.6 Beam 1.0 Base
- 72.1
Falcon H1R 7B
- 71.6
Gemini 2.0 Flash-Lite
- 71.2
Qwen2.5-VL-72B
- 71.1
Qwen2.5-72B
- 70.4
Phi-4
- 69.6
Cohere Command A
- 69.4
Gemma 4 E4B
- 69.1
Mistral Small 3.2
- 69.0
Qwen2.5-32B
- 69.0 Mercury
- 68.9
Llama 3.3 70B Instruct
- 68.5 Kimi-VL-A3B-Thinking-2506
- 67.5
Gemma 3 27B
- 67.5
Nemotron 3.5 Lightning 30B-A3B Base
- 66.8
Mistral Small 3.1
- 66.5
Qwen3.5-2B
- 66.4
Llama 3.1-70B
- 66.3
Mistral Small 3
- 65.0
Nemotron 3 Nano 30B-A3B Base
- 65.0 Claude 3.5 Haiku
- 65.0 Reka Flash 3
- 64.5
Qwen-Turbo
- 64.4
Qwen2-72B
- 64.1 Granite 4.1 30B
- 63.7
Qwen2.5-14B
- 63.3
Qwen2.5-14B-Instruct-1M
- 60.6
Gemma 3 12B
- 60.2
Code World Model (CWM)
- 60.0
Gemma 4 E2B
- 58.7
Falcon-H1-34B
- 58.6 MiMo-7B-RL
- 57.6
InternLM3-8B-Instruct
- 55.5 Granite 4.0 H Small
- 54.3
Phi-3.5-MoE
- 52.8
Phi-4 Mini
- 52.7
OLMo 2 32B Instruct
- 50.6
Gemma 3n E4B
- 49.6 LFM2.5-1.2B-Thinking
- 48.3
Llama 3.1-8B
- 47.5
Phi-3.5-mini
- 47.0
Qwen2.5-Omni-7B
- 45.0
Qwen2.5-7B
- 44.4 LFM2.5-1.2B-Instruct
- 43.6
Gemma 3 4B
- 42.4 Moonlight-16B-A3B
- 42.3
Qwen3.5-0.8B
- 37.4 LFM2-8B-A1B
- 34.0 granite-4.0-1b
- 30.7
North Micro Vision Instruct
- 19.7 LFM2-1.2B
- 14.7
Gemma 3 1B
- 1
Intern-S2-397B 89.8%
- 2
Qwen3.7-Max 89.6%
- 3
Qwen 3.6 Plus 88.5%
- 4
Qwen3.7-Plus 88.5%
- 5
MiniMax-M2.1 88%
- 6
Qwen3.5 397B-A17B 87.8%
- 7
DeepSeek-V4-Pro 87.5%
- 8
Hunyuan-T1 87.2%
- 9 Kimi K2.5 87.1%
- 10
Seed-Thinking-v1.5 87%
- 11
Seed2.0 Pro 87%
- 12
Nemotron 3 Ultra 86.8%
- 13
Qwen3.5-122B-A10B 86.7%
- 14
Intern-S1-Pro 86.6%
- 15
Solar Pro 4 86.3%
- 16
Qwen3.6 27B 86.2%
- 17
DeepSeek-V4-Flash 86.2%
- 18
Solar Open 2 86.2%
- 19
Qwen3.5-27B 86.1%
- 20
Qwen3.5-Omni-Plus 85.9%
- 21
Qwen3-Max-2026-01-23 85.7%
- 22
Cogito v2 Preview 671B MoE 85.4%
- 23
Qwen3.5-35B-A3B 85.3%
- 24
Gemma 4 31B IT 85.2%
- 25
Qwen 3.6 35B-A3B 85.2%
- 26
MiniMax-M2.5 85.2%
- 27
DeepSeek-R1 May 2025 85%
- 28
DeepSeek-V3.1-Terminus 85%
- 29
DeepSeek-V3.2-Exp 85%
- 30
DeepSeek-V3.2 85%
- 31
MAI-Thinking-1 85%
- 32 MiMo-V2-Flash 84.9%
- 33
Seed1.8 84.9%
- 34
DeepSeek-V3.1 84.8%
- 35
Cogito v2.1 84.7%
- 36 GLM-4.5 84.6%
- 37 Kimi K2 Thinking 84.6%
- 38
Qwen3-235B-A22B-Thinking Jul 2025 84.4%
- 39
Step 3.5 Flash 84.4%
- 40 GLM-4.7 84.3%
- 41
ERNIE 5.1 84.3%
- 42
DeepSeek-R1 84%
- 43
Qwen3-VL-235B-A22B 83.8%
- 44
K-EXAONE 83.8%
- 45
ERNIE 5.0 83.8%
- 46
Nemotron 3 Super 83.7%
- 47
Intern-S1 83.5%
- 48
K-EXAONE 2.0 83.5%
- 49
MiniCPM-V-4.7-35B-A3B 83.5%
- 50
Trinity Large Thinking 83.4%
- 51
EXAONE 4.5 33B 83.3%
- 52
LongCat-Flash-Omni 82.7%
- 53
seed-oss-36b-instruct 82.7%
- 54
Qwen3-Next-80B-A3B 82.7%
- 55
LongCat-Flash-Chat 82.7%
- 56
LongCat-Flash-Thinking 82.6%
- 57
Gemma 4 26B A4B 82.6%
- 58
Qwen3.5-9B 82.5%
- 59
Llama 4 Behemoth 82.2%
- 60
MiniMax-M2 82%
- 61
Nemotron 3.5 Lightning 30B-A3B 81.9%
- 62 INTELLECT-3 81.9%
- 63
EXAONE 4.0 32B 81.8%
- 64
MiniMax-M2.7 81.8%
- 65 Sarvam 105B 81.7%
- 66
Nova 2.0 Pro Preview 81.6%
- 67 GLM-4.5-Air 81.4%
- 68
InternVL3.5-241B-A28B 81.3%
- 69
Solar Pro 3 (March 2026 update) 81.2%
- 70
DeepSeek-V3 Mar 2025 81.2%
- 71
MiniMax-M1-80k 81.1%
- 72 Kimi K2 Jul 2025 81.1%
- 73
Nova 2 Lite 80.9%
- 74
Nova 2 Omni Preview 80.7%
- 75 Hermes 4.3 36B 80.7%
- 76 Hermes 4 405B 80.6%
- 77
Ring-1T 80.5%
- 78
Qwen3-Coder-Next 80.5%
- 79
Llama 4 Maverick 80.5%
- 80
Solar Open 100B 80.4%
- 81 Sarvam 30B 80%
- 82 Grok 3 79.9%
- 83
Nemotron-Cascade 2 30B-A3B 79.8%
- 84
Llama-3.3-Nemotron-Super-49B-v1.5 79.5%
- 85
Gemini 2.0 Pro 79.1%
- 86
Qwen3.5-4B 79.1%
- 87
Nemotron 3 Ultra 550B-A55B Base 79.1%
- 88
Hunyuan-TurboS 79%
- 89 Grok-3 mini 78.9%
- 90
Cogito v1 Preview 70B 78.5%
- 91
ERNIE-4.5-300B-A47B 78.4%
- 92
Nemotron 3 Nano 30B-A3B 78.3%
- 93 Claude 3.5 Sonnet (October 2024) 78%
- 94
Mistral Small 4 78%
- 95
Gemini 2.0 Flash Feb 2025 77.6%
- 96
DiffusionGemma 26B A4B 77.6%
- 97 Granite 4.2 30B 77.6%
- 98
Nemotron 3 Nano Omni 30B-A3B 77.3%
- 99
Mistral Medium 3 77.2%
- 100
Gemma 4 12B 77.2%
- 101
Nemotron Nano 12B v2 VL 77.1%
- 102
Solar Pro 3 77%
- 103
Solar Pro 2 76.5%
- 104
Gemini 2.0 Flash Dec 2024 76.4%
- 105
Nemotron 3.5 Super 120B-A12B Base 76.3%
- 106
STEP3-VL-10B 76%
- 107
Phi-4-Reasoning-plus 76%
- 108
DeepSeek-V3 75.9%
- 109
MiniMax-Text-01 75.7%
- 110 Grok-2 Aug 2024 75.5%
- 111
Trinity Large Preview 75.3%
- 112
Llama 4 Scout 74.3%
- 113
Phi-4-reasoning 74.3%
- 114
NVIDIA Nemotron 3.5 Transcribe 30B-A4B 74.1%
- 115
DeepSeek V4.1 Flash 74.1%
- 116
DeepSeek-V4.1-Flash Base 74.1%
- 117
EXAONE Deep 32B 74%
- 118
Llama 3.1-405B 73.3%
- 119 Beam 1.0 Base 72.6%
- 120
Falcon H1R 7B 72.1%
- 121
Gemini 2.0 Flash-Lite 71.6%
- 122
Qwen2.5-VL-72B 71.2%
- 123
Qwen2.5-72B 71.1%
- 124
Phi-4 70.4%
- 125
Cohere Command A 69.6%
- 126
Gemma 4 E4B 69.4%
- 127
Mistral Small 3.2 69.1%
- 128
Qwen2.5-32B 69%
- 129 Mercury 69%
- 130
Llama 3.3 70B Instruct 68.9%
- 131 Kimi-VL-A3B-Thinking-2506 68.5%
- 132
Gemma 3 27B 67.5%
- 133
Nemotron 3.5 Lightning 30B-A3B Base 67.5%
- 134
Mistral Small 3.1 66.8%
- 135
Qwen3.5-2B 66.5%
- 136
Llama 3.1-70B 66.4%
- 137
Mistral Small 3 66.3%
- 138
Nemotron 3 Nano 30B-A3B Base 65%
- 139 Claude 3.5 Haiku 65%
- 140 Reka Flash 3 65%
- 141
Qwen-Turbo 64.5%
- 142
Qwen2-72B 64.4%
- 143 Granite 4.1 30B 64.1%
- 144
Qwen2.5-14B 63.7%
- 145
Qwen2.5-14B-Instruct-1M 63.3%
- 146
Gemma 3 12B 60.6%
- 147
Code World Model (CWM) 60.2%
- 148
Gemma 4 E2B 60%
- 149
Falcon-H1-34B 58.7%
- 150 MiMo-7B-RL 58.6%
- 151
InternLM3-8B-Instruct 57.6%
- 152 Granite 4.0 H Small 55.5%
- 153
Phi-3.5-MoE 54.3%
- 154
Phi-4 Mini 52.8%
- 155
OLMo 2 32B Instruct 52.7%
- 156
Gemma 3n E4B 50.6%
- 157 LFM2.5-1.2B-Thinking 49.6%
- 158
Llama 3.1-8B 48.3%
- 159
Phi-3.5-mini 47.5%
- 160
Qwen2.5-Omni-7B 47%
- 161
Qwen2.5-7B 45%
- 162 LFM2.5-1.2B-Instruct 44.4%
- 163
Gemma 3 4B 43.6%
- 164 Moonlight-16B-A3B 42.4%
- 165
Qwen3.5-0.8B 42.3%
- 166 LFM2-8B-A1B 37.4%
- 167 granite-4.0-1b 34%
- 168
North Micro Vision Instruct 30.7%
- 169 LFM2-1.2B 19.7%
- 170
Gemma 3 1B 14.7%
Source: Labs’ model cards
- 91.2
Gemini 3.1 Pro
- 90.1
Gemini 3 Pro
- 89.3 o1
- 89.1 Claude Opus 4.6
- 88.6
Gemini 3 Flash
- 88.0
MiniMax-M2.1
- 87.8
Qwen3.5 397B-A17B
- 87.7
Seed2.0 Lite (0215)
- 87.5 GPT-5.4
- 87.4 Claude Sonnet 4.5
- 87.4 GPT-5.2
- 87.3 Claude Opus 4
- 87.3 Claude Opus 4.5
- 87.3 Claude Sonnet 4.6
- 87.2
Hunyuan-T1
- 87.1 GPT-5
- 87.0
Seed-Thinking-v1.5
- 87.0 Grok 4
- 87.0
Seed2.0 Pro
- 86.7
Qwen3.5-122B-A10B
- 86.6
Seed1.6-Thinking
- 86.4 GPT-5.1
- 86.2
Gemini 3.1 Flash-Lite
- 86.1 GPT-4.5
- 86.1
Qwen3.5-27B
- 86.0
Gemini 2.5 Pro
- 86.0 GLM-5
- 85.7
Qwen3-Max
- 85.3
Qwen3.5-35B-A3B
- 85.0 o3
- 85.0
DeepSeek-V3.2
- 84.8
DeepSeek-V3.1
- 84.6 GLM-4.5
- 84.5
Qwen3-235B-A22B-Thinking
- 84.2 Grok 4.1 Fast
- 84.0 Claude 3.7 Sonnet
- 84.0
DeepSeek-R1
- 83.7 Claude Sonnet 4
- 83.6
Seed2.0 Mini
- 83.5
Intern-S1
- 83.0 Grok-3 mini
- 83.0
Qwen3-235B-A22B-Instruct
- 82.8
Llama 4 Behemoth
- 82.7
seed-oss-36b-instruct
- 82.7
LongCat-Flash-Chat
- 82.5
Qwen3.5-9B
- 82.0
MiniMax-M2
- 81.8 GPT-4.1
- 81.4 GLM-4.5-Air
- 81.3
DeepSeek-V3
- 81.1
MiniMax-M1-80k
- 81.0 Kimi K2
- 80.9
Qwen3-30B-A3B-Thinking
- 80.8 gpt-oss-120b
- 80.5
Llama 4 Maverick
- 80.3 o1-mini
- 80.1
Doubao-1.5-pro
- 80.1
MiniMax-M2.5
- 79.9 Grok 3
- 79.4 o3-mini
- 79.1
Gemini 2.0 Pro
- 79.1
Qwen3.5-4B
- 79.0
Hunyuan-TurboS
- 78.4
ERNIE-4.5-300B-A47B
- 78.3
Nemotron 3 Nano 30B-A3B
- 78.0 Claude 3.5 Sonnet (October 2024)
- 77.9 GPT-4o
- 77.6
Gemini 2.0 Flash
- 76.1 Claude 3.5 Sonnet
- 76.1
Qwen2.5-Max
- 76.0
Phi-4-Reasoning-plus
- 75.7
MiniMax-Text-01
- 75.5 Grok-2
- 74.7 GPT-4o
- 74.3
Llama 4 Scout
- 74.3
Phi-4-reasoning
- 73.6 gpt-oss-20b
- 73.3
Llama 3.1-405B
- 73.1
Athene-V2-Chat
- 72.5 GPT-4o
- 71.8 Grok-2 mini
- 71.6
Gemini 2.0 Flash-Lite
- 71.6
Qwen2.5-72B
- 71.0
QwQ-32B-Preview
- 70.4
Phi-4
- 70.3
Gemini 1.5 Pro
- 69.5
ERNIE-4.5-300B-A47B-Base
- 69.2
Qwen2.5-32B
- 69.1
QwQ-32B
- 68.5 Claude 3 Opus
- 68.2
Qwen3-235B-A22B
- 67.9
Mistral Large 2
- 67.5
Gemma 3 27B
- 67.3
Hunyuan-A13B-Instruct
- 66.8
Mistral Small 3.1
- 65.9
Llama 3.3 70B Instruct
- 65.9
Mistral Large 2
- 65.8
DeepSeek-V2.5
- 65.1
Seed-OSS-36B-Base
- 65.1
Nemotron 3 Nano 30B-A3B Base
- 64.4
Qwen2-72B
- 64.1
Gemini 1.5 Flash
- 63.7 GPT-4 Turbo
- 63.7
Qwen2.5-14B
- 63.6
DeepSeek-Coder-V2 236B
- 63.1 GPT-4o mini
- 62.8
Llama 3.1-70B
- 62.8
Llama-3.1-Nemotron-70B-Instruct
- 62.4
Yi-Lightning
- 62.1 Claude 3.5 Haiku
- 61.7
Qwen3-30B-A3B-Base
- 60.6
Gemma 3 12B
- 60.5
Nemotron-H-56B-Base
- 60.4
Seed-OSS-36B-Base-woSyn
- 58.9
EXAONE-3.5-32B-Instruct
- 58.6 MiMo-7B-RL
- 58.1
Yi-Large
- 57.6
InternLM3-8B-Instruct
- 56.8 Claude 3 Sonnet
- 56.7
ERNIE-4.5-21B-A3B-Base
- 56.5
Gemma 2 27B
- 56.3
Mixtral 8x22B
- 56.2
Llama 3-70B
- 55.7
phi-3-medium 14B
- 55.6
Qwen2.5-Turbo
- 55.3
Qwen3.5-2B
- 54.8
DeepSeek-V2 (MoE-236B, May 2024)
- 52.8
Phi-4 Mini
- 52.6
Qwen1.5-72B
- 52.3
Yi-1.5-34B
- 52.1
Gemma 2 9B
- 51.9
Phi-3-medium-128k-instruct
- 50.4
MAmmoTH2-8x7B-Plus
- 49.9
Qwen1.5-110B
- 49.5 Jamba 1.5 Large
- 48.4
Mistral Small v24.09
- 48.0 GLM-4-9B-Chat
- 47.9 GLM-4-9B
- 47.9
Phi-3.5-mini
- 47.2
Qwen2-7B-Instruct
- 46.2
EXAONE-3.5-7.8B-Instruct
- 46.0
Yi-1.5-9B-Chat
- 45.7
phi-3-mini 3.8B
- 45.4
Aya Expanse 32B
- 45.0
Qwen2.5-7B
- 44.8
Mistral NeMo
- 44.3
Llama 3.1-8B
- 44.0
Nemotron-H-8B-Base
- 43.9
Phi-3-mini-128k-instruct
- 43.7
Qwen2.5-3B
- 43.6
Gemma 3 4B
- 43.4
MAmmoTH2-8B-Plus
- 43.3
Mixtral 8x7B
- 43.0
Yi-34B
- 42.3 Claude 3 Haiku
- 42.0
Mathstral-7B-v0.1
- 41.9 MiMo-7B-Base
- 41.6 DeepSeek-Coder-V2-Lite-Instruct
- 41.0 Granite-3.1-8B-Instruct
- 41.0
Llama 3-8B
- 40.9
MAmmoTH2-7B-Plus
- 40.7
Qwen2-7B
- 39.8
Mistral-Nemo-Base-2407
- 39.2
WizardLM-2 8x22B
- 39.1
EXAONE-3.5-2.4B-Instruct
- 38.2
Yi-1.5-6B-Chat
- 38.0
Qwen1.5-14B-Chat
- 37.9
Ministral 8B
- 37.9
Command R (03-2024)
- 37.5
Llama 2-70B
- 35.3
DeepSeekMath-7B-Instruct
- 34.4 DeepSeek-Coder-V2-Lite-Base
- 33.7
Aya Expanse 8B
- 33.7
Gemma 7B
- 33.1 Granite-3.1-8B-Base
- 33.0
Zephyr 7B Beta
- 32.1
Qwen2.5-1.5B
- 32.0 Granite-3.1-2B-Instruct
- 31.0 Granite-3.0-8B-Base
- 30.9
Mistral 7B v0.1
- 30.8
Mistral-7B-Instruct-v0.2
- 30.4
Mistral 7B v0.2
- 29.7
Qwen3.5-0.8B
- 29.1
Qwen1.5-7B-Chat
- 28.8
Yi-6B-Chat
- 26.5
Yi 6B
- 25.8
Mistral-7B-Instruct-v0.1
- 25.4 Granite-3.1-3B-A800M-Instruct
- 25.3
Llama 2-13B
- 23.9 Granite-3.1-2B-Base
- 23.4 Llemma 7B
- 22.6
Qwen2-1.5B-Instruct
- 22.6
Qwen2-1.5B
- 22.2
Llama 3.2 3B
- 21.7 Granite-3.0-2B-Base
- 20.4 Granite-3.1-3B-A800M-Base
- 20.3
Llama 2-7B
- 18.3
SmolLM2-1.7B
- 15.9
Qwen2-0.5B-Instruct
- 15.8
Gemma 2B
- 15.6
Gemma 2 2B IT
- 15.0
Qwen2-0.5B
- 14.9
Qwen2.5-0.5B
- 14.7
Gemma 3 1B
- 13.3 Granite-3.1-1B-A400M-Instruct
- 12.3 Granite-3.1-1B-A400M-Base
- 11.9
Llama 3.2 1B
- 11.9
SmolLM-1.7B
- 11.4
SmolLM2-360M
- 11.2
SmolLM-135M
- 10.9
SmolLM-360M
- 10.8
SmolLM2-135M
- 1
Gemini 3.1 Pro 91.2%
- 2
Gemini 3 Pro 90.1%
- 3 o1 89.3%
- 4 Claude Opus 4.6 thinking 89.1%
- 5
Gemini 3 Flash 88.6%
- 6
MiniMax-M2.1 88%
- 7
Qwen3.5 397B-A17B 87.8%
- 8
Seed2.0 Lite (0215) 87.7%
- 9 GPT-5.4 87.5%
- 10 Claude Sonnet 4.5 thinking 87.4%
- 11 GPT-5.2 87.4%
- 12 Claude Opus 4 thinking 87.3%
- 13 Claude Opus 4.5 thinking 87.3%
- 14 Claude Sonnet 4.6 thinking 87.3%
- 15
Hunyuan-T1 87.2%
- 16 GPT-5 high 87.1%
- 17
Seed-Thinking-v1.5 87%
- 18 Grok 4 87%
- 19
Seed2.0 Pro 87%
- 20
Qwen3.5-122B-A10B 86.7%
- 21
Seed1.6-Thinking 86.6%
- 22 GPT-5.1 86.4%
- 23
Gemini 3.1 Flash-Lite 86.2%
- 24 GPT-4.5 86.1%
- 25
Qwen3.5-27B 86.1%
- 26
Gemini 2.5 Pro Jun 2025 86%
- 27 GLM-5 86%
- 28
Qwen3-Max 85.7%
- 29
Qwen3.5-35B-A3B 85.3%
- 30 o3 high 85%
- 31
DeepSeek-V3.2 85%
- 32
DeepSeek-V3.1 84.8%
- 33 GLM-4.5 84.6%
- 34
Qwen3-235B-A22B-Thinking Jul 2025 84.5%
- 35 Grok 4.1 Fast reasoning 84.2%
- 36 Claude 3.7 Sonnet thinking 84%
- 37
DeepSeek-R1 May 2025 84%
- 38 Claude Sonnet 4 83.7%
- 39
Seed2.0 Mini 83.6%
- 40
Intern-S1 83.5%
- 41 Grok-3 mini 83%
- 42
Qwen3-235B-A22B-Instruct Jul 2025 83%
- 43
Llama 4 Behemoth 82.8%
- 44
seed-oss-36b-instruct 82.7%
- 45
LongCat-Flash-Chat 82.7%
- 46
Qwen3.5-9B 82.5%
- 47
MiniMax-M2 82%
- 48 GPT-4.1 81.8%
- 49 GLM-4.5-Air 81.4%
- 50
DeepSeek-V3 Mar 2025 81.3%
- 51
MiniMax-M1-80k 81.1%
- 52 Kimi K2 Jul 2025 81%
- 53
Qwen3-30B-A3B-Thinking Jul 2025 80.9%
- 54 gpt-oss-120b high 80.8%
- 55
Llama 4 Maverick 80.5%
- 56 o1-mini 80.3%
- 57
Doubao-1.5-pro 80.1%
- 58
MiniMax-M2.5 80.1%
- 59 Grok 3 79.9%
- 60 o3-mini 79.4%
- 61
Gemini 2.0 Pro 79.1%
- 62
Qwen3.5-4B 79.1%
- 63
Hunyuan-TurboS 79%
- 64
ERNIE-4.5-300B-A47B 78.4%
- 65
Nemotron 3 Nano 30B-A3B bf16 78.3%
- 66 Claude 3.5 Sonnet (October 2024) 78%
- 67 GPT-4o Nov 2024 77.9%
- 68
Gemini 2.0 Flash Feb 2025 77.6%
- 69 Claude 3.5 Sonnet 76.1%
- 70
Qwen2.5-Max 76.1%
- 71
Phi-4-Reasoning-plus 76%
- 72
MiniMax-Text-01 75.7%
- 73 Grok-2 Dec 2024 75.5%
- 74 GPT-4o Aug 2024 74.7%
- 75
Llama 4 Scout 74.3%
- 76
Phi-4-reasoning 74.3%
- 77 gpt-oss-20b high 73.6%
- 78
Llama 3.1-405B 73.3%
- 79
Athene-V2-Chat 73.1%
- 80 GPT-4o May 2024 72.5%
- 81 Grok-2 mini 71.8%
- 82
Gemini 2.0 Flash-Lite 71.6%
- 83
Qwen2.5-72B 71.6%
- 84
QwQ-32B-Preview 71%
- 85
Phi-4 70.4%
- 86
Gemini 1.5 Pro Sep 2024 70.3%
- 87
ERNIE-4.5-300B-A47B-Base 69.5%
- 88
Qwen2.5-32B 69.2%
- 89
QwQ-32B 69.1%
- 90 Claude 3 Opus 68.5%
- 91
Qwen3-235B-A22B 68.2%
- 92
Mistral Large 2 Nov 2024 67.9%
- 93
Gemma 3 27B 67.5%
- 94
Hunyuan-A13B-Instruct 67.3%
- 95
Mistral Small 3.1 66.8%
- 96
Llama 3.3 70B Instruct 65.9%
- 97
Mistral Large 2 Jul 2024 65.9%
- 98
DeepSeek-V2.5 Sep 2024 65.8%
- 99
Seed-OSS-36B-Base 65.1%
- 100
Nemotron 3 Nano 30B-A3B Base 65.1%
- 101
Qwen2-72B 64.4%
- 102
Gemini 1.5 Flash Sep 2024 64.1%
- 103 GPT-4 Turbo Apr 2024 63.7%
- 104
Qwen2.5-14B 63.7%
- 105
DeepSeek-Coder-V2 236B 63.6%
- 106 GPT-4o mini 63.1%
- 107
Llama 3.1-70B 62.8%
- 108
Llama-3.1-Nemotron-70B-Instruct 62.8%
- 109
Yi-Lightning 62.4%
- 110 Claude 3.5 Haiku 62.1%
- 111
Qwen3-30B-A3B-Base 61.7%
- 112
Gemma 3 12B 60.6%
- 113
Nemotron-H-56B-Base 60.5%
- 114
Seed-OSS-36B-Base-woSyn 60.4%
- 115
EXAONE-3.5-32B-Instruct 58.9%
- 116 MiMo-7B-RL 58.6%
- 117
Yi-Large 58.1%
- 118
InternLM3-8B-Instruct 57.6%
- 119 Claude 3 Sonnet 56.8%
- 120
ERNIE-4.5-21B-A3B-Base 56.7%
- 121
Gemma 2 27B 56.5%
- 122
Mixtral 8x22B 56.3%
- 123
Llama 3-70B 56.2%
- 124
phi-3-medium 14B 55.7%
- 125
Qwen2.5-Turbo 55.6%
- 126
Qwen3.5-2B 55.3%
- 127
DeepSeek-V2 (MoE-236B, May 2024) 54.8%
- 128
Phi-4 Mini 52.8%
- 129
Qwen1.5-72B 52.6%
- 130
Yi-1.5-34B 52.3%
- 131
Gemma 2 9B 52.1%
- 132
Phi-3-medium-128k-instruct 51.9%
- 133
MAmmoTH2-8x7B-Plus 50.4%
- 134
Qwen1.5-110B 49.9%
- 135 Jamba 1.5 Large 49.5%
- 136
Mistral Small v24.09 48.4%
- 137 GLM-4-9B-Chat 48%
- 138 GLM-4-9B 47.9%
- 139
Phi-3.5-mini 47.9%
- 140
Qwen2-7B-Instruct 47.2%
- 141
EXAONE-3.5-7.8B-Instruct 46.2%
- 142
Yi-1.5-9B-Chat 46%
- 143
phi-3-mini 3.8B 45.7%
- 144
Aya Expanse 32B 45.4%
- 145
Qwen2.5-7B 45%
- 146
Mistral NeMo 44.8%
- 147
Llama 3.1-8B 44.3%
- 148
Nemotron-H-8B-Base 44%
- 149
Phi-3-mini-128k-instruct 43.9%
- 150
Qwen2.5-3B 43.7%
- 151
Gemma 3 4B 43.6%
- 152
MAmmoTH2-8B-Plus 43.4%
- 153
Mixtral 8x7B 43.3%
- 154
Yi-34B 43%
- 155 Claude 3 Haiku 42.3%
- 156
Mathstral-7B-v0.1 42%
- 157 MiMo-7B-Base 41.9%
- 158 DeepSeek-Coder-V2-Lite-Instruct 41.6%
- 159 Granite-3.1-8B-Instruct 41%
- 160
Llama 3-8B 41%
- 161
MAmmoTH2-7B-Plus 40.9%
- 162
Qwen2-7B 40.7%
- 163
Mistral-Nemo-Base-2407 39.8%
- 164
WizardLM-2 8x22B 39.2%
- 165
EXAONE-3.5-2.4B-Instruct 39.1%
- 166
Yi-1.5-6B-Chat 38.2%
- 167
Qwen1.5-14B-Chat 38%
- 168
Ministral 8B 37.9%
- 169
Command R (03-2024) 37.9%
- 170
Llama 2-70B 37.5%
- 171
DeepSeekMath-7B-Instruct 35.3%
- 172 DeepSeek-Coder-V2-Lite-Base 34.4%
- 173
Aya Expanse 8B 33.7%
- 174
Gemma 7B 33.7%
- 175 Granite-3.1-8B-Base 33.1%
- 176
Zephyr 7B Beta 33%
- 177
Qwen2.5-1.5B 32.1%
- 178 Granite-3.1-2B-Instruct 32%
- 179 Granite-3.0-8B-Base 31%
- 180
Mistral 7B v0.1 30.9%
- 181
Mistral-7B-Instruct-v0.2 30.8%
- 182
Mistral 7B v0.2 30.4%
- 183
Qwen3.5-0.8B 29.7%
- 184
Qwen1.5-7B-Chat 29.1%
- 185
Yi-6B-Chat 28.8%
- 186
Yi 6B 26.5%
- 187
Mistral-7B-Instruct-v0.1 25.8%
- 188 Granite-3.1-3B-A800M-Instruct 25.4%
- 189
Llama 2-13B 25.3%
- 190 Granite-3.1-2B-Base 23.9%
- 191 Llemma 7B 23.4%
- 192
Qwen2-1.5B-Instruct 22.6%
- 193
Qwen2-1.5B 22.6%
- 194
Llama 3.2 3B 22.2%
- 195 Granite-3.0-2B-Base 21.7%
- 196 Granite-3.1-3B-A800M-Base 20.4%
- 197
Llama 2-7B 20.3%
- 198
SmolLM2-1.7B 18.3%
- 199
Qwen2-0.5B-Instruct 15.9%
- 200
Gemma 2B 15.8%
- 201
Gemma 2 2B IT 15.6%
- 202
Qwen2-0.5B 15%
- 203
Qwen2.5-0.5B 14.9%
- 204
Gemma 3 1B 14.7%
- 205 Granite-3.1-1B-A400M-Instruct 13.3%
- 206 Granite-3.1-1B-A400M-Base 12.3%
- 207
Llama 3.2 1B 11.9%
- 208
SmolLM-1.7B 11.9%
- 209
SmolLM2-360M 11.4%
- 210
SmolLM-135M 11.2%
- 211
SmolLM-360M 10.9%
- 212
SmolLM2-135M 10.8%
Source: Official MMLU-Pro leaderboard
- 91.2
Gemini 3.1 Pro
- 90.1
Gemini 3 Pro
- 89.8
Intern-S2-397B
- 89.6
Qwen3.7-Max
- 89.3 o1
- 89.1 Claude Opus 4.6
- 88.6
Gemini 3 Flash
- 88.5
Qwen 3.6 Plus
- 88.5
Qwen3.7-Plus
- 88.0
MiniMax-M2.1
- 87.8
Qwen3.5 397B-A17B
- 87.7
Seed2.0 Lite (0215)
- 87.5 GPT-5.4
- 87.5
DeepSeek-V4-Pro
- 87.4 Claude Sonnet 4.5
- 87.4 GPT-5.2
- 87.3 Claude Opus 4
- 87.3 Claude Opus 4.5
- 87.3 Claude Sonnet 4.6
- 87.2
Hunyuan-T1
- 87.1 GPT-5
- 87.1 Kimi K2.5
- 87.0
Seed-Thinking-v1.5
- 87.0 Grok 4
- 87.0
Seed2.0 Pro
- 86.8
Nemotron 3 Ultra
- 86.7
Qwen3.5-122B-A10B
- 86.6
Seed1.6-Thinking
- 86.6
Intern-S1-Pro
- 86.4 GPT-5.1
- 86.3
Solar Pro 4
- 86.2
Gemini 3.1 Flash-Lite
- 86.2
Qwen3.6 27B
- 86.2
DeepSeek-V4-Flash
- 86.2
Solar Open 2
- 86.1 GPT-4.5
- 86.1
Qwen3.5-27B
- 86.0
Gemini 2.5 Pro
- 86.0 GLM-5
- 85.9
Qwen3.5-Omni-Plus
- 85.7
Qwen3-Max
- 85.7
Qwen3-Max-2026-01-23
- 85.4
Cogito v2 Preview 671B MoE
- 85.3
Qwen3.5-35B-A3B
- 85.2
Gemma 4 31B IT
- 85.2
Qwen 3.6 35B-A3B
- 85.2
MiniMax-M2.5
- 85.0 o3
- 85.0
DeepSeek-R1
- 85.0
DeepSeek-V3.1-Terminus
- 85.0
DeepSeek-V3.2-Exp
- 85.0
DeepSeek-V3.2
- 85.0
MAI-Thinking-1
- 84.9 MiMo-V2-Flash
- 84.9
Seed1.8
- 84.8
DeepSeek-V3.1
- 84.7
Cogito v2.1
- 84.6 GLM-4.5
- 84.6 Kimi K2 Thinking
- 84.5
Qwen3-235B-A22B-Thinking
- 84.4
Step 3.5 Flash
- 84.3 GLM-4.7
- 84.3
ERNIE 5.1
- 84.2 Grok 4.1 Fast
- 84.0
DeepSeek-R1
- 84.0 Claude 3.7 Sonnet
- 83.8
Qwen3-VL-235B-A22B
- 83.8
K-EXAONE
- 83.8
ERNIE 5.0
- 83.7
Nemotron 3 Super
- 83.7 Claude Sonnet 4
- 83.6
Seed2.0 Mini
- 83.5
Intern-S1
- 83.5
K-EXAONE 2.0
- 83.5
MiniCPM-V-4.7-35B-A3B
- 83.4
Trinity Large Thinking
- 83.3
EXAONE 4.5 33B
- 83.0 Grok-3 mini
- 83.0
Qwen3-235B-A22B-Instruct
- 82.8
Llama 4 Behemoth
- 82.7
LongCat-Flash-Omni
- 82.7
seed-oss-36b-instruct
- 82.7
LongCat-Flash-Chat
- 82.7
Qwen3-Next-80B-A3B
- 82.6
LongCat-Flash-Thinking
- 82.6
Gemma 4 26B A4B
- 82.5
Qwen3.5-9B
- 82.0
MiniMax-M2
- 81.9
Nemotron 3.5 Lightning 30B-A3B
- 81.9 INTELLECT-3
- 81.8 GPT-4.1
- 81.8
EXAONE 4.0 32B
- 81.8
MiniMax-M2.7
- 81.7 Sarvam 105B
- 81.6
Nova 2.0 Pro Preview
- 81.4 GLM-4.5-Air
- 81.3
DeepSeek-V3
- 81.3
InternVL3.5-241B-A28B
- 81.2
Solar Pro 3 (March 2026 update)
- 81.1
MiniMax-M1-80k
- 81.1 Kimi K2
- 80.9
Qwen3-30B-A3B-Thinking
- 80.9
Nova 2 Lite
- 80.8 gpt-oss-120b
- 80.7
Nova 2 Omni Preview
- 80.7 Hermes 4.3 36B
- 80.6 Hermes 4 405B
- 80.5
Ring-1T
- 80.5
Qwen3-Coder-Next
- 80.5
Llama 4 Maverick
- 80.4
Solar Open 100B
- 80.3 o1-mini
- 80.1
Doubao-1.5-pro
- 80.0 Sarvam 30B
- 79.9 Grok 3
- 79.8
Nemotron-Cascade 2 30B-A3B
- 79.5
Llama-3.3-Nemotron-Super-49B-v1.5
- 79.4 o3-mini
- 79.1
Gemini 2.0 Pro
- 79.1
Qwen3.5-4B
- 79.1
Nemotron 3 Ultra 550B-A55B Base
- 79.0
Hunyuan-TurboS
- 78.5
Cogito v1 Preview 70B
- 78.4
ERNIE-4.5-300B-A47B
- 78.3
Nemotron 3 Nano 30B-A3B
- 78.0 Claude 3.5 Sonnet (October 2024)
- 78.0
Mistral Small 4
- 77.9 GPT-4o
- 77.6
Gemini 2.0 Flash
- 77.6
DiffusionGemma 26B A4B
- 77.6 Granite 4.2 30B
- 77.3
Nemotron 3 Nano Omni 30B-A3B
- 77.2
Mistral Medium 3
- 77.2
Gemma 4 12B
- 77.1
Nemotron Nano 12B v2 VL
- 77.0
Solar Pro 3
- 76.5
Solar Pro 2
- 76.4
Gemini 2.0 Flash
- 76.3
Nemotron 3.5 Super 120B-A12B Base
- 76.1 Claude 3.5 Sonnet
- 76.1
Qwen2.5-Max
- 76.0
STEP3-VL-10B
- 76.0
Phi-4-Reasoning-plus
- 75.9
DeepSeek-V3
- 75.7
MiniMax-Text-01
- 75.5 Grok-2
- 75.5 Grok-2
- 75.3
Trinity Large Preview
- 74.7 GPT-4o
- 74.3
Llama 4 Scout
- 74.3
Phi-4-reasoning
- 74.1
NVIDIA Nemotron 3.5 Transcribe 30B-A4B
- 74.1
DeepSeek V4.1 Flash
- 74.1
DeepSeek-V4.1-Flash Base
- 74.0
EXAONE Deep 32B
- 73.6 gpt-oss-20b
- 73.3
Llama 3.1-405B
- 73.1
Athene-V2-Chat
- 72.6 Beam 1.0 Base
- 72.5 GPT-4o
- 72.1
Falcon H1R 7B
- 71.8 Grok-2 mini
- 71.6
Gemini 2.0 Flash-Lite
- 71.6
Qwen2.5-72B
- 71.2
Qwen2.5-VL-72B
- 71.0
QwQ-32B-Preview
- 70.4
Phi-4
- 70.3
Gemini 1.5 Pro
- 69.6
Cohere Command A
- 69.5
ERNIE-4.5-300B-A47B-Base
- 69.4
Gemma 4 E4B
- 69.2
Qwen2.5-32B
- 69.1
QwQ-32B
- 69.1
Mistral Small 3.2
- 69.0 Mercury
- 68.9
Llama 3.3 70B Instruct
- 68.5 Kimi-VL-A3B-Thinking-2506
- 68.5 Claude 3 Opus
- 68.2
Qwen3-235B-A22B
- 67.9
Mistral Large 2
- 67.5
Gemma 3 27B
- 67.5
Nemotron 3.5 Lightning 30B-A3B Base
- 67.3
Hunyuan-A13B-Instruct
- 66.8
Mistral Small 3.1
- 66.5
Qwen3.5-2B
- 66.4
Llama 3.1-70B
- 66.3
Mistral Small 3
- 65.9
Mistral Large 2
- 65.8
DeepSeek-V2.5
- 65.1
Seed-OSS-36B-Base
- 65.1
Nemotron 3 Nano 30B-A3B Base
- 65.0 Claude 3.5 Haiku
- 65.0 Reka Flash 3
- 64.5
Qwen-Turbo
- 64.4
Qwen2-72B
- 64.1
Gemini 1.5 Flash
- 64.1 Granite 4.1 30B
- 63.7 GPT-4 Turbo
- 63.7
Qwen2.5-14B
- 63.6
DeepSeek-Coder-V2 236B
- 63.3
Qwen2.5-14B-Instruct-1M
- 63.1 GPT-4o mini
- 62.8
Llama-3.1-Nemotron-70B-Instruct
- 62.4
Yi-Lightning
- 61.7
Qwen3-30B-A3B-Base
- 60.6
Gemma 3 12B
- 60.5
Nemotron-H-56B-Base
- 60.4
Seed-OSS-36B-Base-woSyn
- 60.2
Code World Model (CWM)
- 60.0
Gemma 4 E2B
- 58.9
EXAONE-3.5-32B-Instruct
- 58.7
Falcon-H1-34B
- 58.6 MiMo-7B-RL
- 58.1
Yi-Large
- 57.6
InternLM3-8B-Instruct
- 56.8 Claude 3 Sonnet
- 56.7
ERNIE-4.5-21B-A3B-Base
- 56.5
Gemma 2 27B
- 56.3
Mixtral 8x22B
- 56.2
Llama 3-70B
- 55.7
phi-3-medium 14B
- 55.6
Qwen2.5-Turbo
- 55.5 Granite 4.0 H Small
- 54.8
DeepSeek-V2 (MoE-236B, May 2024)
- 54.3
Phi-3.5-MoE
- 52.8
Phi-4 Mini
- 52.7
OLMo 2 32B Instruct
- 52.6
Qwen1.5-72B
- 52.3
Yi-1.5-34B
- 52.1
Gemma 2 9B
- 51.9
Phi-3-medium-128k-instruct
- 50.6
Gemma 3n E4B
- 50.4
MAmmoTH2-8x7B-Plus
- 49.9
Qwen1.5-110B
- 49.6 LFM2.5-1.2B-Thinking
- 49.5 Jamba 1.5 Large
- 48.4
Mistral Small v24.09
- 48.3
Llama 3.1-8B
- 48.0 GLM-4-9B-Chat
- 47.9 GLM-4-9B
- 47.9
Phi-3.5-mini
- 47.2
Qwen2-7B-Instruct
- 47.0
Qwen2.5-Omni-7B
- 46.2
EXAONE-3.5-7.8B-Instruct
- 46.0
Yi-1.5-9B-Chat
- 45.7
phi-3-mini 3.8B
- 45.4
Aya Expanse 32B
- 45.0
Qwen2.5-7B
- 44.8
Mistral NeMo
- 44.4 LFM2.5-1.2B-Instruct
- 44.0
Nemotron-H-8B-Base
- 43.9
Phi-3-mini-128k-instruct
- 43.7
Qwen2.5-3B
- 43.6
Gemma 3 4B
- 43.4
MAmmoTH2-8B-Plus
- 43.3
Mixtral 8x7B
- 43.0
Yi-34B
- 42.4 Moonlight-16B-A3B
- 42.3
Qwen3.5-0.8B
- 42.3 Claude 3 Haiku
- 42.0
Mathstral-7B-v0.1
- 41.9 MiMo-7B-Base
- 41.6 DeepSeek-Coder-V2-Lite-Instruct
- 41.0 Granite-3.1-8B-Instruct
- 41.0
Llama 3-8B
- 40.9
MAmmoTH2-7B-Plus
- 40.7
Qwen2-7B
- 39.8
Mistral-Nemo-Base-2407
- 39.2
WizardLM-2 8x22B
- 39.1
EXAONE-3.5-2.4B-Instruct
- 38.2
Yi-1.5-6B-Chat
- 38.0
Qwen1.5-14B-Chat
- 37.9
Ministral 8B
- 37.9
Command R (03-2024)
- 37.5
Llama 2-70B
- 37.4 LFM2-8B-A1B
- 35.3
DeepSeekMath-7B-Instruct
- 34.4 DeepSeek-Coder-V2-Lite-Base
- 34.0 granite-4.0-1b
- 33.7
Aya Expanse 8B
- 33.7
Gemma 7B
- 33.1 Granite-3.1-8B-Base
- 33.0
Zephyr 7B Beta
- 32.1
Qwen2.5-1.5B
- 32.0 Granite-3.1-2B-Instruct
- 31.0 Granite-3.0-8B-Base
- 30.9
Mistral 7B v0.1
- 30.8
Mistral-7B-Instruct-v0.2
- 30.7
North Micro Vision Instruct
- 30.4
Mistral 7B v0.2
- 29.1
Qwen1.5-7B-Chat
- 28.8
Yi-6B-Chat
- 26.5
Yi 6B
- 25.8
Mistral-7B-Instruct-v0.1
- 25.4 Granite-3.1-3B-A800M-Instruct
- 25.3
Llama 2-13B
- 23.9 Granite-3.1-2B-Base
- 23.4 Llemma 7B
- 22.6
Qwen2-1.5B-Instruct
- 22.6
Qwen2-1.5B
- 22.2
Llama 3.2 3B
- 21.7 Granite-3.0-2B-Base
- 20.4 Granite-3.1-3B-A800M-Base
- 20.3
Llama 2-7B
- 19.7 LFM2-1.2B
- 18.3
SmolLM2-1.7B
- 15.9
Qwen2-0.5B-Instruct
- 15.8
Gemma 2B
- 15.6
Gemma 2 2B IT
- 15.0
Qwen2-0.5B
- 14.9
Qwen2.5-0.5B
- 14.7
Gemma 3 1B
- 13.3 Granite-3.1-1B-A400M-Instruct
- 12.3 Granite-3.1-1B-A400M-Base
- 11.9
Llama 3.2 1B
- 11.9
SmolLM-1.7B
- 11.4
SmolLM2-360M
- 11.2
SmolLM-135M
- 10.9
SmolLM-360M
- 10.8
SmolLM2-135M
- 1
Gemini 3.1 Pro 91.2%
- 2
Gemini 3 Pro 90.1%
- 3
Intern-S2-397B 89.8%
- 4
Qwen3.7-Max 89.6%
- 5 o1 89.3%
- 6 Claude Opus 4.6 thinking 89.1%
- 7
Gemini 3 Flash 88.6%
- 8
Qwen 3.6 Plus 88.5%
- 9
Qwen3.7-Plus 88.5%
- 10
MiniMax-M2.1 88%
- 11
Qwen3.5 397B-A17B 87.8%
- 12
Seed2.0 Lite (0215) 87.7%
- 13 GPT-5.4 87.5%
- 14
DeepSeek-V4-Pro 87.5%
- 15 Claude Sonnet 4.5 thinking 87.4%
- 16 GPT-5.2 87.4%
- 17 Claude Opus 4 thinking 87.3%
- 18 Claude Opus 4.5 thinking 87.3%
- 19 Claude Sonnet 4.6 thinking 87.3%
- 20
Hunyuan-T1 87.2%
- 21 GPT-5 high 87.1%
- 22 Kimi K2.5 87.1%
- 23
Seed-Thinking-v1.5 87%
- 24 Grok 4 87%
- 25
Seed2.0 Pro 87%
- 26
Nemotron 3 Ultra 86.8%
- 27
Qwen3.5-122B-A10B 86.7%
- 28
Seed1.6-Thinking 86.6%
- 29
Intern-S1-Pro 86.6%
- 30 GPT-5.1 86.4%
- 31
Solar Pro 4 86.3%
- 32
Gemini 3.1 Flash-Lite 86.2%
- 33
Qwen3.6 27B 86.2%
- 34
DeepSeek-V4-Flash 86.2%
- 35
Solar Open 2 86.2%
- 36 GPT-4.5 86.1%
- 37
Qwen3.5-27B 86.1%
- 38
Gemini 2.5 Pro Jun 2025 86%
- 39 GLM-5 86%
- 40
Qwen3.5-Omni-Plus 85.9%
- 41
Qwen3-Max 85.7%
- 42
Qwen3-Max-2026-01-23 85.7%
- 43
Cogito v2 Preview 671B MoE 85.4%
- 44
Qwen3.5-35B-A3B 85.3%
- 45
Gemma 4 31B IT 85.2%
- 46
Qwen 3.6 35B-A3B 85.2%
- 47
MiniMax-M2.5 85.2%
- 48 o3 high 85%
- 49
DeepSeek-R1 May 2025 85%
- 50
DeepSeek-V3.1-Terminus 85%
- 51
DeepSeek-V3.2-Exp 85%
- 52
DeepSeek-V3.2 85%
- 53
MAI-Thinking-1 85%
- 54 MiMo-V2-Flash 84.9%
- 55
Seed1.8 84.9%
- 56
DeepSeek-V3.1 84.8%
- 57
Cogito v2.1 84.7%
- 58 GLM-4.5 84.6%
- 59 Kimi K2 Thinking 84.6%
- 60
Qwen3-235B-A22B-Thinking Jul 2025 84.5%
- 61
Step 3.5 Flash 84.4%
- 62 GLM-4.7 84.3%
- 63
ERNIE 5.1 84.3%
- 64 Grok 4.1 Fast reasoning 84.2%
- 65
DeepSeek-R1 84%
- 66 Claude 3.7 Sonnet thinking 84%
- 67
Qwen3-VL-235B-A22B 83.8%
- 68
K-EXAONE 83.8%
- 69
ERNIE 5.0 83.8%
- 70
Nemotron 3 Super 83.7%
- 71 Claude Sonnet 4 83.7%
- 72
Seed2.0 Mini 83.6%
- 73
Intern-S1 83.5%
- 74
K-EXAONE 2.0 83.5%
- 75
MiniCPM-V-4.7-35B-A3B 83.5%
- 76
Trinity Large Thinking 83.4%
- 77
EXAONE 4.5 33B 83.3%
- 78 Grok-3 mini 83%
- 79
Qwen3-235B-A22B-Instruct Jul 2025 83%
- 80
Llama 4 Behemoth 82.8%
- 81
LongCat-Flash-Omni 82.7%
- 82
seed-oss-36b-instruct 82.7%
- 83
LongCat-Flash-Chat 82.7%
- 84
Qwen3-Next-80B-A3B 82.7%
- 85
LongCat-Flash-Thinking 82.6%
- 86
Gemma 4 26B A4B 82.6%
- 87
Qwen3.5-9B 82.5%
- 88
MiniMax-M2 82%
- 89
Nemotron 3.5 Lightning 30B-A3B 81.9%
- 90 INTELLECT-3 81.9%
- 91 GPT-4.1 81.8%
- 92
EXAONE 4.0 32B 81.8%
- 93
MiniMax-M2.7 81.8%
- 94 Sarvam 105B 81.7%
- 95
Nova 2.0 Pro Preview 81.6%
- 96 GLM-4.5-Air 81.4%
- 97
DeepSeek-V3 Mar 2025 81.3%
- 98
InternVL3.5-241B-A28B 81.3%
- 99
Solar Pro 3 (March 2026 update) 81.2%
- 100
MiniMax-M1-80k 81.1%
- 101 Kimi K2 Jul 2025 81.1%
- 102
Qwen3-30B-A3B-Thinking Jul 2025 80.9%
- 103
Nova 2 Lite 80.9%
- 104 gpt-oss-120b high 80.8%
- 105
Nova 2 Omni Preview 80.7%
- 106 Hermes 4.3 36B 80.7%
- 107 Hermes 4 405B 80.6%
- 108
Ring-1T 80.5%
- 109
Qwen3-Coder-Next 80.5%
- 110
Llama 4 Maverick 80.5%
- 111
Solar Open 100B 80.4%
- 112 o1-mini 80.3%
- 113
Doubao-1.5-pro 80.1%
- 114 Sarvam 30B 80%
- 115 Grok 3 79.9%
- 116
Nemotron-Cascade 2 30B-A3B 79.8%
- 117
Llama-3.3-Nemotron-Super-49B-v1.5 79.5%
- 118 o3-mini 79.4%
- 119
Gemini 2.0 Pro 79.1%
- 120
Qwen3.5-4B 79.1%
- 121
Nemotron 3 Ultra 550B-A55B Base 79.1%
- 122
Hunyuan-TurboS 79%
- 123
Cogito v1 Preview 70B 78.5%
- 124
ERNIE-4.5-300B-A47B 78.4%
- 125
Nemotron 3 Nano 30B-A3B bf16 78.3%
- 126 Claude 3.5 Sonnet (October 2024) 78%
- 127
Mistral Small 4 78%
- 128 GPT-4o Nov 2024 77.9%
- 129
Gemini 2.0 Flash Feb 2025 77.6%
- 130
DiffusionGemma 26B A4B 77.6%
- 131 Granite 4.2 30B 77.6%
- 132
Nemotron 3 Nano Omni 30B-A3B 77.3%
- 133
Mistral Medium 3 77.2%
- 134
Gemma 4 12B 77.2%
- 135
Nemotron Nano 12B v2 VL 77.1%
- 136
Solar Pro 3 77%
- 137
Solar Pro 2 76.5%
- 138
Gemini 2.0 Flash Dec 2024 76.4%
- 139
Nemotron 3.5 Super 120B-A12B Base 76.3%
- 140 Claude 3.5 Sonnet 76.1%
- 141
Qwen2.5-Max 76.1%
- 142
STEP3-VL-10B 76%
- 143
Phi-4-Reasoning-plus 76%
- 144
DeepSeek-V3 75.9%
- 145
MiniMax-Text-01 75.7%
- 146 Grok-2 Aug 2024 75.5%
- 147 Grok-2 Dec 2024 75.5%
- 148
Trinity Large Preview 75.3%
- 149 GPT-4o Aug 2024 74.7%
- 150
Llama 4 Scout 74.3%
- 151
Phi-4-reasoning 74.3%
- 152
NVIDIA Nemotron 3.5 Transcribe 30B-A4B 74.1%
- 153
DeepSeek V4.1 Flash 74.1%
- 154
DeepSeek-V4.1-Flash Base 74.1%
- 155
EXAONE Deep 32B 74%
- 156 gpt-oss-20b high 73.6%
- 157
Llama 3.1-405B 73.3%
- 158
Athene-V2-Chat 73.1%
- 159 Beam 1.0 Base 72.6%
- 160 GPT-4o May 2024 72.5%
- 161
Falcon H1R 7B 72.1%
- 162 Grok-2 mini 71.8%
- 163
Gemini 2.0 Flash-Lite 71.6%
- 164
Qwen2.5-72B 71.6%
- 165
Qwen2.5-VL-72B 71.2%
- 166
QwQ-32B-Preview 71%
- 167
Phi-4 70.4%
- 168
Gemini 1.5 Pro Sep 2024 70.3%
- 169
Cohere Command A 69.6%
- 170
ERNIE-4.5-300B-A47B-Base 69.5%
- 171
Gemma 4 E4B 69.4%
- 172
Qwen2.5-32B 69.2%
- 173
QwQ-32B 69.1%
- 174
Mistral Small 3.2 69.1%
- 175 Mercury 69%
- 176
Llama 3.3 70B Instruct 68.9%
- 177 Kimi-VL-A3B-Thinking-2506 68.5%
- 178 Claude 3 Opus 68.5%
- 179
Qwen3-235B-A22B 68.2%
- 180
Mistral Large 2 Nov 2024 67.9%
- 181
Gemma 3 27B 67.5%
- 182
Nemotron 3.5 Lightning 30B-A3B Base 67.5%
- 183
Hunyuan-A13B-Instruct 67.3%
- 184
Mistral Small 3.1 66.8%
- 185
Qwen3.5-2B 66.5%
- 186
Llama 3.1-70B 66.4%
- 187
Mistral Small 3 66.3%
- 188
Mistral Large 2 Jul 2024 65.9%
- 189
DeepSeek-V2.5 Sep 2024 65.8%
- 190
Seed-OSS-36B-Base 65.1%
- 191
Nemotron 3 Nano 30B-A3B Base 65.1%
- 192 Claude 3.5 Haiku 65%
- 193 Reka Flash 3 65%
- 194
Qwen-Turbo 64.5%
- 195
Qwen2-72B 64.4%
- 196
Gemini 1.5 Flash Sep 2024 64.1%
- 197 Granite 4.1 30B 64.1%
- 198 GPT-4 Turbo Apr 2024 63.7%
- 199
Qwen2.5-14B 63.7%
- 200
DeepSeek-Coder-V2 236B 63.6%
- 201
Qwen2.5-14B-Instruct-1M 63.3%
- 202 GPT-4o mini 63.1%
- 203
Llama-3.1-Nemotron-70B-Instruct 62.8%
- 204
Yi-Lightning 62.4%
- 205
Qwen3-30B-A3B-Base 61.7%
- 206
Gemma 3 12B 60.6%
- 207
Nemotron-H-56B-Base 60.5%
- 208
Seed-OSS-36B-Base-woSyn 60.4%
- 209
Code World Model (CWM) 60.2%
- 210
Gemma 4 E2B 60%
- 211
EXAONE-3.5-32B-Instruct 58.9%
- 212
Falcon-H1-34B 58.7%
- 213 MiMo-7B-RL 58.6%
- 214
Yi-Large 58.1%
- 215
InternLM3-8B-Instruct 57.6%
- 216 Claude 3 Sonnet 56.8%
- 217
ERNIE-4.5-21B-A3B-Base 56.7%
- 218
Gemma 2 27B 56.5%
- 219
Mixtral 8x22B 56.3%
- 220
Llama 3-70B 56.2%
- 221
phi-3-medium 14B 55.7%
- 222
Qwen2.5-Turbo 55.6%
- 223 Granite 4.0 H Small 55.5%
- 224
DeepSeek-V2 (MoE-236B, May 2024) 54.8%
- 225
Phi-3.5-MoE 54.3%
- 226
Phi-4 Mini 52.8%
- 227
OLMo 2 32B Instruct 52.7%
- 228
Qwen1.5-72B 52.6%
- 229
Yi-1.5-34B 52.3%
- 230
Gemma 2 9B 52.1%
- 231
Phi-3-medium-128k-instruct 51.9%
- 232
Gemma 3n E4B 50.6%
- 233
MAmmoTH2-8x7B-Plus 50.4%
- 234
Qwen1.5-110B 49.9%
- 235 LFM2.5-1.2B-Thinking 49.6%
- 236 Jamba 1.5 Large 49.5%
- 237
Mistral Small v24.09 48.4%
- 238
Llama 3.1-8B 48.3%
- 239 GLM-4-9B-Chat 48%
- 240 GLM-4-9B 47.9%
- 241
Phi-3.5-mini 47.9%
- 242
Qwen2-7B-Instruct 47.2%
- 243
Qwen2.5-Omni-7B 47%
- 244
EXAONE-3.5-7.8B-Instruct 46.2%
- 245
Yi-1.5-9B-Chat 46%
- 246
phi-3-mini 3.8B 45.7%
- 247
Aya Expanse 32B 45.4%
- 248
Qwen2.5-7B 45%
- 249
Mistral NeMo 44.8%
- 250 LFM2.5-1.2B-Instruct 44.4%
- 251
Nemotron-H-8B-Base 44%
- 252
Phi-3-mini-128k-instruct 43.9%
- 253
Qwen2.5-3B 43.7%
- 254
Gemma 3 4B 43.6%
- 255
MAmmoTH2-8B-Plus 43.4%
- 256
Mixtral 8x7B 43.3%
- 257
Yi-34B 43%
- 258 Moonlight-16B-A3B 42.4%
- 259
Qwen3.5-0.8B 42.3%
- 260 Claude 3 Haiku 42.3%
- 261
Mathstral-7B-v0.1 42%
- 262 MiMo-7B-Base 41.9%
- 263 DeepSeek-Coder-V2-Lite-Instruct 41.6%
- 264 Granite-3.1-8B-Instruct 41%
- 265
Llama 3-8B 41%
- 266
MAmmoTH2-7B-Plus 40.9%
- 267
Qwen2-7B 40.7%
- 268
Mistral-Nemo-Base-2407 39.8%
- 269
WizardLM-2 8x22B 39.2%
- 270
EXAONE-3.5-2.4B-Instruct 39.1%
- 271
Yi-1.5-6B-Chat 38.2%
- 272
Qwen1.5-14B-Chat 38%
- 273
Ministral 8B 37.9%
- 274
Command R (03-2024) 37.9%
- 275
Llama 2-70B 37.5%
- 276 LFM2-8B-A1B 37.4%
- 277
DeepSeekMath-7B-Instruct 35.3%
- 278 DeepSeek-Coder-V2-Lite-Base 34.4%
- 279 granite-4.0-1b 34%
- 280
Aya Expanse 8B 33.7%
- 281
Gemma 7B 33.7%
- 282 Granite-3.1-8B-Base 33.1%
- 283
Zephyr 7B Beta 33%
- 284
Qwen2.5-1.5B 32.1%
- 285 Granite-3.1-2B-Instruct 32%
- 286 Granite-3.0-8B-Base 31%
- 287
Mistral 7B v0.1 30.9%
- 288
Mistral-7B-Instruct-v0.2 30.8%
- 289
North Micro Vision Instruct 30.7%
- 290
Mistral 7B v0.2 30.4%
- 291
Qwen1.5-7B-Chat 29.1%
- 292
Yi-6B-Chat 28.8%
- 293
Yi 6B 26.5%
- 294
Mistral-7B-Instruct-v0.1 25.8%
- 295 Granite-3.1-3B-A800M-Instruct 25.4%
- 296
Llama 2-13B 25.3%
- 297 Granite-3.1-2B-Base 23.9%
- 298 Llemma 7B 23.4%
- 299
Qwen2-1.5B-Instruct 22.6%
- 300
Qwen2-1.5B 22.6%
- 301
Llama 3.2 3B 22.2%
- 302 Granite-3.0-2B-Base 21.7%
- 303 Granite-3.1-3B-A800M-Base 20.4%
- 304
Llama 2-7B 20.3%
- 305 LFM2-1.2B 19.7%
- 306
SmolLM2-1.7B 18.3%
- 307
Qwen2-0.5B-Instruct 15.9%
- 308
Gemma 2B 15.8%
- 309
Gemma 2 2B IT 15.6%
- 310
Qwen2-0.5B 15%
- 311
Qwen2.5-0.5B 14.9%
- 312
Gemma 3 1B 14.7%
- 313 Granite-3.1-1B-A400M-Instruct 13.3%
- 314 Granite-3.1-1B-A400M-Base 12.3%
- 315
Llama 3.2 1B 11.9%
- 316
SmolLM-1.7B 11.9%
- 317
SmolLM2-360M 11.4%
- 318
SmolLM-135M 11.2%
- 319
SmolLM-360M 10.9%
- 320
SmolLM2-135M 10.8%
Sources: Official MMLU-Pro leaderboard and labs’ model cards
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 89.8% | Sep 2026 | Lab model card | |
| 2 | | 89.6% | May 2026 | Lab model card | |
| 3 | | 88.5% | Apr 2026 | Lab model card | |
| 4 | | 88.5% | Jun 2026 | Lab model card | |
| 5 | | 88% | Dec 2025 | Lab model card | |
| 6 | | 87.8% | Feb 2026 | Lab model card | |
| 7 | | 87.5% | Apr 2026 | Lab model card | |
| 8 | | 87.2% | Mar 2025 | Lab model card | |
| 9 | Kimi K2.5 | 87.1% | Jan 2026 | Lab model card | |
| 10 | | 87% | Apr 2025 | Lab model card | |
| 11 | | 87% | Feb 2026 | Lab model card | |
| 12 | | 86.8% | Jun 2026 | Lab model card | |
| 13 | | 86.7% | Feb 2026 | Lab model card | |
| 14 | | 86.6% | Feb 2026 | Lab model card | |
| 15 | | 86.3% | Aug 2026 | Lab model card | |
| 16 | | 86.2% | Apr 2026 | Lab model card | |
| 17 | | 86.2% | Apr 2026 | Lab model card | |
| 18 | | 86.2% | Jul 2026 | Lab model card | |
| 19 | | 86.1% | Feb 2026 | Lab model card | |
| 20 | | 85.9% | Mar 2026 | Lab model card | |
| 21 | | 85.7% | Jan 2026 | Lab model card | |
| 22 | | 85.4% | Jul 2025 | Lab model card | |
| 23 | | 85.3% | Feb 2026 | Lab model card | |
| 24 | | 85.2% | Apr 2026 | Lab model card | |
| 25 | | 85.2% | Apr 2026 | Lab model card | |
| 26 | | 85.2% | May 2026 | Lab model card | |
| 27 | | 85% | May 2025 | Lab model card | |
| 28 | | 85% | Sep 2025 | Lab model card | |
| 29 | | 85% | Sep 2025 | Lab model card | |
| 30 | | 85% | Dec 2025 | Lab model card | |
| 31 | | 85% | Jun 2026 | Lab model card | |
| 32 | MiMo-V2-Flash | 84.9% | Dec 2025 | Lab model card | |
| 33 | | 84.9% | Dec 2025 | Lab model card | |
| 34 | | 84.8% | Aug 2025 | Lab model card | |
| 35 | | 84.7% | Nov 2025 | Lab model card | |
| 36 | GLM-4.5 | 84.6% | Jul 2025 | Lab model card | |
| 37 | Kimi K2 Thinking | 84.6% | Nov 2025 | Lab model card | |
| 38 | | 84.4% | Jul 2025 | Lab model card | |
| 39 | | 84.4% | Feb 2026 | Lab model card | |
| 40 | GLM-4.7 | 84.3% | Dec 2025 | Lab model card | |
| 41 | | 84.3% | May 2026 | Lab model card | |
| 42 | | 84% | Jan 2025 | Lab model card | |
| 43 | | 83.8% | Sep 2025 | Lab model card | |
| 44 | | 83.8% | Dec 2025 | Lab model card | |
| 45 | | 83.8% | Feb 2026 | Lab model card | |
| 46 | | 83.7% | Mar 2026 | Lab model card | |
| 47 | | 83.5% | Jul 2025 | Lab model card | |
| 48 | | 83.5% | Jul 2026 | Lab model card | |
| 49 | | 83.5% | Oct 2026 | Lab model card | |
| 50 | | 83.4% | Apr 2026 | Lab model card | |
| 51 | | 83.3% | Apr 2026 | Lab model card | |
| 52 | | 82.7% | Oct 2025 | Lab model card | |
| 53 | | 82.7% | Aug 2025 | Lab model card | |
| 54 | | 82.7% | Sep 2025 | Lab model card | |
| 55 | | 82.7% | Aug 2025 | Lab model card | |
| 56 | | 82.6% | Sep 2025 | Lab model card | |
| 57 | | 82.6% | Apr 2026 | Lab model card | |
| 58 | | 82.5% | Mar 2026 | Lab model card | |
| 59 | | 82.2% | Apr 2025 | Lab model card | |
| 60 | | 82% | Oct 2025 | Lab model card | |
| 61 | | 81.9% | Aug 2026 | Lab model card | |
| 62 | INTELLECT-3 | 81.9% | Nov 2025 | Lab model card | |
| 63 | | 81.8% | Jul 2025 | Lab model card | |
| 64 | | 81.8% | May 2026 | Lab model card | |
| 65 | Sarvam 105B | 81.7% | Mar 2026 | Lab model card | |
| 66 | | 81.6% | Dec 2025 | Lab model card | |
| 67 | GLM-4.5-Air | 81.4% | Aug 2025 | Lab model card | |
| 68 | | 81.3% | Aug 2025 | Lab model card | |
| 69 | | 81.2% | Mar 2026 | Lab model card | |
| 70 | | 81.2% | Mar 2025 | Lab model card | |
| 71 | | 81.1% | Jun 2025 | Lab model card | |
| 72 | Kimi K2 Jul 2025 | 81.1% | Jul 2025 | Lab model card | |
| 73 | | 80.9% | Dec 2025 | Lab model card | |
| 74 | | 80.7% | Dec 2025 | Lab model card | |
| 75 | Hermes 4.3 36B | 80.7% | Dec 2025 | Lab model card | |
| 76 | Hermes 4 405B | 80.6% | Aug 2025 | Lab model card | |
| 77 | | 80.5% | Oct 2025 | Lab model card | |
| 78 | | 80.5% | Feb 2026 | Lab model card | |
| 79 | | 80.5% | Apr 2025 | Lab model card | |
| 80 | | 80.4% | Jan 2026 | Lab model card | |
| 81 | Sarvam 30B | 80% | Mar 2026 | Lab model card | |
| 82 | Grok 3 | 79.9% | Feb 2025 | Lab model card | |
| 83 | | 79.8% | Mar 2026 | Lab model card | |
| 84 | | 79.5% | Jul 2025 | Lab model card | |
| 85 | | 79.1% | Feb 2025 | Lab model card | |
| 86 | | 79.1% | Mar 2026 | Lab model card | |
| 87 | | 79.1% | Jun 2026 | Lab model card | |
| 88 | | 79% | Feb 2025 | Lab model card | |
| 89 | Grok-3 mini | 78.9% | Feb 2025 | Lab model card | |
| 90 | | 78.5% | Apr 2025 | Lab model card | |
| 91 | | 78.4% | Jun 2025 | Lab model card | |
| 92 | | 78.3% | Dec 2025 | Lab model card | |
| 93 | Claude 3.5 Sonnet (October 2024) | 78% | Oct 2024 | Lab model card | |
| 94 | | 78% | Mar 2026 | Lab model card | |
| 95 | | 77.6% | Feb 2025 | Lab model card | |
| 96 | | 77.6% | Jun 2026 | Lab model card | |
| 97 | Granite 4.2 30B | 77.6% | Aug 2026 | Lab model card | |
| 98 | | 77.3% | Apr 2026 | Lab model card | |
| 99 | | 77.2% | May 2025 | Lab model card | |
| 100 | | 77.2% | Jun 2026 | Lab model card | |
| 101 | | 77.1% | Nov 2025 | Lab model card | |
| 102 | | 77% | Jan 2026 | Lab model card | |
| 103 | | 76.5% | Jul 2025 | Lab model card | |
| 104 | | 76.4% | Dec 2024 | Lab model card | |
| 105 | | 76.3% | Oct 2026 | Lab model card | |
| 106 | | 76% | Jan 2026 | Lab model card | |
| 107 | | 76% | Apr 2025 | Lab model card | |
| 108 | | 75.9% | Dec 2024 | Lab model card | |
| 109 | | 75.7% | Jan 2025 | Lab model card | |
| 110 | Grok-2 Aug 2024 | 75.5% | Aug 2024 | Lab model card | |
| 111 | | 75.3% | Feb 2026 | Lab model card | |
| 112 | | 74.3% | Apr 2025 | Lab model card | |
| 113 | | 74.3% | Apr 2025 | Lab model card | |
| 114 | | 74.1% | Oct 2026 | Lab model card | |
| 115 | | 74.1% | Sep 2026 | Lab model card | |
| 116 | | 74.1% | Sep 2026 | Lab model card | |
| 117 | | 74% | Mar 2025 | Lab model card | |
| 118 | | 73.3% | Jul 2024 | Lab model card | |
| 119 | Beam 1.0 Base | 72.6% | Oct 2026 | Lab model card | |
| 120 | | 72.1% | Jan 2026 | Lab model card | |
| 121 | | 71.6% | Feb 2025 | Lab model card | |
| 122 | | 71.2% | Feb 2025 | Lab model card | |
| 123 | | 71.1% | Sep 2024 | Lab model card | |
| 124 | | 70.4% | Dec 2024 | Lab model card | |
| 125 | | 69.6% | Apr 2025 | Lab model card | |
| 126 | | 69.4% | Apr 2026 | Lab model card | |
| 127 | | 69.1% | Jun 2025 | Lab model card | |
| 128 | | 69% | Dec 2024 | Lab model card | |
| 129 | Mercury | 69% | Jun 2025 | Lab model card | |
| 130 | | 68.9% | Dec 2024 | Lab model card | |
| 131 | Kimi-VL-A3B-Thinking-2506 | 68.5% | Jun 2025 | Lab model card | |
| 132 | | 67.5% | Mar 2025 | Lab model card | |
| 133 | | 67.5% | Oct 2026 | Lab model card | |
| 134 | | 66.8% | Mar 2025 | Lab model card | |
| 135 | | 66.5% | Mar 2026 | Lab model card | |
| 136 | | 66.4% | Jul 2024 | Lab model card | |
| 137 | | 66.3% | Jan 2025 | Lab model card | |
| 138 | | 65% | Dec 2025 | Lab model card | |
| 139 | Claude 3.5 Haiku | 65% | Oct 2024 | Lab model card | |
| 140 | Reka Flash 3 | 65% | Mar 2025 | Lab model card | |
| 141 | | 64.5% | Dec 2024 | Lab model card | |
| 142 | | 64.4% | Jun 2024 | Lab model card | |
| 143 | Granite 4.1 30B | 64.1% | Apr 2026 | Lab model card | |
| 144 | | 63.7% | Dec 2024 | Lab model card | |
| 145 | | 63.3% | Jan 2025 | Lab model card | |
| 146 | | 60.6% | Mar 2025 | Lab model card | |
| 147 | | 60.2% | Sep 2025 | Lab model card | |
| 148 | | 60% | Apr 2026 | Lab model card | |
| 149 | | 58.7% | May 2025 | Lab model card | |
| 150 | MiMo-7B-RL | 58.6% | Apr 2025 | Lab model card | |
| 151 | | 57.6% | Jan 2025 | Lab model card | |
| 152 | Granite 4.0 H Small | 55.5% | Oct 2025 | Lab model card | |
| 153 | | 54.3% | Aug 2024 | Lab model card | |
| 154 | | 52.8% | Feb 2025 | Lab model card | |
| 155 | | 52.7% | Dec 2024 | Lab model card | |
| 156 | | 50.6% | Jun 2025 | Lab model card | |
| 157 | LFM2.5-1.2B-Thinking | 49.6% | Jan 2026 | Lab model card | |
| 158 | | 48.3% | Jul 2024 | Lab model card | |
| 159 | | 47.5% | Aug 2024 | Lab model card | |
| 160 | | 47% | Mar 2025 | Lab model card | |
| 161 | | 45% | Dec 2024 | Lab model card | |
| 162 | LFM2.5-1.2B-Instruct | 44.4% | Jan 2026 | Lab model card | |
| 163 | | 43.6% | Mar 2025 | Lab model card | |
| 164 | Moonlight-16B-A3B | 42.4% | Feb 2025 | Lab model card | |
| 165 | | 42.3% | Mar 2026 | Lab model card | |
| 166 | LFM2-8B-A1B | 37.4% | Oct 2025 | Lab model card | |
| 167 | granite-4.0-1b | 34% | Oct 2025 | Lab model card | |
| 168 | | 30.7% | Aug 2026 | Lab model card | |
| 169 | LFM2-1.2B | 19.7% | Nov 2025 | Lab model card | |
| 170 | | 14.7% | Mar 2025 | Lab model card |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | 91.2% | Feb 2026 | Official leaderboard | ||
| 2 | 90.1% | Nov 2025 | Official leaderboard | ||
| 3 | o1 | 89.3% | Dec 2024 | Official leaderboard | |
| 4 | Claude Opus 4.6 | 89.1% | thinking | Feb 2026 | Official leaderboard |
| 5 | 88.6% | Dec 2025 | Official leaderboard | ||
| 6 | 88% | Dec 2025 | Official leaderboard | ||
| 7 | 87.8% | Feb 2026 | Official leaderboard | ||
| 8 | 87.7% | Feb 2026 | Official leaderboard | ||
| 9 | GPT-5.4 | 87.5% | Mar 2026 | Official leaderboard | |
| 10 | Claude Sonnet 4.5 | 87.4% | thinking | Sep 2025 | Official leaderboard |
| 11 | GPT-5.2 | 87.4% | Dec 2025 | Official leaderboard | |
| 12 | Claude Opus 4 | 87.3% | thinking | May 2025 | Official leaderboard |
| 13 | Claude Opus 4.5 | 87.3% | thinking | Nov 2025 | Official leaderboard |
| 14 | Claude Sonnet 4.6 | 87.3% | thinking | Feb 2026 | Official leaderboard |
| 15 | 87.2% | Mar 2025 | Official leaderboard | ||
| 16 | GPT-5 | 87.1% | high | Aug 2025 | Official leaderboard |
| 17 | 87% | Apr 2025 | Official leaderboard | ||
| 18 | Grok 4 | 87% | Jul 2025 | Official leaderboard | |
| 19 | 87% | Feb 2026 | Official leaderboard | ||
| 20 | 86.7% | Feb 2026 | Official leaderboard | ||
| 21 | 86.6% | Jun 2025 | Official leaderboard | ||
| 22 | GPT-5.1 | 86.4% | Nov 2025 | Official leaderboard | |
| 23 | 86.2% | Mar 2026 | Official leaderboard | ||
| 24 | GPT-4.5 | 86.1% | Feb 2025 | Official leaderboard | |
| 25 | 86.1% | Feb 2026 | Official leaderboard | ||
| 26 | 86% | Jun 2025 | Official leaderboard | ||
| 27 | GLM-5 | 86% | Feb 2026 | Official leaderboard | |
| 28 | 85.7% | Sep 2025 | Official leaderboard | ||
| 29 | 85.3% | Feb 2026 | Official leaderboard | ||
| 30 | o3 | 85% | high | Apr 2025 | Official leaderboard |
| 31 | 85% | Dec 2025 | Official leaderboard | ||
| 32 | 84.8% | Aug 2025 | Official leaderboard | ||
| 33 | GLM-4.5 | 84.6% | Aug 2025 | Official leaderboard | |
| 34 | 84.5% | Jul 2025 | Official leaderboard | ||
| 35 | Grok 4.1 Fast | 84.2% | reasoning | Nov 2025 | Official leaderboard |
| 36 | Claude 3.7 Sonnet | 84% | thinking | Feb 2025 | Official leaderboard |
| 37 | 84% | May 2025 | Official leaderboard | ||
| 38 | Claude Sonnet 4 | 83.7% | May 2025 | Official leaderboard | |
| 39 | 83.6% | Feb 2026 | Official leaderboard | ||
| 40 | 83.5% | Jul 2025 | Official leaderboard | ||
| 41 | Grok-3 mini | 83% | Apr 2025 | Official leaderboard | |
| 42 | 83% | Jul 2025 | Official leaderboard | ||
| 43 | 82.8% | Apr 2025 | Official leaderboard | ||
| 44 | 82.7% | Aug 2025 | Official leaderboard | ||
| 45 | 82.7% | Aug 2025 | Official leaderboard | ||
| 46 | 82.5% | Feb 2026 | Official leaderboard | ||
| 47 | 82% | Oct 2025 | Official leaderboard | ||
| 48 | GPT-4.1 | 81.8% | Apr 2025 | Official leaderboard | |
| 49 | GLM-4.5-Air | 81.4% | Jul 2025 | Official leaderboard | |
| 50 | 81.3% | Mar 2025 | Official leaderboard | ||
| 51 | 81.1% | Jun 2025 | Official leaderboard | ||
| 52 | Kimi K2 Jul 2025 | 81% | Jul 2025 | Official leaderboard | |
| 53 | 80.9% | Jul 2025 | Official leaderboard | ||
| 54 | gpt-oss-120b | 80.8% | high | Aug 2025 | Official leaderboard |
| 55 | 80.5% | Apr 2025 | Official leaderboard | ||
| 56 | o1-mini | 80.3% | Sep 2024 | Official leaderboard | |
| 57 | 80.1% | Jan 2025 | Official leaderboard | ||
| 58 | 80.1% | Feb 2026 | Official leaderboard | ||
| 59 | Grok 3 | 79.9% | Apr 2025 | Official leaderboard | |
| 60 | o3-mini | 79.4% | Jan 2025 | Official leaderboard | |
| 61 | 79.1% | Feb 2025 | Official leaderboard | ||
| 62 | 79.1% | Feb 2026 | Official leaderboard | ||
| 63 | 79% | Feb 2025 | Official leaderboard | ||
| 64 | 78.4% | Jun 2025 | Official leaderboard | ||
| 65 | 78.3% | bf16 | Dec 2025 | Official leaderboard | |
| 66 | Claude 3.5 Sonnet (October 2024) | 78% | Oct 2024 | Official leaderboard | |
| 67 | GPT-4o Nov 2024 | 77.9% | Nov 2024 | Official leaderboard | |
| 68 | 77.6% | Feb 2025 | Official leaderboard | ||
| 69 | Claude 3.5 Sonnet | 76.1% | Jun 2024 | Official leaderboard | |
| 70 | 76.1% | Jan 2025 | Official leaderboard | ||
| 71 | 76% | Apr 2025 | Official leaderboard | ||
| 72 | 75.7% | Jan 2025 | Official leaderboard | ||
| 73 | Grok-2 Dec 2024 | 75.5% | Dec 2024 | Official leaderboard | |
| 74 | GPT-4o Aug 2024 | 74.7% | Aug 2024 | Official leaderboard | |
| 75 | 74.3% | Apr 2025 | Official leaderboard | ||
| 76 | 74.3% | Apr 2025 | Official leaderboard | ||
| 77 | gpt-oss-20b | 73.6% | high | Aug 2025 | Official leaderboard |
| 78 | 73.3% | Jul 2024 | Official leaderboard | ||
| 79 | 73.1% | Nov 2024 | Official leaderboard | ||
| 80 | GPT-4o May 2024 | 72.5% | May 2024 | Official leaderboard | |
| 81 | Grok-2 mini | 71.8% | Aug 2024 | Official leaderboard | |
| 82 | 71.6% | Feb 2025 | Official leaderboard | ||
| 83 | 71.6% | Sep 2024 | Official leaderboard | ||
| 84 | 71% | Nov 2024 | Official leaderboard | ||
| 85 | 70.4% | Dec 2024 | Official leaderboard | ||
| 86 | 70.3% | Sep 2024 | Official leaderboard | ||
| 87 | 69.5% | Jun 2025 | Official leaderboard | ||
| 88 | 69.2% | Sep 2024 | Official leaderboard | ||
| 89 | 69.1% | Mar 2025 | Official leaderboard | ||
| 90 | Claude 3 Opus | 68.5% | Feb 2024 | Official leaderboard | |
| 91 | 68.2% | Apr 2025 | Official leaderboard | ||
| 92 | 67.9% | Nov 2024 | Official leaderboard | ||
| 93 | 67.5% | Mar 2025 | Official leaderboard | ||
| 94 | 67.3% | Jun 2025 | Official leaderboard | ||
| 95 | 66.8% | Mar 2025 | Official leaderboard | ||
| 96 | 65.9% | Dec 2024 | Official leaderboard | ||
| 97 | 65.9% | Jul 2024 | Official leaderboard | ||
| 98 | 65.8% | Sep 2024 | Official leaderboard | ||
| 99 | 65.1% | Aug 2025 | Official leaderboard | ||
| 100 | 65.1% | Dec 2025 | Official leaderboard | ||
| 101 | 64.4% | Jun 2024 | Official leaderboard | ||
| 102 | 64.1% | Aug 2024 | Official leaderboard | ||
| 103 | GPT-4 Turbo Apr 2024 | 63.7% | Apr 2024 | Official leaderboard | |
| 104 | 63.7% | Sep 2024 | Official leaderboard | ||
| 105 | 63.6% | Jun 2024 | Official leaderboard | ||
| 106 | GPT-4o mini | 63.1% | Jul 2024 | Official leaderboard | |
| 107 | 62.8% | Jul 2024 | Official leaderboard | ||
| 108 | 62.8% | Oct 2024 | Official leaderboard | ||
| 109 | 62.4% | Dec 2024 | Official leaderboard | ||
| 110 | Claude 3.5 Haiku | 62.1% | Oct 2024 | Official leaderboard | |
| 111 | 61.7% | Apr 2025 | Official leaderboard | ||
| 112 | 60.6% | Mar 2025 | Official leaderboard | ||
| 113 | 60.5% | Apr 2025 | Official leaderboard | ||
| 114 | 60.4% | Aug 2025 | Official leaderboard | ||
| 115 | 58.9% | Dec 2024 | Official leaderboard | ||
| 116 | MiMo-7B-RL | 58.6% | Apr 2025 | Official leaderboard | |
| 117 | 58.1% | May 2024 | Official leaderboard | ||
| 118 | 57.6% | Jan 2025 | Official leaderboard | ||
| 119 | Claude 3 Sonnet | 56.8% | Feb 2024 | Official leaderboard | |
| 120 | 56.7% | Jun 2025 | Official leaderboard | ||
| 121 | 56.5% | Jun 2024 | Official leaderboard | ||
| 122 | 56.3% | Apr 2024 | Official leaderboard | ||
| 123 | 56.2% | Apr 2024 | Official leaderboard | ||
| 124 | 55.7% | Apr 2024 | Official leaderboard | ||
| 125 | 55.6% | Nov 2024 | Official leaderboard | ||
| 126 | 55.3% | Feb 2026 | Official leaderboard | ||
| 127 | 54.8% | May 2024 | Official leaderboard | ||
| 128 | 52.8% | Feb 2025 | Official leaderboard | ||
| 129 | 52.6% | Feb 2024 | Official leaderboard | ||
| 130 | 52.3% | May 2024 | Official leaderboard | ||
| 131 | 52.1% | Jun 2024 | Official leaderboard | ||
| 132 | 51.9% | May 2024 | Official leaderboard | ||
| 133 | 50.4% | May 2024 | Official leaderboard | ||
| 134 | 49.9% | Apr 2024 | Official leaderboard | ||
| 135 | Jamba 1.5 Large | 49.5% | Aug 2024 | Official leaderboard | |
| 136 | 48.4% | Sep 2024 | Official leaderboard | ||
| 137 | GLM-4-9B-Chat | 48% | Jun 2024 | Official leaderboard | |
| 138 | GLM-4-9B | 47.9% | Jun 2024 | Official leaderboard | |
| 139 | 47.9% | Aug 2024 | Official leaderboard | ||
| 140 | 47.2% | Jun 2024 | Official leaderboard | ||
| 141 | 46.2% | Dec 2024 | Official leaderboard | ||
| 142 | 46% | May 2024 | Official leaderboard | ||
| 143 | 45.7% | Apr 2024 | Official leaderboard | ||
| 144 | 45.4% | Oct 2024 | Official leaderboard | ||
| 145 | 45% | Sep 2024 | Official leaderboard | ||
| 146 | 44.8% | Jul 2024 | Official leaderboard | ||
| 147 | 44.3% | Jul 2024 | Official leaderboard | ||
| 148 | 44% | Mar 2025 | Official leaderboard | ||
| 149 | 43.9% | Apr 2024 | Official leaderboard | ||
| 150 | 43.7% | Sep 2024 | Official leaderboard | ||
| 151 | 43.6% | Mar 2025 | Official leaderboard | ||
| 152 | 43.4% | May 2024 | Official leaderboard | ||
| 153 | 43.3% | Dec 2023 | Official leaderboard | ||
| 154 | 43% | Nov 2023 | Official leaderboard | ||
| 155 | Claude 3 Haiku | 42.3% | Mar 2024 | Official leaderboard | |
| 156 | 42% | Jul 2024 | Official leaderboard | ||
| 157 | MiMo-7B-Base | 41.9% | Apr 2025 | Official leaderboard | |
| 158 | DeepSeek-Coder-V2-Lite-Instruct | 41.6% | Jun 2024 | Official leaderboard | |
| 159 | Granite-3.1-8B-Instruct | 41% | Dec 2024 | Official leaderboard | |
| 160 | 41% | Apr 2024 | Official leaderboard | ||
| 161 | 40.9% | May 2024 | Official leaderboard | ||
| 162 | 40.7% | Jun 2024 | Official leaderboard | ||
| 163 | 39.8% | Jul 2024 | Official leaderboard | ||
| 164 | 39.2% | Apr 2024 | Official leaderboard | ||
| 165 | 39.1% | Dec 2024 | Official leaderboard | ||
| 166 | 38.2% | May 2024 | Official leaderboard | ||
| 167 | 38% | Feb 2024 | Official leaderboard | ||
| 168 | 37.9% | Oct 2024 | Official leaderboard | ||
| 169 | 37.9% | Mar 2024 | Official leaderboard | ||
| 170 | 37.5% | Jul 2023 | Official leaderboard | ||
| 171 | 35.3% | Feb 2024 | Official leaderboard | ||
| 172 | DeepSeek-Coder-V2-Lite-Base | 34.4% | Jun 2024 | Official leaderboard | |
| 173 | 33.7% | Oct 2024 | Official leaderboard | ||
| 174 | 33.7% | Feb 2024 | Official leaderboard | ||
| 175 | Granite-3.1-8B-Base | 33.1% | Dec 2024 | Official leaderboard | |
| 176 | 33% | Oct 2023 | Official leaderboard | ||
| 177 | 32.1% | Sep 2024 | Official leaderboard | ||
| 178 | Granite-3.1-2B-Instruct | 32% | Dec 2024 | Official leaderboard | |
| 179 | Granite-3.0-8B-Base | 31% | Oct 2024 | Official leaderboard | |
| 180 | 30.9% | Sep 2023 | Official leaderboard | ||
| 181 | 30.8% | Dec 2023 | Official leaderboard | ||
| 182 | 30.4% | Dec 2023 | Official leaderboard | ||
| 183 | 29.7% | Feb 2026 | Official leaderboard | ||
| 184 | 29.1% | Feb 2024 | Official leaderboard | ||
| 185 | 28.8% | Nov 2023 | Official leaderboard | ||
| 186 | 26.5% | Nov 2023 | Official leaderboard | ||
| 187 | 25.8% | Sep 2023 | Official leaderboard | ||
| 188 | Granite-3.1-3B-A800M-Instruct | 25.4% | Dec 2024 | Official leaderboard | |
| 189 | 25.3% | Jul 2023 | Official leaderboard | ||
| 190 | Granite-3.1-2B-Base | 23.9% | Dec 2024 | Official leaderboard | |
| 191 | Llemma 7B | 23.4% | Oct 2023 | Official leaderboard | |
| 192 | 22.6% | Jun 2024 | Official leaderboard | ||
| 193 | 22.6% | Jun 2024 | Official leaderboard | ||
| 194 | 22.2% | Sep 2024 | Official leaderboard | ||
| 195 | Granite-3.0-2B-Base | 21.7% | Oct 2024 | Official leaderboard | |
| 196 | Granite-3.1-3B-A800M-Base | 20.4% | Dec 2024 | Official leaderboard | |
| 197 | 20.3% | Jul 2023 | Official leaderboard | ||
| 198 | 18.3% | Oct 2024 | Official leaderboard | ||
| 199 | 15.9% | Jun 2024 | Official leaderboard | ||
| 200 | 15.8% | Feb 2024 | Official leaderboard | ||
| 201 | 15.6% | Jul 2024 | Official leaderboard | ||
| 202 | 15% | Jun 2024 | Official leaderboard | ||
| 203 | 14.9% | Sep 2024 | Official leaderboard | ||
| 204 | 14.7% | Mar 2025 | Official leaderboard | ||
| 205 | Granite-3.1-1B-A400M-Instruct | 13.3% | Dec 2024 | Official leaderboard | |
| 206 | Granite-3.1-1B-A400M-Base | 12.3% | Dec 2024 | Official leaderboard | |
| 207 | 11.9% | Sep 2024 | Official leaderboard | ||
| 208 | 11.9% | Jul 2024 | Official leaderboard | ||
| 209 | 11.4% | Oct 2024 | Official leaderboard | ||
| 210 | 11.2% | Jul 2024 | Official leaderboard | ||
| 211 | 10.9% | Jul 2024 | Official leaderboard | ||
| 212 | 10.8% | Oct 2024 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | 91.2% | Feb 2026 | Official leaderboard | ||
| 2 | 90.1% | Nov 2025 | Official leaderboard | ||
| 3 | 89.8% | Sep 2026 | Lab model card | ||
| 4 | 89.6% | May 2026 | Lab model card | ||
| 5 | o1 | 89.3% | Dec 2024 | Official leaderboard | |
| 6 | Claude Opus 4.6 | 89.1% | thinking | Feb 2026 | Official leaderboard |
| 7 | 88.6% | Dec 2025 | Official leaderboard | ||
| 8 | 88.5% | Apr 2026 | Lab model card | ||
| 9 | 88.5% | Jun 2026 | Lab model card | ||
| 10 | 88% | Dec 2025 | Official leaderboard | ||
| 11 | 87.8% | Feb 2026 | Official leaderboard | ||
| 12 | 87.7% | Feb 2026 | Official leaderboard | ||
| 13 | GPT-5.4 | 87.5% | Mar 2026 | Official leaderboard | |
| 14 | 87.5% | Apr 2026 | Lab model card | ||
| 15 | Claude Sonnet 4.5 | 87.4% | thinking | Sep 2025 | Official leaderboard |
| 16 | GPT-5.2 | 87.4% | Dec 2025 | Official leaderboard | |
| 17 | Claude Opus 4 | 87.3% | thinking | May 2025 | Official leaderboard |
| 18 | Claude Opus 4.5 | 87.3% | thinking | Nov 2025 | Official leaderboard |
| 19 | Claude Sonnet 4.6 | 87.3% | thinking | Feb 2026 | Official leaderboard |
| 20 | 87.2% | Mar 2025 | Official leaderboard | ||
| 21 | GPT-5 | 87.1% | high | Aug 2025 | Official leaderboard |
| 22 | Kimi K2.5 | 87.1% | Jan 2026 | Lab model card | |
| 23 | 87% | Apr 2025 | Official leaderboard | ||
| 24 | Grok 4 | 87% | Jul 2025 | Official leaderboard | |
| 25 | 87% | Feb 2026 | Official leaderboard | ||
| 26 | 86.8% | Jun 2026 | Lab model card | ||
| 27 | 86.7% | Feb 2026 | Official leaderboard | ||
| 28 | 86.6% | Jun 2025 | Official leaderboard | ||
| 29 | 86.6% | Feb 2026 | Lab model card | ||
| 30 | GPT-5.1 | 86.4% | Nov 2025 | Official leaderboard | |
| 31 | 86.3% | Aug 2026 | Lab model card | ||
| 32 | 86.2% | Mar 2026 | Official leaderboard | ||
| 33 | 86.2% | Apr 2026 | Lab model card | ||
| 34 | 86.2% | Apr 2026 | Lab model card | ||
| 35 | 86.2% | Jul 2026 | Lab model card | ||
| 36 | GPT-4.5 | 86.1% | Feb 2025 | Official leaderboard | |
| 37 | 86.1% | Feb 2026 | Official leaderboard | ||
| 38 | 86% | Jun 2025 | Official leaderboard | ||
| 39 | GLM-5 | 86% | Feb 2026 | Official leaderboard | |
| 40 | 85.9% | Mar 2026 | Lab model card | ||
| 41 | 85.7% | Sep 2025 | Official leaderboard | ||
| 42 | 85.7% | Jan 2026 | Lab model card | ||
| 43 | 85.4% | Jul 2025 | Lab model card | ||
| 44 | 85.3% | Feb 2026 | Official leaderboard | ||
| 45 | 85.2% | Apr 2026 | Lab model card | ||
| 46 | 85.2% | Apr 2026 | Lab model card | ||
| 47 | 85.2% | May 2026 | Lab model card | ||
| 48 | o3 | 85% | high | Apr 2025 | Official leaderboard |
| 49 | 85% | May 2025 | Lab model card | ||
| 50 | 85% | Sep 2025 | Lab model card | ||
| 51 | 85% | Sep 2025 | Lab model card | ||
| 52 | 85% | Dec 2025 | Official leaderboard | ||
| 53 | 85% | Jun 2026 | Lab model card | ||
| 54 | MiMo-V2-Flash | 84.9% | Dec 2025 | Lab model card | |
| 55 | 84.9% | Dec 2025 | Lab model card | ||
| 56 | 84.8% | Aug 2025 | Official leaderboard | ||
| 57 | 84.7% | Nov 2025 | Lab model card | ||
| 58 | GLM-4.5 | 84.6% | Aug 2025 | Official leaderboard | |
| 59 | Kimi K2 Thinking | 84.6% | Nov 2025 | Lab model card | |
| 60 | 84.5% | Jul 2025 | Official leaderboard | ||
| 61 | 84.4% | Feb 2026 | Lab model card | ||
| 62 | GLM-4.7 | 84.3% | Dec 2025 | Lab model card | |
| 63 | 84.3% | May 2026 | Lab model card | ||
| 64 | Grok 4.1 Fast | 84.2% | reasoning | Nov 2025 | Official leaderboard |
| 65 | 84% | Jan 2025 | Lab model card | ||
| 66 | Claude 3.7 Sonnet | 84% | thinking | Feb 2025 | Official leaderboard |
| 67 | 83.8% | Sep 2025 | Lab model card | ||
| 68 | 83.8% | Dec 2025 | Lab model card | ||
| 69 | 83.8% | Feb 2026 | Lab model card | ||
| 70 | 83.7% | Mar 2026 | Lab model card | ||
| 71 | Claude Sonnet 4 | 83.7% | May 2025 | Official leaderboard | |
| 72 | 83.6% | Feb 2026 | Official leaderboard | ||
| 73 | 83.5% | Jul 2025 | Official leaderboard | ||
| 74 | 83.5% | Jul 2026 | Lab model card | ||
| 75 | 83.5% | Oct 2026 | Lab model card | ||
| 76 | 83.4% | Apr 2026 | Lab model card | ||
| 77 | 83.3% | Apr 2026 | Lab model card | ||
| 78 | Grok-3 mini | 83% | Apr 2025 | Official leaderboard | |
| 79 | 83% | Jul 2025 | Official leaderboard | ||
| 80 | 82.8% | Apr 2025 | Official leaderboard | ||
| 81 | 82.7% | Oct 2025 | Lab model card | ||
| 82 | 82.7% | Aug 2025 | Official leaderboard | ||
| 83 | 82.7% | Aug 2025 | Official leaderboard | ||
| 84 | 82.7% | Sep 2025 | Lab model card | ||
| 85 | 82.6% | Sep 2025 | Lab model card | ||
| 86 | 82.6% | Apr 2026 | Lab model card | ||
| 87 | 82.5% | Feb 2026 | Official leaderboard | ||
| 88 | 82% | Oct 2025 | Official leaderboard | ||
| 89 | 81.9% | Aug 2026 | Lab model card | ||
| 90 | INTELLECT-3 | 81.9% | Nov 2025 | Lab model card | |
| 91 | GPT-4.1 | 81.8% | Apr 2025 | Official leaderboard | |
| 92 | 81.8% | Jul 2025 | Lab model card | ||
| 93 | 81.8% | May 2026 | Lab model card | ||
| 94 | Sarvam 105B | 81.7% | Mar 2026 | Lab model card | |
| 95 | 81.6% | Dec 2025 | Lab model card | ||
| 96 | GLM-4.5-Air | 81.4% | Jul 2025 | Official leaderboard | |
| 97 | 81.3% | Mar 2025 | Official leaderboard | ||
| 98 | 81.3% | Aug 2025 | Lab model card | ||
| 99 | 81.2% | Mar 2026 | Lab model card | ||
| 100 | 81.1% | Jun 2025 | Official leaderboard | ||
| 101 | Kimi K2 Jul 2025 | 81.1% | Jul 2025 | Lab model card | |
| 102 | 80.9% | Jul 2025 | Official leaderboard | ||
| 103 | 80.9% | Dec 2025 | Lab model card | ||
| 104 | gpt-oss-120b | 80.8% | high | Aug 2025 | Official leaderboard |
| 105 | 80.7% | Dec 2025 | Lab model card | ||
| 106 | Hermes 4.3 36B | 80.7% | Dec 2025 | Lab model card | |
| 107 | Hermes 4 405B | 80.6% | Aug 2025 | Lab model card | |
| 108 | 80.5% | Oct 2025 | Lab model card | ||
| 109 | 80.5% | Feb 2026 | Lab model card | ||
| 110 | 80.5% | Apr 2025 | Official leaderboard | ||
| 111 | 80.4% | Jan 2026 | Lab model card | ||
| 112 | o1-mini | 80.3% | Sep 2024 | Official leaderboard | |
| 113 | 80.1% | Jan 2025 | Official leaderboard | ||
| 114 | Sarvam 30B | 80% | Mar 2026 | Lab model card | |
| 115 | Grok 3 | 79.9% | Apr 2025 | Official leaderboard | |
| 116 | 79.8% | Mar 2026 | Lab model card | ||
| 117 | 79.5% | Jul 2025 | Lab model card | ||
| 118 | o3-mini | 79.4% | Jan 2025 | Official leaderboard | |
| 119 | 79.1% | Feb 2025 | Official leaderboard | ||
| 120 | 79.1% | Feb 2026 | Official leaderboard | ||
| 121 | 79.1% | Jun 2026 | Lab model card | ||
| 122 | 79% | Feb 2025 | Official leaderboard | ||
| 123 | 78.5% | Apr 2025 | Lab model card | ||
| 124 | 78.4% | Jun 2025 | Official leaderboard | ||
| 125 | 78.3% | bf16 | Dec 2025 | Official leaderboard | |
| 126 | Claude 3.5 Sonnet (October 2024) | 78% | Oct 2024 | Official leaderboard | |
| 127 | 78% | Mar 2026 | Lab model card | ||
| 128 | GPT-4o Nov 2024 | 77.9% | Nov 2024 | Official leaderboard | |
| 129 | 77.6% | Feb 2025 | Official leaderboard | ||
| 130 | 77.6% | Jun 2026 | Lab model card | ||
| 131 | Granite 4.2 30B | 77.6% | Aug 2026 | Lab model card | |
| 132 | 77.3% | Apr 2026 | Lab model card | ||
| 133 | 77.2% | May 2025 | Lab model card | ||
| 134 | 77.2% | Jun 2026 | Lab model card | ||
| 135 | 77.1% | Nov 2025 | Lab model card | ||
| 136 | 77% | Jan 2026 | Lab model card | ||
| 137 | 76.5% | Jul 2025 | Lab model card | ||
| 138 | 76.4% | Dec 2024 | Lab model card | ||
| 139 | 76.3% | Oct 2026 | Lab model card | ||
| 140 | Claude 3.5 Sonnet | 76.1% | Jun 2024 | Official leaderboard | |
| 141 | 76.1% | Jan 2025 | Official leaderboard | ||
| 142 | 76% | Jan 2026 | Lab model card | ||
| 143 | 76% | Apr 2025 | Official leaderboard | ||
| 144 | 75.9% | Dec 2024 | Lab model card | ||
| 145 | 75.7% | Jan 2025 | Official leaderboard | ||
| 146 | Grok-2 Aug 2024 | 75.5% | Aug 2024 | Lab model card | |
| 147 | Grok-2 Dec 2024 | 75.5% | Dec 2024 | Official leaderboard | |
| 148 | 75.3% | Feb 2026 | Lab model card | ||
| 149 | GPT-4o Aug 2024 | 74.7% | Aug 2024 | Official leaderboard | |
| 150 | 74.3% | Apr 2025 | Official leaderboard | ||
| 151 | 74.3% | Apr 2025 | Official leaderboard | ||
| 152 | 74.1% | Oct 2026 | Lab model card | ||
| 153 | 74.1% | Sep 2026 | Lab model card | ||
| 154 | 74.1% | Sep 2026 | Lab model card | ||
| 155 | 74% | Mar 2025 | Lab model card | ||
| 156 | gpt-oss-20b | 73.6% | high | Aug 2025 | Official leaderboard |
| 157 | 73.3% | Jul 2024 | Official leaderboard | ||
| 158 | 73.1% | Nov 2024 | Official leaderboard | ||
| 159 | Beam 1.0 Base | 72.6% | Oct 2026 | Lab model card | |
| 160 | GPT-4o May 2024 | 72.5% | May 2024 | Official leaderboard | |
| 161 | 72.1% | Jan 2026 | Lab model card | ||
| 162 | Grok-2 mini | 71.8% | Aug 2024 | Official leaderboard | |
| 163 | 71.6% | Feb 2025 | Official leaderboard | ||
| 164 | 71.6% | Sep 2024 | Official leaderboard | ||
| 165 | 71.2% | Feb 2025 | Lab model card | ||
| 166 | 71% | Nov 2024 | Official leaderboard | ||
| 167 | 70.4% | Dec 2024 | Official leaderboard | ||
| 168 | 70.3% | Sep 2024 | Official leaderboard | ||
| 169 | 69.6% | Apr 2025 | Lab model card | ||
| 170 | 69.5% | Jun 2025 | Official leaderboard | ||
| 171 | 69.4% | Apr 2026 | Lab model card | ||
| 172 | 69.2% | Sep 2024 | Official leaderboard | ||
| 173 | 69.1% | Mar 2025 | Official leaderboard | ||
| 174 | 69.1% | Jun 2025 | Lab model card | ||
| 175 | Mercury | 69% | Jun 2025 | Lab model card | |
| 176 | 68.9% | Dec 2024 | Lab model card | ||
| 177 | Kimi-VL-A3B-Thinking-2506 | 68.5% | Jun 2025 | Lab model card | |
| 178 | Claude 3 Opus | 68.5% | Feb 2024 | Official leaderboard | |
| 179 | 68.2% | Apr 2025 | Official leaderboard | ||
| 180 | 67.9% | Nov 2024 | Official leaderboard | ||
| 181 | 67.5% | Mar 2025 | Official leaderboard | ||
| 182 | 67.5% | Oct 2026 | Lab model card | ||
| 183 | 67.3% | Jun 2025 | Official leaderboard | ||
| 184 | 66.8% | Mar 2025 | Official leaderboard | ||
| 185 | 66.5% | Mar 2026 | Lab model card | ||
| 186 | 66.4% | Jul 2024 | Lab model card | ||
| 187 | 66.3% | Jan 2025 | Lab model card | ||
| 188 | 65.9% | Jul 2024 | Official leaderboard | ||
| 189 | 65.8% | Sep 2024 | Official leaderboard | ||
| 190 | 65.1% | Aug 2025 | Official leaderboard | ||
| 191 | 65.1% | Dec 2025 | Official leaderboard | ||
| 192 | Claude 3.5 Haiku | 65% | Oct 2024 | Lab model card | |
| 193 | Reka Flash 3 | 65% | Mar 2025 | Lab model card | |
| 194 | 64.5% | Dec 2024 | Lab model card | ||
| 195 | 64.4% | Jun 2024 | Lab model card | ||
| 196 | 64.1% | Aug 2024 | Official leaderboard | ||
| 197 | Granite 4.1 30B | 64.1% | Apr 2026 | Lab model card | |
| 198 | GPT-4 Turbo Apr 2024 | 63.7% | Apr 2024 | Official leaderboard | |
| 199 | 63.7% | Dec 2024 | Lab model card | ||
| 200 | 63.6% | Jun 2024 | Official leaderboard | ||
| 201 | 63.3% | Jan 2025 | Lab model card | ||
| 202 | GPT-4o mini | 63.1% | Jul 2024 | Official leaderboard | |
| 203 | 62.8% | Oct 2024 | Official leaderboard | ||
| 204 | 62.4% | Dec 2024 | Official leaderboard | ||
| 205 | 61.7% | Apr 2025 | Official leaderboard | ||
| 206 | 60.6% | Mar 2025 | Official leaderboard | ||
| 207 | 60.5% | Apr 2025 | Official leaderboard | ||
| 208 | 60.4% | Aug 2025 | Official leaderboard | ||
| 209 | 60.2% | Sep 2025 | Lab model card | ||
| 210 | 60% | Apr 2026 | Lab model card | ||
| 211 | 58.9% | Dec 2024 | Official leaderboard | ||
| 212 | 58.7% | May 2025 | Lab model card | ||
| 213 | MiMo-7B-RL | 58.6% | Apr 2025 | Official leaderboard | |
| 214 | 58.1% | May 2024 | Official leaderboard | ||
| 215 | 57.6% | Jan 2025 | Official leaderboard | ||
| 216 | Claude 3 Sonnet | 56.8% | Feb 2024 | Official leaderboard | |
| 217 | 56.7% | Jun 2025 | Official leaderboard | ||
| 218 | 56.5% | Jun 2024 | Official leaderboard | ||
| 219 | 56.3% | Apr 2024 | Official leaderboard | ||
| 220 | 56.2% | Apr 2024 | Official leaderboard | ||
| 221 | 55.7% | Apr 2024 | Official leaderboard | ||
| 222 | 55.6% | Nov 2024 | Official leaderboard | ||
| 223 | Granite 4.0 H Small | 55.5% | Oct 2025 | Lab model card | |
| 224 | 54.8% | May 2024 | Official leaderboard | ||
| 225 | 54.3% | Aug 2024 | Lab model card | ||
| 226 | 52.8% | Feb 2025 | Official leaderboard | ||
| 227 | 52.7% | Dec 2024 | Lab model card | ||
| 228 | 52.6% | Feb 2024 | Official leaderboard | ||
| 229 | 52.3% | May 2024 | Official leaderboard | ||
| 230 | 52.1% | Jun 2024 | Official leaderboard | ||
| 231 | 51.9% | May 2024 | Official leaderboard | ||
| 232 | 50.6% | Jun 2025 | Lab model card | ||
| 233 | 50.4% | May 2024 | Official leaderboard | ||
| 234 | 49.9% | Apr 2024 | Official leaderboard | ||
| 235 | LFM2.5-1.2B-Thinking | 49.6% | Jan 2026 | Lab model card | |
| 236 | Jamba 1.5 Large | 49.5% | Aug 2024 | Official leaderboard | |
| 237 | 48.4% | Sep 2024 | Official leaderboard | ||
| 238 | 48.3% | Jul 2024 | Lab model card | ||
| 239 | GLM-4-9B-Chat | 48% | Jun 2024 | Official leaderboard | |
| 240 | GLM-4-9B | 47.9% | Jun 2024 | Official leaderboard | |
| 241 | 47.9% | Aug 2024 | Official leaderboard | ||
| 242 | 47.2% | Jun 2024 | Official leaderboard | ||
| 243 | 47% | Mar 2025 | Lab model card | ||
| 244 | 46.2% | Dec 2024 | Official leaderboard | ||
| 245 | 46% | May 2024 | Official leaderboard | ||
| 246 | 45.7% | Apr 2024 | Official leaderboard | ||
| 247 | 45.4% | Oct 2024 | Official leaderboard | ||
| 248 | 45% | Sep 2024 | Official leaderboard | ||
| 249 | 44.8% | Jul 2024 | Official leaderboard | ||
| 250 | LFM2.5-1.2B-Instruct | 44.4% | Jan 2026 | Lab model card | |
| 251 | 44% | Mar 2025 | Official leaderboard | ||
| 252 | 43.9% | Apr 2024 | Official leaderboard | ||
| 253 | 43.7% | Sep 2024 | Official leaderboard | ||
| 254 | 43.6% | Mar 2025 | Official leaderboard | ||
| 255 | 43.4% | May 2024 | Official leaderboard | ||
| 256 | 43.3% | Dec 2023 | Official leaderboard | ||
| 257 | 43% | Nov 2023 | Official leaderboard | ||
| 258 | Moonlight-16B-A3B | 42.4% | Feb 2025 | Lab model card | |
| 259 | 42.3% | Mar 2026 | Lab model card | ||
| 260 | Claude 3 Haiku | 42.3% | Mar 2024 | Official leaderboard | |
| 261 | 42% | Jul 2024 | Official leaderboard | ||
| 262 | MiMo-7B-Base | 41.9% | Apr 2025 | Official leaderboard | |
| 263 | DeepSeek-Coder-V2-Lite-Instruct | 41.6% | Jun 2024 | Official leaderboard | |
| 264 | Granite-3.1-8B-Instruct | 41% | Dec 2024 | Official leaderboard | |
| 265 | 41% | Apr 2024 | Official leaderboard | ||
| 266 | 40.9% | May 2024 | Official leaderboard | ||
| 267 | 40.7% | Jun 2024 | Official leaderboard | ||
| 268 | 39.8% | Jul 2024 | Official leaderboard | ||
| 269 | 39.2% | Apr 2024 | Official leaderboard | ||
| 270 | 39.1% | Dec 2024 | Official leaderboard | ||
| 271 | 38.2% | May 2024 | Official leaderboard | ||
| 272 | 38% | Feb 2024 | Official leaderboard | ||
| 273 | 37.9% | Oct 2024 | Official leaderboard | ||
| 274 | 37.9% | Mar 2024 | Official leaderboard | ||
| 275 | 37.5% | Jul 2023 | Official leaderboard | ||
| 276 | LFM2-8B-A1B | 37.4% | Oct 2025 | Lab model card | |
| 277 | 35.3% | Feb 2024 | Official leaderboard | ||
| 278 | DeepSeek-Coder-V2-Lite-Base | 34.4% | Jun 2024 | Official leaderboard | |
| 279 | granite-4.0-1b | 34% | Oct 2025 | Lab model card | |
| 280 | 33.7% | Oct 2024 | Official leaderboard | ||
| 281 | 33.7% | Feb 2024 | Official leaderboard | ||
| 282 | Granite-3.1-8B-Base | 33.1% | Dec 2024 | Official leaderboard | |
| 283 | 33% | Oct 2023 | Official leaderboard | ||
| 284 | 32.1% | Sep 2024 | Official leaderboard | ||
| 285 | Granite-3.1-2B-Instruct | 32% | Dec 2024 | Official leaderboard | |
| 286 | Granite-3.0-8B-Base | 31% | Oct 2024 | Official leaderboard | |
| 287 | 30.9% | Sep 2023 | Official leaderboard | ||
| 288 | 30.8% | Dec 2023 | Official leaderboard | ||
| 289 | 30.7% | Aug 2026 | Lab model card | ||
| 290 | 30.4% | Dec 2023 | Official leaderboard | ||
| 291 | 29.1% | Feb 2024 | Official leaderboard | ||
| 292 | 28.8% | Nov 2023 | Official leaderboard | ||
| 293 | 26.5% | Nov 2023 | Official leaderboard | ||
| 294 | 25.8% | Sep 2023 | Official leaderboard | ||
| 295 | Granite-3.1-3B-A800M-Instruct | 25.4% | Dec 2024 | Official leaderboard | |
| 296 | 25.3% | Jul 2023 | Official leaderboard | ||
| 297 | Granite-3.1-2B-Base | 23.9% | Dec 2024 | Official leaderboard | |
| 298 | Llemma 7B | 23.4% | Oct 2023 | Official leaderboard | |
| 299 | 22.6% | Jun 2024 | Official leaderboard | ||
| 300 | 22.6% | Jun 2024 | Official leaderboard | ||
| 301 | 22.2% | Sep 2024 | Official leaderboard | ||
| 302 | Granite-3.0-2B-Base | 21.7% | Oct 2024 | Official leaderboard | |
| 303 | Granite-3.1-3B-A800M-Base | 20.4% | Dec 2024 | Official leaderboard | |
| 304 | 20.3% | Jul 2023 | Official leaderboard | ||
| 305 | LFM2-1.2B | 19.7% | Nov 2025 | Lab model card | |
| 306 | 18.3% | Oct 2024 | Official leaderboard | ||
| 307 | 15.9% | Jun 2024 | Official leaderboard | ||
| 308 | 15.8% | Feb 2024 | Official leaderboard | ||
| 309 | 15.6% | Jul 2024 | Official leaderboard | ||
| 310 | 15% | Jun 2024 | Official leaderboard | ||
| 311 | 14.9% | Sep 2024 | Official leaderboard | ||
| 312 | 14.7% | Mar 2025 | Official leaderboard | ||
| 313 | Granite-3.1-1B-A400M-Instruct | 13.3% | Dec 2024 | Official leaderboard | |
| 314 | Granite-3.1-1B-A400M-Base | 12.3% | Dec 2024 | Official leaderboard | |
| 315 | 11.9% | Sep 2024 | Official leaderboard | ||
| 316 | 11.9% | Jul 2024 | Official leaderboard | ||
| 317 | 11.4% | Oct 2024 | Official leaderboard | ||
| 318 | 11.2% | Jul 2024 | Official leaderboard | ||
| 319 | 10.9% | Jul 2024 | Official leaderboard | ||
| 320 | 10.8% | Oct 2024 | Official leaderboard |
The benchmark's own leaderboard has 240 entries, 212 of them models we track. Switch the scores above to rank them together.
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: Official MMLU-Pro leaderboard
10 models have held number one in labs’ own numbers since its release in June 2024 6 models have held number one across all independent runs since its release in June 2024 6 models have held number one with claims included since its release in June 2024
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Sources: Official MMLU-Pro leaderboard and labs’ model cards
O1 on the official leaderboard passed 85% in December 2024
That was 6 months after its release. By the same age, 15% of the benchmarks from 2024 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official MMLU-Pro leaderboard and labs’ model cards
11 labs put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Oct 28, 2026 | 74.1 | MMLU-Pro — Accuracy | |
| Oct 7, 2026 | 83.5 | MMLU-Pro | |
| Sep 13, 2026 | 89.8 | MMLU Pro | |
| Sep 10, 2026 | 74.1 | MMLU-Pro (EM) | |
| Aug 25, 2026 | Granite 4.2 30B | 77.6 | MMLU-Pro |
| Aug 12, 2026 | 30.7 | MMLU-Pro test | |
| Aug 11, 2026 | 81.9 | MMLU Pro | |
| Aug 11, 2026 | 86.3 | MMLU-Pro* (in-house) | |
| Jul 31, 2026 | 83.5 | MMLU-Pro | |
| Jul 22, 2026 | 86.2 | MMLU-Pro | |
| Jun 10, 2026 | 77.6 | MMLU Pro | |
| Jun 4, 2026 | 86.8 | MMLU-Pro | |
| Jun 3, 2026 | 77.2 | MMLU Pro | |
| Jun 2, 2026 | 85 | MMLU Pro | |
| Jun 1, 2026 | 88.5 | MMLU-Pro | |
| May 20, 2026 | 89.6 | MMLU-Pro | |
| May 9, 2026 | 84.3 | MMLU-Pro (acc) | |
| Apr 29, 2026 | Granite 4.1 30B | 64.1 | MMLU-Pro (5-shot, CoT) |
| Apr 24, 2026 | 86.2 | MMLU-Pro (EM), V4-Flash Max | |
| Apr 24, 2026 | 87.5 | MMLU-Pro (EM) | |
| Apr 22, 2026 | 86.2 | MMLU-Pro | |
| Apr 15, 2026 | 85.2 | MMLU-Pro | |
| Apr 9, 2026 | 83.3 | MMLU-Pro | |
| Apr 2, 2026 | 69.4 | MMLU Pro | |
| Apr 2, 2026 | 60 | MMLU Pro | |
| Apr 2, 2026 | 82.6 | MMLU Pro | |
| Apr 2, 2026 | 88.5 | MMLU-Pro | |
| Apr 2, 2026 | 85.2 | MMLU Pro | |
| Apr 1, 2026 | 83.4 | MMLU-Pro | |
| Mar 30, 2026 | 85.9 | MMLU-Pro | |
| Mar 24, 2026 | 81.2 | MMLU PRO (0-shot, CoT) | |
| Mar 19, 2026 | 79.8 | MMLU-Pro | |
| Mar 16, 2026 | 78 | MMLU Pro (reasoning; instruct = 73.5) | |
| Mar 11, 2026 | 83.7 | MMLU-Pro | |
| Mar 6, 2026 | Sarvam 30B | 80 | MMLU Pro |
| Mar 6, 2026 | Sarvam 105B | 81.7 | MMLU Pro |
| Mar 2, 2026 | 42.3 | MMLU-Pro (Thinking) | |
| Mar 2, 2026 | 66.5 | MMLU-Pro (Thinking) | |
| Mar 2, 2026 | 79.1 | MMLU-Pro | |
| Mar 2, 2026 | 82.5 | MMLU-Pro | |
| Feb 24, 2026 | 85.3 | MMLU-Pro | |
| Feb 24, 2026 | 86.1 | MMLU-Pro | |
| Feb 24, 2026 | 86.7 | MMLU-Pro | |
| Feb 16, 2026 | 87.8 | MMLU-Pro | |
| Feb 14, 2026 | 87 | MMLU-Pro | |
| Feb 6, 2026 | 83.8 | MMLU-Pro | |
| Feb 2, 2026 | 86.6 | MMLU-Pro | |
| Jan 27, 2026 | 75.2 | MMLU-Pro | |
| Jan 27, 2026 | Kimi K2.5 | 87.1 | MMLU-Pro |
| Jan 26, 2026 | 77 | MMLU-Pro | |
| Jan 23, 2026 | 85.7 | MMLU-Pro | |
| Jan 20, 2026 | LFM2.5-1.2B-Thinking | 49.6 | MMLU-Pro (± 0.18) |
| Jan 5, 2026 | 72.1 | MMLU-Pro | |
| Jan 5, 2026 | LFM2.5-1.2B-Instruct | 44.4 | MMLU-Pro |
| Jan 5, 2026 | 80.4 | MMLU-Pro | |
| Dec 31, 2025 | 83.8 | MMLU-Pro | |
| Dec 23, 2025 | 88 | MMLU-Pro | |
| Dec 22, 2025 | GLM-4.7 | 84.3 | MMLU-Pro |
| Dec 18, 2025 | 84.9 | MMLU-pro | |
| Dec 16, 2025 | MiMo-V2-Flash | 84.9 | MMLU-Pro |
| Dec 15, 2025 | 78.3 | MMLU-Pro | |
| Dec 3, 2025 | Hermes 4.3 36B | 80.7 | MMLU-Pro |
| Dec 2, 2025 | 80.7 | MMLU-Pro acc | |
| Dec 2, 2025 | 80.9 | MMLU-Pro acc | |
| Dec 2, 2025 | 81.6 | MMLU-Pro acc | |
| Dec 1, 2025 | 85 | MMLU-Pro (EM) | |
| Nov 26, 2025 | INTELLECT-3 | 81.9 | MMLU-Pro |
| Nov 19, 2025 | 84.7 | MMLU Pro | |
| Nov 6, 2025 | Kimi K2 Thinking | 84.6 | MMLU-Pro, no tools |
| Oct 31, 2025 | 82.7 | MMLU-Pro(acc) | |
| Oct 28, 2025 | granite-4.0-1b | 34 | MMLU-Pro (5-shot, CoT) |
| Oct 27, 2025 | 82 | MMLU-Pro (AA) | |
| Oct 7, 2025 | LFM2-8B-A1B | 37.4 | MMLU-Pro |
| Oct 2, 2025 | Granite 4.0 H Small | 55.5 | MMLU-Pro (5-shot, CoT) |
| Sep 29, 2025 | 85 | MMLU-Pro | |
| Sep 23, 2025 | 83.8 | MMLU-Pro | |
| Sep 22, 2025 | 85 | MMLU-Pro | |
| Sep 21, 2025 | 82.6 | MMLU-Pro(acc) | |
| Sep 11, 2025 | 82.7 | MMLU-Pro | |
| Aug 29, 2025 | 82.7 | MMLU-Pro(acc) | |
| Aug 26, 2025 | Hermes 4 405B | 80.5 | MMLU-Pro (reasoning) |
| Aug 25, 2025 | 81.3 | MMLU-Pro | |
| Aug 21, 2025 | 84.8 | MMLU-Pro (EM) | |
| Aug 20, 2025 | 82.7 | MMLU-Pro | |
| Jul 31, 2025 | 85.4 | MMLU-Pro (thinking) | |
| Jul 28, 2025 | GLM-4.5 | 84.6 | MMLU Pro |
| Jul 25, 2025 | 79.5 | MMLU Pro (CoT) | |
| Jul 25, 2025 | 84.4 | MMLU-Pro | |
| Jul 24, 2025 | 83.5 | MMLU-Pro | |
| Jul 15, 2025 | 81.8 | MMLU-Pro (reasoning) | |
| Jul 11, 2025 | Kimi K2 | 81.1 | MMLU-Pro, EM |
| Jul 10, 2025 | 76.5 | MMLU-Pro | |
| Jun 30, 2025 | 78.4 | MMLU-Pro | |
| Jun 26, 2025 | 50.6 | MMLU (Pro) 0-shot (IT) | |
| Jun 26, 2025 | Mercury | 69 | MMLU-Pro (% accuracy) |
| Jun 21, 2025 | Kimi-VL-A3B-Thinking-2506 | 68.5 | MMLU-Pro |
| Jun 20, 2025 | 69.1 | MMLU Pro (5-shot CoT) | |
| Jun 16, 2025 | 81.1 | MMLU-Pro | |
| May 28, 2025 | 85 | MMLU-Pro (EM) | |
| May 20, 2025 | 58.7 | MMLU-Pro | |
| May 7, 2025 | 77.2 | MMLU Pro 5-shot CoT | |
| Apr 30, 2025 | 74.3 | MMLUPro | |
| Apr 30, 2025 | 76 | MMLUPro | |
| Apr 29, 2025 | MiMo-7B-RL | 58.6 | MMLU-Pro (EM) |
| Apr 10, 2025 | 87 | MMLU-PRO | |
| Apr 8, 2025 | 78.5 | MMLU-Pro (thinking) | |
| Apr 5, 2025 | 74.3 | MMLU Pro, 0-shot, macro_avg/acc | |
| Apr 5, 2025 | 80.5 | MMLU Pro, 0-shot, macro_avg/acc | |
| Apr 5, 2025 | 82.2 | MMLU Pro | |
| Mar 27, 2025 | 47 | MMLU-Pro | |
| Mar 24, 2025 | 81.2 | MMLU-Pro | |
| Mar 21, 2025 | 87.2 | MMLU PRO | |
| Mar 17, 2025 | 66.8 | MMLU Pro (5-shot CoT) | |
| Mar 16, 2025 | MMLU-Pro | ||
| Mar 12, 2025 | 67.5 | MMLU (Pro) 0-shot (IT); PT MMLU (Pro COT) 5-shot = 52.2 | |
| Mar 10, 2025 | Reka Flash 3 | 65 | MMLU-Pro |
| Feb 27, 2025 | 79 | MMLU-pro | |
| Feb 26, 2025 | 52.8 | MMLU-Pro (0-shot, CoT) | |
| Feb 22, 2025 | Moonlight-16B-A3B | 42.4 | MMLU-pro |
| Feb 19, 2025 | Grok-3 mini | 78.9 | MMLU-pro [Grok 3 mini, non-reasoning table] |
| Feb 19, 2025 | Grok 3 | 79.9 | MMLU-pro [Grok 3 Beta, non-reasoning table] |
| Feb 5, 2025 | 71.6 | MMLU-Pro | |
| Feb 5, 2025 | 77.6 | MMLU-Pro | |
| Feb 5, 2025 | 79.1 | MMLU-Pro | |
| Jan 30, 2025 | 66.3 | mmlu_pro_5shot_cot_instruct | |
| Jan 28, 2025 | MMLU-Pro | ||
| Jan 20, 2025 | 84 | MMLU-Pro (EM) | |
| Jan 15, 2025 | 75.7 | MMLU-Pro* | |
| Jan 13, 2025 | 57.6 | MMLU-Pro(0-shot) | |
| Dec 26, 2024 | 75.9 | MMLU-Pro (EM) | |
| Dec 12, 2024 | 70.4 | MMLUPro | |
| Dec 11, 2024 | 76.4 | MMLU-Pro | |
| Dec 6, 2024 | 68.9 | MMLU Pro (CoT), 5-shot, macro_avg/acc | |
| Oct 22, 2024 | Claude 3.5 Sonnet (October 2024) | 78 | MMLU Pro (0-shot CoT) |
| Sep 19, 2024 | 71.1 | MMLU-Pro | |
| Aug 20, 2024 | 54.3 | MMLU-Pro (0-shot, CoT) | |
| Aug 13, 2024 | Grok-2 | 75.5 | MMLU-Pro (0-shot CoT) |
| Jul 23, 2024 | 73.3 | MMLU-Pro (5-shot, CoT) | |
| Jun 7, 2024 | 64.4 | MMLU-Pro |
103 system cards report it, from 28 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Oct 15, 2026 | | 76.3 | MMLU-Pro, exact match, Table 6 | Capabilities |
| Oct 15, 2026 | | 67.5 | MMLU-Pro, exact match, Table 6 | Capabilities |
| Oct 5, 2026 | Beam 1.0 Base technical report | 72.6 | MMLU-Pro, 5-shot CoT greedy, Table 23 | Capabilities |
| Sep 10, 2026 | | 74.1 | MMLU-Pro, EM, 5-shot | Capabilities |
| Aug 5, 2026 | | 83.5 | MMLU-Pro (Table 6, Reasoning mode) | Capabilities |
| Jul 31, 2026 | | 77.6 | MMLU-Pro, TD, 77.9 no-think (Table 3) | Capabilities |
| Jul 24, 2026 | | 69.4 | MMLU Pro, thinking (Table 5) | Capabilities |
| Jul 24, 2026 | | 60 | MMLU Pro, thinking (Table 5) | Capabilities |
| Jul 24, 2026 | | 85.2 | MMLU Pro, thinking (Table 5) | Capabilities |
| Jul 24, 2026 | | 82.6 | MMLU Pro, thinking (Table 5) | Capabilities |
| Jul 22, 2026 | | 86.2 | MMLU-Pro (Table 3) | Capabilities |
| Jun 9, 2026 | | 86.8 | MMLU-Pro (Table 10) | Capabilities |
| Jun 4, 2026 | | 79.1 | MMLU-Pro, 5-shot CoT EM | Capabilities |
| Jun 2, 2026 | | 85 | MMLU Pro (Table 12, rounded) | Capabilities |
| May 26, 2026 | | 81.8 | MMLU-Pro, AA Index v4.0 protocol (Table 4) | Capabilities |
| May 26, 2026 | | 85.2 | MMLU-Pro, AA Index v4.0 protocol (Table 4) | Capabilities |
| Apr 27, 2026 | | 77.3 | MMLU-Pro (Table 10) | Capabilities |
| Apr 26, 2026 | | 87.5 | MMLU-Pro (EM), DS-V4-Pro Max (Table 6). High 87.1, Non-Think 82.9 (Table 7) | Capabilities |
| Apr 26, 2026 | | 86.2 | MMLU-Pro (EM), DeepSeek-V4-Flash Max (Table 7). High 86.4, Non-Think 83.0 | Capabilities |
| Apr 17, 2026 | | 85.9 | MMLU-Pro (Table 4) | Capabilities |
| Apr 9, 2026 | | 83.3 | MMLU-Pro (Table 3, Reasoning mode) | Capabilities |
| Mar 26, 2026 | | 86.6 | MMLU-Pro, thinking (Table 2) | Capabilities |
| Mar 21, 2026 | | 84.9 | MMLU-pro (Table 1) | Capabilities |
| Mar 19, 2026 | | 79.8 | MMLU-Pro (Table 1) | Capabilities |
| Mar 11, 2026 | | 83.7 | MMLU-Pro (Table 5) | Capabilities |
| Feb 28, 2026 | | 80.5 | MMLU-Pro (Table 8) | Capabilities |
| Feb 19, 2026 | | 75.3 | MMLU-Pro, Trinity Large Preview (Table 4) | Capabilities |
| Feb 14, 2026 | | 87 | MMLU-Pro (Table 3) | Capabilities |
| Feb 11, 2026 | | 84.4 | MMLU-Pro (Table 5). PaCoRe 84.8 | Capabilities |
| Feb 4, 2026 | | 83.8 | MMLU-Pro (Table 2) | Capabilities |
| Jan 30, 2026 | Kimi K2.5 technical report | 87.1 | MMLU-Pro (Table 4) | Capabilities |
| Jan 14, 2026 | | 76 | MMLU-Pro, SeRe (Tables 2 and 3). PaCoRe 77.09 | Capabilities |
| Jan 11, 2026 | | 80.4 | MMLU-Pro (Table 6) | Capabilities |
| Jan 6, 2026 | MiMo-V2-Flash technical report | 84.9 | MMLU-Pro (Table 9) | Capabilities |
| Jan 5, 2026 | | 72.1 | MMLU-Pro (Table 6) | Capabilities |
| Jan 5, 2026 | | 83.8 | MMLU-Pro (Table 3, Reasoning mode). Non-reasoning 81.0 (Table 4) | Capabilities |
| Dec 23, 2025 | | 78.3 | MMLU-Pro (Table 3) | Capabilities |
| Dec 23, 2025 | | 65 | MMLU-Pro, 5-shot CoT EM, base model Table 2 | Capabilities |
| Dec 18, 2025 | INTELLECT-3 technical report | 81.9 | MMLU-Pro, Avg@1 (Table 2, Appendix A.1) | Capabilities |
| Dec 2, 2025 | | 80.7 | MMLU-Pro acc (Table 5) | Capabilities |
| Dec 2, 2025 | | 80.9 | MMLU-Pro acc (Table 1) | Capabilities |
| Dec 2, 2025 | | 81.6 | MMLU-Pro acc (Table 1) | Capabilities |
| Dec 1, 2025 | | 85 | MMLU-Pro (EM), DeepSeek-V3.2 Thinking (Table 2) | Capabilities |
| Nov 28, 2025 | LFM2-8B-A1B technical report | 37.4 | MMLU-Pro, LFM2-8B-A1B (Table 7) | Capabilities |
| Nov 28, 2025 | LFM2-1.2B technical report | 19.7 | MMLU-Pro, LFM2-1.2B (Table 6) | Capabilities |
| Nov 26, 2025 | | 83.8 | MMLU-Pro (Table 6) | Capabilities |
| Nov 6, 2025 | | 77.1 | MMLU-Pro-1000, a 1,000-sample subset, reasoning on, SFT stage 4 (Table 6) | Capabilities |
| Oct 31, 2025 | | 82.7 | MMLU-Pro (acc) (Table 13) | Capabilities |
| Oct 21, 2025 | | 80.5 | MMLU-Pro (Table 1) | Capabilities |
| Sep 30, 2025 | | 60.2 | MMLU-Pro, without reasoning (Table 13) | Capabilities |
| Sep 29, 2025 | | 85 | MMLU-Pro (EM), DeepSeek-V3.2-Exp (Table 1) | Capabilities |
| Sep 23, 2025 | | 82.6 | MMLU-Pro(acc), model-judged answer matching (Table 2) | Capabilities |
| Sep 1, 2025 | | 82.7 | MMLU-Pro (acc) (Table 3) | Capabilities |
| Aug 25, 2025 | | 81.3 | MMLU-Pro (Table 2) | Capabilities |
| Aug 25, 2025 | Hermes 4 405B technical report | 80.6 | MMLU-Pro, reasoning mode (Table 3). Non-reasoning 58.3. v1 printed 80.5 | Capabilities |
| Aug 21, 2025 | | 83.5 | MMLU-Pro (Table 2) | Capabilities |
| Aug 8, 2025 | GLM-4.5 technical report | 84.6 | MMLU Pro (Table 4) | Capabilities |
| Aug 8, 2025 | GLM-4.5-Air technical report | 81.4 | MMLU Pro (Table 4) | Capabilities |
| Jul 30, 2025 | | 58.7 | MMLU-Pro, logprobs 5-shot (Table 24) | Capabilities |
| Jul 21, 2025 | Kimi K2 technical report | 81.1 | MMLU-Pro (EM) (Table 3) | Capabilities |
| Jul 15, 2025 | | 81.8 | MMLU-Pro (Table 3, Reasoning). Non-reasoning 77.6 (Table 4) | Capabilities |
| Jun 29, 2025 | | 78.4 | MMLU-Pro (Table 5) | Capabilities |
| Jun 16, 2025 | | 81.1 | MMLU-Pro (Table 2) | Capabilities |
| May 12, 2025 | MiMo-7B-RL technical report | 58.6 | MMLU-Pro (EM) (Table 4) | Capabilities |
| Apr 30, 2025 | | 74.3 | MMLUPro (Table 2) | Capabilities |
| Apr 30, 2025 | | 76 | MMLUPro (Table 2) | Capabilities |
| Apr 15, 2025 | | 77.6 | MMLU-Pro (Table, p.3, column Gemini 2.0 Flash (GA)) | Capabilities |
| Apr 10, 2025 | | 71.6 | MMLU-Pro (Table, p.3, column Gemini 2.0 Flash-Lite (Public Preview)) | Capabilities |
| Apr 10, 2025 | | 87 | MMLU-PRO (Table 2) | Capabilities |
| Apr 5, 2025 | | 74.3 | MMLU Pro, 0-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) | Capabilities |
| Apr 5, 2025 | | 80.5 | MMLU Pro, 0-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) | Capabilities |
| Apr 1, 2025 | | 69.6 | MMLU-Pro, 5-shot CoT (Table 3) | Capabilities |
| Mar 26, 2025 | | 47 | MMLU-Pro (Table 1) | Capabilities |
| Mar 25, 2025 | | 43.6 | MMLU-Pro, IT, zero-shot (Table 6) | Capabilities |
| Mar 25, 2025 | | 67.5 | MMLU-Pro, IT, zero-shot (Table 6) | Capabilities |
| Mar 25, 2025 | | 14.7 | MMLU-Pro, IT, zero-shot (Table 6) | Capabilities |
| Mar 25, 2025 | | 60.6 | MMLU-Pro, IT, zero-shot (Table 6) | Capabilities |
| Mar 16, 2025 | | 74 | MMLU-Pro, accuracy (Table 3) | Capabilities |
| Mar 3, 2025 | | 52.8 | MMLU-Pro (0-Shot, CoT) (Table 7) | Capabilities |
| Feb 19, 2025 | | 71.2 | MMLU-Pro (Table 4) | Capabilities |
| Jan 26, 2025 | | 63.3 | MMLU-Pro (Table 6) | Capabilities |
| Jan 20, 2025 | | 84 | MMLU-Pro (EM), DeepSeek-R1 (Table 8, same as final column of Table 3) | Capabilities |
| Jan 14, 2025 | | 75.7 | MMLU-Pro, 0-shot CoT (Table 8) | Capabilities |
| Dec 31, 2024 | | 52.7 | MMLU Pro, 0-shot CoT, unseen suite (Table 24) | Capabilities |
| Dec 26, 2024 | | 75.9 | MMLU-Pro (EM), DeepSeek-V3 (Table 6) | Capabilities |
| Dec 19, 2024 | | 45 | MMLU-Pro 45.0 (base model, Table 8. 56.3 is the instruction-tuned model) | Capabilities |
| Dec 19, 2024 | | 71.1 | MMLU-Pro, Qwen2.5-72B-Instruct (Table 6) | Capabilities |
| Dec 19, 2024 | | 69 | MMLU-Pro, Qwen2.5-32B-Instruct (Table 7) | Capabilities |
| Dec 19, 2024 | | 63.7 | MMLU-Pro, Qwen2.5-14B-Instruct (Table 7) | Capabilities |
| Dec 19, 2024 | | 64.5 | MMLU-Pro, Qwen2.5-Turbo (Table 7) | Capabilities |
| Dec 12, 2024 | | 70.4 | MMLUPro, internal framework (Table 1) | Capabilities |
| Dec 6, 2024 | | 68.9 | MMLU Pro (CoT), 5-shot, macro_avg/acc (Instruction tuned models table) | Capabilities |
| Nov 23, 2024 | | 48.3 | MMLU-Pro (5-shot, CoT), post-trained (Table 2) | Capabilities |
| Nov 23, 2024 | | 66.4 | MMLU-Pro (5-shot, CoT), post-trained (Table 2) | Capabilities |
| Nov 23, 2024 | | 73.3 | MMLU-Pro (5-shot, CoT), post-trained (Table 2) | Capabilities |
| Oct 22, 2024 | Claude 3.5 Sonnet (October 2024) model card | 78 | MMLU Pro, 0-shot CoT (Table 7) | Capabilities |
| Oct 22, 2024 | Claude 3.5 Haiku model card | 65 | MMLU Pro, 0-shot CoT (Table 8) | Capabilities |
| Aug 30, 2024 | | 54.3 | MMLU-Pro (0-shot, CoT) (Table 3) | Capabilities |
| Aug 30, 2024 | | 47.5 | MMLU-Pro (0-shot, CoT) (Table 3) | Capabilities |
| Jul 23, 2024 | | 48.3 | MMLU-Pro (CoT), 5-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) | Capabilities |
| Jul 23, 2024 | | 66.4 | MMLU-Pro (CoT), 5-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) | Capabilities |
| Jul 23, 2024 | | 73.3 | MMLU-Pro (CoT), 5-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) | Capabilities |
| Jul 15, 2024 | | 64.4 | MMLU-Pro, Qwen2-72B-Instruct (Table 6) | Capabilities |
The lab's number comes out ahead in 8 of 13 pairs that differ
Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 49 of them match, which usually means one side copied the other. Every benchmark.
| Model | Lab says | Board says | Gap | What differs |
|---|---|---|---|---|
| | 42.3% MMLU-Pro (Thinking) | 29.7% Leaderboard | +13 pts | Qwen reports 42.3 with thinking enabled. The board's 29.7 matches the card's nonthinking result. |
| | 66.5% MMLU-Pro (Thinking) | 55.3% Leaderboard | +11 pts | Qwen reports 66.5% with thinking enabled. The board lists the self-reported 55.3% result from the model card's non-thinking mode. |
| | 85.2% MMLU-Pro, AA Index v4.0 protocol (Table 4) | 80.1% Leaderboard | +5.1 pts | |
| Grok-3 mini | 78.9% MMLU-pro [Grok 3 mini, non-reasoning table] | 83% Leaderboard | −4.1 pts | |
| | 48.3% MMLU-Pro (CoT), 5-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) | 44.3% Leaderboard | +4.0 pts | |
| | 66.4% MMLU-Pro (CoT), 5-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) | 62.8% Leaderboard | +3.6 pts | |
| | 68.9% MMLU Pro (CoT), 5-shot, macro_avg/acc | 65.9% Leaderboard | +3.0 pts | |
| Claude 3.5 Haiku | 65% MMLU Pro, 0-shot CoT (Table 8) | 62.1% Leaderboard | +2.9 pts | |
| | 85% MMLU-Pro (EM) | 84% Leaderboard | +1.0 pts | |
| | 82.2% MMLU Pro | 82.8% Leaderboard | −0.6 pts | |
| | 71.1% MMLU-Pro | 71.6% Leaderboard | −0.5 pts | |
| | 47.5% MMLU-Pro (0-shot, CoT) (Table 3) | 47.9% Leaderboard | −0.4 pts | |
| | 69% MMLU-Pro, Qwen2.5-32B-Instruct (Table 7) | 69.2% Leaderboard | −0.2 pts | |
| | 81.2% MMLU-Pro | 81.3% Leaderboard | Same | |
| Kimi K2 Jul 2025 | 81.1% MMLU-Pro, EM | 81% Leaderboard | Same | |
| | 84.4% MMLU-Pro | 84.5% Leaderboard | Same | |
| | 65% MMLU-Pro, 5-shot CoT EM, base model Table 2 | 65.1% Leaderboard | Same | |
| | 66.8% MMLU Pro (5-shot CoT) | 66.8% Leaderboard | Same | |
| | 64.4% MMLU-Pro | 64.4% Leaderboard | Same | |
| | 82.7% MMLU-Pro(acc) | 82.7% Leaderboard | Same | |
| | 63.7% MMLU-Pro, Qwen2.5-14B-Instruct (Table 7) | 63.7% Leaderboard | Same | |
| | 73.3% MMLU-Pro (5-shot, CoT) | 73.3% Leaderboard | Same | |
| Claude 3.5 Sonnet (October 2024) | 78% MMLU Pro (0-shot CoT) | 78% Leaderboard | Same | |
| | 70.4% MMLUPro | 70.4% Leaderboard | Same | |
| | 57.6% MMLU-Pro(0-shot) | 57.6% Leaderboard | Same | |
| | 75.7% MMLU-Pro* | 75.7% Leaderboard | Same | |
| | 79.1% MMLU-Pro | 79.1% Leaderboard | Same | |
| | 77.6% MMLU-Pro | 77.6% Leaderboard | Same | |
| | 71.6% MMLU-Pro | 71.6% Leaderboard | Same | |
| Grok 3 | 79.9% MMLU-pro [Grok 3 Beta, non-reasoning table] | 79.9% Leaderboard | Same | |
| | 52.8% MMLU-Pro (0-shot, CoT) | 52.8% Leaderboard | Same | |
| | 79% MMLU-pro | 79% Leaderboard | Same | |
| | 67.5% MMLU (Pro) 0-shot (IT); PT MMLU (Pro COT) 5-shot = 52.2 | 67.5% Leaderboard | Same | |
| | 87.2% MMLU PRO | 87.2% Leaderboard | Same | |
| | 80.5% MMLU Pro, 0-shot, macro_avg/acc | 80.5% Leaderboard | Same | |
| | 74.3% MMLU Pro, 0-shot, macro_avg/acc | 74.3% Leaderboard | Same | |
| | 87% MMLU-PRO | 87% Leaderboard | Same | |
| MiMo-7B-RL | 58.6% MMLU-Pro (EM) | 58.6% Leaderboard | Same | |
| | 76% MMLUPro | 76% Leaderboard | Same | |
| | 74.3% MMLUPro | 74.3% Leaderboard | Same | |
| | 81.1% MMLU-Pro | 81.1% Leaderboard | Same | |
| | 78.4% MMLU-Pro | 78.4% Leaderboard | Same | |
| | 83.5% MMLU-Pro | 83.5% Leaderboard | Same | |
| GLM-4.5 | 84.6% MMLU Pro | 84.6% Leaderboard | Same | |
| | 82.7% MMLU-Pro | 82.7% Leaderboard | Same | |
| | 84.8% MMLU-Pro (EM) | 84.8% Leaderboard | Same | |
| | 82% MMLU-Pro (AA) | 82% Leaderboard | Same | |
| | 85% MMLU-Pro (EM) | 85% Leaderboard | Same | |
| | 78.3% MMLU-Pro | 78.3% Leaderboard, bf16 | Same | |
| | 88% MMLU-Pro | 88% Leaderboard | Same | |
| | 87% MMLU-Pro | 87% Leaderboard | Same | |
| | 87.8% MMLU-Pro | 87.8% Leaderboard | Same | |
| | 86.7% MMLU-Pro | 86.7% Leaderboard | Same | |
| | 86.1% MMLU-Pro | 86.1% Leaderboard | Same | |
| | 85.3% MMLU-Pro | 85.3% Leaderboard | Same | |
| | 82.5% MMLU-Pro | 82.5% Leaderboard | Same | |
| | 79.1% MMLU-Pro | 79.1% Leaderboard | Same | |
| | 45% MMLU-Pro 45.0 (base model, Table 8. 56.3 is the instruction-tuned model) | 45% Leaderboard | Same | |
| | 60.6% MMLU-Pro, IT, zero-shot (Table 6) | 60.6% Leaderboard | Same | |
| | 14.7% MMLU-Pro, IT, zero-shot (Table 6) | 14.7% Leaderboard | Same | |
| | 43.6% MMLU-Pro, IT, zero-shot (Table 6) | 43.6% Leaderboard | Same | |
| GLM-4.5-Air | 81.4% MMLU Pro (Table 4) | 81.4% Leaderboard | Same |
One logged change that moves scores
Changes by the maintainers, so a score from before a change may not compare with one from after it. Newest first, each with the maintainers' own source. The list may be incomplete. What we log.
- Questions corrected January 18th, 2026
The maintainers removed inconsistent leading spaces from answer options in STEM subjects. This correction removed a formatting shortcut that could inflate earlier scores. Source
Versions
More in Knowledge
- MMLU In use 57-subject multiple-choice exam questions from elementary to professional level. Scored by accuracy.
Seed1.8 92.3%
- SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy.
Qwen 3.6 Max (Preview) 73.9%
- WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1.
Tencent Hy4 preview 83.9%
- BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly.
Gemma 4 31B IT 74.4%
- HLE-Diamond In use Models answer 1,000 cleaned expert questions from Humanity's Last Exam, 500 on reasoning and 500 on knowledge, closed book. Graded on exact answers. Claude Fable 5.2 77.9%
- HLE-Verified In use Verified and revised version of Humanity's Last Exam (1,811 items) with uncertain items removed.
Gemini 3 Pro 48.2%
Data audit
All 9 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (240 entries), launch posts (137 scores) and system cards (103 scores).
- Pass Models scored 320 models we track, plus 28 other entries on its leaderboard.
- Pass Newest score The newest is from October 2026, for NVIDIA Nemotron 3.5 Transcribe 30B-A4B.
- Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28).
- Note Headroom Saturated. The best is 91.2%, past 85% since December 2024.
Leaderboard import 3 pass
- Pass Importer We read the board on huggingface.co: 233 rows for models we track and 28 other entries at the last read.
- Pass Last read November 19th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims 2 large gaps of 62 pairs, each explained in our notes.
- Pass Known issues No published audit found problems.