Benchmark Almanac

MMLU-Pro

Intern-S2-397B 89.8% leads since September 2026
Gemini 3.1 Pro 91.2% leads since February 2026
Gemini 3.1 Pro 91.2% leads since February 2026
In use

The official leaderboard hasn’t scored a model released after GPT-5.4 (March 2026). Labs report their own numbers for 42 newer models, such as Intern-S2-397B at 89.8%.

Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects.

Launch posts report it widely. The makers fixed answers and formatting many times from 2024-07 to 2026-01, including a leading-space shortcut in STEM options on January 18th, 2026, and kept one dataset.

Released
June 2024
Kind
Exam questions
Level
University
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Text
Interaction
Single turn
Items from
Another benchmark
Refresh
One fixed set
Run by
Anyone
Based on
MMLU
Strongest model tested
GPT-5.4 #28 on the capability index
Last in a launch post
October 2026
Last model added
October 2026
Scores refreshed
November 19th, 2026

In the authors’ words

In the age of large-scale language models, benchmarks like the Massive Multitask Language Understanding (MMLU) have been pivotal in pushing the boundaries of what AI can achieve in language comprehension and reasoning across diverse domains. However, as models continue to improve, their performance on these benchmarks has begun to plateau, making it increasingly difficult to discern differences in model capabilities. This paper introduces MMLU-Pro, an enhanced dataset designed to extend the mostly knowledge-driven MMLU benchmark by integrating more challenging, reasoning-focused questions and expanding the choice set from four to ten options. Additionally, MMLU-Pro eliminates the trivial and noisy questions in MMLU. Our experimental results show that MMLU-Pro not only raises the challenge, causing a significant drop in accuracy by 16% to 33% compared to MMLU but also demonstrates greater stability under varying prompts. With 24 different prompt styles tested, the sensitivity of model scores to prompt variations decreased from 4-5% in MMLU to just 2% in MMLU-Pro. Additionally, we found that models utilizing Chain of Thought (CoT) reasoning achieved better performance on MMLU-Pro compared to direct answering, which is in stark contrast to the findings on the original MMLU, indicating that MMLU-Pro includes more complex reasoning questions. Our assessments confirm that MMLU-Pro is a more discriminative benchmark to better track progress in the field.

Abstract of MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark (arXiv:2406.01574, CC0)
170
Models scored in lab cards
139
Launch posts list it, from 33 labs
103
System cards report it, from 28 labs

Intern-S2-397B leads with 89.8%

Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.

Gemini 3.1 Pro leads with 91.2% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.

Gemini 3.1 Pro leads with 91.2%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.

170 models. Scroll sideways for the rest.

  1. 89.8 Intern-S2-397B
  2. 89.6 Qwen3.7-Max
  3. 88.5 Qwen 3.6 Plus
  4. 88.5 Qwen3.7-Plus
  5. 88.0 MiniMax-M2.1
  6. 87.8 Qwen3.5 397B-A17B
  7. 87.5 DeepSeek-V4-Pro
  8. 87.2 Hunyuan-T1
  9. 87.1 Kimi K2.5
  10. 87.0 Seed-Thinking-v1.5
  11. 87.0 Seed2.0 Pro
  12. 86.8 Nemotron 3 Ultra
  13. 86.7 Qwen3.5-122B-A10B
  14. 86.6 Intern-S1-Pro
  15. 86.3 Solar Pro 4
  16. 86.2 Qwen3.6 27B
  17. 86.2 DeepSeek-V4-Flash
  18. 86.2 Solar Open 2
  19. 86.1 Qwen3.5-27B
  20. 85.9 Qwen3.5-Omni-Plus
  21. 85.7 Qwen3-Max-2026-01-23
  22. 85.4 Cogito v2 Preview 671B MoE
  23. 85.3 Qwen3.5-35B-A3B
  24. 85.2 Gemma 4 31B IT
  25. 85.2 Qwen 3.6 35B-A3B
  26. 85.2 MiniMax-M2.5
  27. 85.0 DeepSeek-R1
  28. 85.0 DeepSeek-V3.1-Terminus
  29. 85.0 DeepSeek-V3.2-Exp
  30. 85.0 DeepSeek-V3.2
  31. 85.0 MAI-Thinking-1
  32. 84.9 MiMo-V2-Flash
  33. 84.9 Seed1.8
  34. 84.8 DeepSeek-V3.1
  35. 84.7 Cogito v2.1
  36. 84.6 GLM-4.5
  37. 84.6 Kimi K2 Thinking
  38. 84.4 Qwen3-235B-A22B-Thinking
  39. 84.4 Step 3.5 Flash
  40. 84.3 GLM-4.7
  41. 84.3 ERNIE 5.1
  42. 84.0 DeepSeek-R1
  43. 83.8 Qwen3-VL-235B-A22B
  44. 83.8 K-EXAONE
  45. 83.8 ERNIE 5.0
  46. 83.7 Nemotron 3 Super
  47. 83.5 Intern-S1
  48. 83.5 K-EXAONE 2.0
  49. 83.5 MiniCPM-V-4.7-35B-A3B
  50. 83.4 Trinity Large Thinking
  51. 83.3 EXAONE 4.5 33B
  52. 82.7 LongCat-Flash-Omni
  53. 82.7 seed-oss-36b-instruct
  54. 82.7 Qwen3-Next-80B-A3B
  55. 82.7 LongCat-Flash-Chat
  56. 82.6 LongCat-Flash-Thinking
  57. 82.6 Gemma 4 26B A4B
  58. 82.5 Qwen3.5-9B
  59. 82.2 Llama 4 Behemoth
  60. 82.0 MiniMax-M2
  61. 81.9 Nemotron 3.5 Lightning 30B-A3B
  62. 81.9 INTELLECT-3
  63. 81.8 EXAONE 4.0 32B
  64. 81.8 MiniMax-M2.7
  65. 81.7 Sarvam 105B
  66. 81.6 Nova 2.0 Pro Preview
  67. 81.4 GLM-4.5-Air
  68. 81.3 InternVL3.5-241B-A28B
  69. 81.2 Solar Pro 3 (March 2026 update)
  70. 81.2 DeepSeek-V3
  71. 81.1 MiniMax-M1-80k
  72. 81.1 Kimi K2
  73. 80.9 Nova 2 Lite
  74. 80.7 Nova 2 Omni Preview
  75. 80.7 Hermes 4.3 36B
  76. 80.6 Hermes 4 405B
  77. 80.5 Ring-1T
  78. 80.5 Qwen3-Coder-Next
  79. 80.5 Llama 4 Maverick
  80. 80.4 Solar Open 100B
  81. 80.0 Sarvam 30B
  82. 79.9 Grok 3
  83. 79.8 Nemotron-Cascade 2 30B-A3B
  84. 79.5 Llama-3.3-Nemotron-Super-49B-v1.5
  85. 79.1 Gemini 2.0 Pro
  86. 79.1 Qwen3.5-4B
  87. 79.1 Nemotron 3 Ultra 550B-A55B Base
  88. 79.0 Hunyuan-TurboS
  89. 78.9 Grok-3 mini
  90. 78.5 Cogito v1 Preview 70B
  91. 78.4 ERNIE-4.5-300B-A47B
  92. 78.3 Nemotron 3 Nano 30B-A3B
  93. 78.0 Claude 3.5 Sonnet (October 2024)
  94. 78.0 Mistral Small 4
  95. 77.6 Gemini 2.0 Flash
  96. 77.6 DiffusionGemma 26B A4B
  97. 77.6 Granite 4.2 30B
  98. 77.3 Nemotron 3 Nano Omni 30B-A3B
  99. 77.2 Mistral Medium 3
  100. 77.2 Gemma 4 12B
  101. 77.1 Nemotron Nano 12B v2 VL
  102. 77.0 Solar Pro 3
  103. 76.5 Solar Pro 2
  104. 76.4 Gemini 2.0 Flash
  105. 76.3 Nemotron 3.5 Super 120B-A12B Base
  106. 76.0 STEP3-VL-10B
  107. 76.0 Phi-4-Reasoning-plus
  108. 75.9 DeepSeek-V3
  109. 75.7 MiniMax-Text-01
  110. 75.5 Grok-2
  111. 75.3 Trinity Large Preview
  112. 74.3 Llama 4 Scout
  113. 74.3 Phi-4-reasoning
  114. 74.1 NVIDIA Nemotron 3.5 Transcribe 30B-A4B
  115. 74.1 DeepSeek V4.1 Flash
  116. 74.1 DeepSeek-V4.1-Flash Base
  117. 74.0 EXAONE Deep 32B
  118. 73.3 Llama 3.1-405B
  119. 72.6 Beam 1.0 Base
  120. 72.1 Falcon H1R 7B
  121. 71.6 Gemini 2.0 Flash-Lite
  122. 71.2 Qwen2.5-VL-72B
  123. 71.1 Qwen2.5-72B
  124. 70.4 Phi-4
  125. 69.6 Cohere Command A
  126. 69.4 Gemma 4 E4B
  127. 69.1 Mistral Small 3.2
  128. 69.0 Qwen2.5-32B
  129. 69.0 Mercury
  130. 68.9 Llama 3.3 70B Instruct
  131. 68.5 Kimi-VL-A3B-Thinking-2506
  132. 67.5 Gemma 3 27B
  133. 67.5 Nemotron 3.5 Lightning 30B-A3B Base
  134. 66.8 Mistral Small 3.1
  135. 66.5 Qwen3.5-2B
  136. 66.4 Llama 3.1-70B
  137. 66.3 Mistral Small 3
  138. 65.0 Nemotron 3 Nano 30B-A3B Base
  139. 65.0 Claude 3.5 Haiku
  140. 65.0 Reka Flash 3
  141. 64.5 Qwen-Turbo
  142. 64.4 Qwen2-72B
  143. 64.1 Granite 4.1 30B
  144. 63.7 Qwen2.5-14B
  145. 63.3 Qwen2.5-14B-Instruct-1M
  146. 60.6 Gemma 3 12B
  147. 60.2 Code World Model (CWM)
  148. 60.0 Gemma 4 E2B
  149. 58.7 Falcon-H1-34B
  150. 58.6 MiMo-7B-RL
  151. 57.6 InternLM3-8B-Instruct
  152. 55.5 Granite 4.0 H Small
  153. 54.3 Phi-3.5-MoE
  154. 52.8 Phi-4 Mini
  155. 52.7 OLMo 2 32B Instruct
  156. 50.6 Gemma 3n E4B
  157. 49.6 LFM2.5-1.2B-Thinking
  158. 48.3 Llama 3.1-8B
  159. 47.5 Phi-3.5-mini
  160. 47.0 Qwen2.5-Omni-7B
  161. 45.0 Qwen2.5-7B
  162. 44.4 LFM2.5-1.2B-Instruct
  163. 43.6 Gemma 3 4B
  164. 42.4 Moonlight-16B-A3B
  165. 42.3 Qwen3.5-0.8B
  166. 37.4 LFM2-8B-A1B
  167. 34.0 granite-4.0-1b
  168. 30.7 North Micro Vision Instruct
  169. 19.7 LFM2-1.2B
  170. 14.7 Gemma 3 1B
  1. 1 Intern-S2-397B 89.8%
  2. 2 Qwen3.7-Max 89.6%
  3. 3 Qwen 3.6 Plus 88.5%
  4. 4 Qwen3.7-Plus 88.5%
  5. 5 MiniMax-M2.1 88%
  6. 6 Qwen3.5 397B-A17B 87.8%
  7. 7 DeepSeek-V4-Pro 87.5%
  8. 8 Hunyuan-T1 87.2%
  9. 9 Kimi K2.5 87.1%
  10. 10 Seed-Thinking-v1.5 87%
  11. 11 Seed2.0 Pro 87%
  12. 12 Nemotron 3 Ultra 86.8%
  13. 13 Qwen3.5-122B-A10B 86.7%
  14. 14 Intern-S1-Pro 86.6%
  15. 15 Solar Pro 4 86.3%
  16. 16 Qwen3.6 27B 86.2%
  17. 17 DeepSeek-V4-Flash 86.2%
  18. 18 Solar Open 2 86.2%
  19. 19 Qwen3.5-27B 86.1%
  20. 20 Qwen3.5-Omni-Plus 85.9%
  21. 21 Qwen3-Max-2026-01-23 85.7%
  22. 22 Cogito v2 Preview 671B MoE 85.4%
  23. 23 Qwen3.5-35B-A3B 85.3%
  24. 24 Gemma 4 31B IT 85.2%
  25. 25 Qwen 3.6 35B-A3B 85.2%
  26. 26 MiniMax-M2.5 85.2%
  27. 27 DeepSeek-R1 May 2025 85%
  28. 28 DeepSeek-V3.1-Terminus 85%
  29. 29 DeepSeek-V3.2-Exp 85%
  30. 30 DeepSeek-V3.2 85%
  31. 31 MAI-Thinking-1 85%
  32. 32 MiMo-V2-Flash 84.9%
  33. 33 Seed1.8 84.9%
  34. 34 DeepSeek-V3.1 84.8%
  35. 35 Cogito v2.1 84.7%
  36. 36 GLM-4.5 84.6%
  37. 37 Kimi K2 Thinking 84.6%
  38. 38 Qwen3-235B-A22B-Thinking Jul 2025 84.4%
  39. 39 Step 3.5 Flash 84.4%
  40. 40 GLM-4.7 84.3%
  41. 41 ERNIE 5.1 84.3%
  42. 42 DeepSeek-R1 84%
  43. 43 Qwen3-VL-235B-A22B 83.8%
  44. 44 K-EXAONE 83.8%
  45. 45 ERNIE 5.0 83.8%
  46. 46 Nemotron 3 Super 83.7%
  47. 47 Intern-S1 83.5%
  48. 48 K-EXAONE 2.0 83.5%
  49. 49 MiniCPM-V-4.7-35B-A3B 83.5%
  50. 50 Trinity Large Thinking 83.4%
  51. 51 EXAONE 4.5 33B 83.3%
  52. 52 LongCat-Flash-Omni 82.7%
  53. 53 seed-oss-36b-instruct 82.7%
  54. 54 Qwen3-Next-80B-A3B 82.7%
  55. 55 LongCat-Flash-Chat 82.7%
  56. 56 LongCat-Flash-Thinking 82.6%
  57. 57 Gemma 4 26B A4B 82.6%
  58. 58 Qwen3.5-9B 82.5%
  59. 59 Llama 4 Behemoth 82.2%
  60. 60 MiniMax-M2 82%
  61. 61 Nemotron 3.5 Lightning 30B-A3B 81.9%
  62. 62 INTELLECT-3 81.9%
  63. 63 EXAONE 4.0 32B 81.8%
  64. 64 MiniMax-M2.7 81.8%
  65. 65 Sarvam 105B 81.7%
  66. 66 Nova 2.0 Pro Preview 81.6%
  67. 67 GLM-4.5-Air 81.4%
  68. 68 InternVL3.5-241B-A28B 81.3%
  69. 69 Solar Pro 3 (March 2026 update) 81.2%
  70. 70 DeepSeek-V3 Mar 2025 81.2%
  71. 71 MiniMax-M1-80k 81.1%
  72. 72 Kimi K2 Jul 2025 81.1%
  73. 73 Nova 2 Lite 80.9%
  74. 74 Nova 2 Omni Preview 80.7%
  75. 75 Hermes 4.3 36B 80.7%
  76. 76 Hermes 4 405B 80.6%
  77. 77 Ring-1T 80.5%
  78. 78 Qwen3-Coder-Next 80.5%
  79. 79 Llama 4 Maverick 80.5%
  80. 80 Solar Open 100B 80.4%
  81. 81 Sarvam 30B 80%
  82. 82 Grok 3 79.9%
  83. 83 Nemotron-Cascade 2 30B-A3B 79.8%
  84. 84 Llama-3.3-Nemotron-Super-49B-v1.5 79.5%
  85. 85 Gemini 2.0 Pro 79.1%
  86. 86 Qwen3.5-4B 79.1%
  87. 87 Nemotron 3 Ultra 550B-A55B Base 79.1%
  88. 88 Hunyuan-TurboS 79%
  89. 89 Grok-3 mini 78.9%
  90. 90 Cogito v1 Preview 70B 78.5%
  91. 91 ERNIE-4.5-300B-A47B 78.4%
  92. 92 Nemotron 3 Nano 30B-A3B 78.3%
  93. 93 Claude 3.5 Sonnet (October 2024) 78%
  94. 94 Mistral Small 4 78%
  95. 95 Gemini 2.0 Flash Feb 2025 77.6%
  96. 96 DiffusionGemma 26B A4B 77.6%
  97. 97 Granite 4.2 30B 77.6%
  98. 98 Nemotron 3 Nano Omni 30B-A3B 77.3%
  99. 99 Mistral Medium 3 77.2%
  100. 100 Gemma 4 12B 77.2%
  101. 101 Nemotron Nano 12B v2 VL 77.1%
  102. 102 Solar Pro 3 77%
  103. 103 Solar Pro 2 76.5%
  104. 104 Gemini 2.0 Flash Dec 2024 76.4%
  105. 105 Nemotron 3.5 Super 120B-A12B Base 76.3%
  106. 106 STEP3-VL-10B 76%
  107. 107 Phi-4-Reasoning-plus 76%
  108. 108 DeepSeek-V3 75.9%
  109. 109 MiniMax-Text-01 75.7%
  110. 110 Grok-2 Aug 2024 75.5%
  111. 111 Trinity Large Preview 75.3%
  112. 112 Llama 4 Scout 74.3%
  113. 113 Phi-4-reasoning 74.3%
  114. 114 NVIDIA Nemotron 3.5 Transcribe 30B-A4B 74.1%
  115. 115 DeepSeek V4.1 Flash 74.1%
  116. 116 DeepSeek-V4.1-Flash Base 74.1%
  117. 117 EXAONE Deep 32B 74%
  118. 118 Llama 3.1-405B 73.3%
  119. 119 Beam 1.0 Base 72.6%
  120. 120 Falcon H1R 7B 72.1%
  121. 121 Gemini 2.0 Flash-Lite 71.6%
  122. 122 Qwen2.5-VL-72B 71.2%
  123. 123 Qwen2.5-72B 71.1%
  124. 124 Phi-4 70.4%
  125. 125 Cohere Command A 69.6%
  126. 126 Gemma 4 E4B 69.4%
  127. 127 Mistral Small 3.2 69.1%
  128. 128 Qwen2.5-32B 69%
  129. 129 Mercury 69%
  130. 130 Llama 3.3 70B Instruct 68.9%
  131. 131 Kimi-VL-A3B-Thinking-2506 68.5%
  132. 132 Gemma 3 27B 67.5%
  133. 133 Nemotron 3.5 Lightning 30B-A3B Base 67.5%
  134. 134 Mistral Small 3.1 66.8%
  135. 135 Qwen3.5-2B 66.5%
  136. 136 Llama 3.1-70B 66.4%
  137. 137 Mistral Small 3 66.3%
  138. 138 Nemotron 3 Nano 30B-A3B Base 65%
  139. 139 Claude 3.5 Haiku 65%
  140. 140 Reka Flash 3 65%
  141. 141 Qwen-Turbo 64.5%
  142. 142 Qwen2-72B 64.4%
  143. 143 Granite 4.1 30B 64.1%
  144. 144 Qwen2.5-14B 63.7%
  145. 145 Qwen2.5-14B-Instruct-1M 63.3%
  146. 146 Gemma 3 12B 60.6%
  147. 147 Code World Model (CWM) 60.2%
  148. 148 Gemma 4 E2B 60%
  149. 149 Falcon-H1-34B 58.7%
  150. 150 MiMo-7B-RL 58.6%
  151. 151 InternLM3-8B-Instruct 57.6%
  152. 152 Granite 4.0 H Small 55.5%
  153. 153 Phi-3.5-MoE 54.3%
  154. 154 Phi-4 Mini 52.8%
  155. 155 OLMo 2 32B Instruct 52.7%
  156. 156 Gemma 3n E4B 50.6%
  157. 157 LFM2.5-1.2B-Thinking 49.6%
  158. 158 Llama 3.1-8B 48.3%
  159. 159 Phi-3.5-mini 47.5%
  160. 160 Qwen2.5-Omni-7B 47%
  161. 161 Qwen2.5-7B 45%
  162. 162 LFM2.5-1.2B-Instruct 44.4%
  163. 163 Gemma 3 4B 43.6%
  164. 164 Moonlight-16B-A3B 42.4%
  165. 165 Qwen3.5-0.8B 42.3%
  166. 166 LFM2-8B-A1B 37.4%
  167. 167 granite-4.0-1b 34%
  168. 168 North Micro Vision Instruct 30.7%
  169. 169 LFM2-1.2B 19.7%
  170. 170 Gemma 3 1B 14.7%

Source: Labs’ model cards

212 models. Scroll sideways for the rest.

  1. 91.2 Gemini 3.1 Pro
  2. 90.1 Gemini 3 Pro
  3. 89.3 o1
  4. 89.1 Claude Opus 4.6
  5. 88.6 Gemini 3 Flash
  6. 88.0 MiniMax-M2.1
  7. 87.8 Qwen3.5 397B-A17B
  8. 87.7 Seed2.0 Lite (0215)
  9. 87.5 GPT-5.4
  10. 87.4 Claude Sonnet 4.5
  11. 87.4 GPT-5.2
  12. 87.3 Claude Opus 4
  13. 87.3 Claude Opus 4.5
  14. 87.3 Claude Sonnet 4.6
  15. 87.2 Hunyuan-T1
  16. 87.1 GPT-5
  17. 87.0 Seed-Thinking-v1.5
  18. 87.0 Grok 4
  19. 87.0 Seed2.0 Pro
  20. 86.7 Qwen3.5-122B-A10B
  21. 86.6 Seed1.6-Thinking
  22. 86.4 GPT-5.1
  23. 86.2 Gemini 3.1 Flash-Lite
  24. 86.1 GPT-4.5
  25. 86.1 Qwen3.5-27B
  26. 86.0 Gemini 2.5 Pro
  27. 86.0 GLM-5
  28. 85.7 Qwen3-Max
  29. 85.3 Qwen3.5-35B-A3B
  30. 85.0 o3
  31. 85.0 DeepSeek-V3.2
  32. 84.8 DeepSeek-V3.1
  33. 84.6 GLM-4.5
  34. 84.5 Qwen3-235B-A22B-Thinking
  35. 84.2 Grok 4.1 Fast
  36. 84.0 Claude 3.7 Sonnet
  37. 84.0 DeepSeek-R1
  38. 83.7 Claude Sonnet 4
  39. 83.6 Seed2.0 Mini
  40. 83.5 Intern-S1
  41. 83.0 Grok-3 mini
  42. 83.0 Qwen3-235B-A22B-Instruct
  43. 82.8 Llama 4 Behemoth
  44. 82.7 seed-oss-36b-instruct
  45. 82.7 LongCat-Flash-Chat
  46. 82.5 Qwen3.5-9B
  47. 82.0 MiniMax-M2
  48. 81.8 GPT-4.1
  49. 81.4 GLM-4.5-Air
  50. 81.3 DeepSeek-V3
  51. 81.1 MiniMax-M1-80k
  52. 81.0 Kimi K2
  53. 80.9 Qwen3-30B-A3B-Thinking
  54. 80.8 gpt-oss-120b
  55. 80.5 Llama 4 Maverick
  56. 80.3 o1-mini
  57. 80.1 Doubao-1.5-pro
  58. 80.1 MiniMax-M2.5
  59. 79.9 Grok 3
  60. 79.4 o3-mini
  61. 79.1 Gemini 2.0 Pro
  62. 79.1 Qwen3.5-4B
  63. 79.0 Hunyuan-TurboS
  64. 78.4 ERNIE-4.5-300B-A47B
  65. 78.3 Nemotron 3 Nano 30B-A3B
  66. 78.0 Claude 3.5 Sonnet (October 2024)
  67. 77.9 GPT-4o
  68. 77.6 Gemini 2.0 Flash
  69. 76.1 Claude 3.5 Sonnet
  70. 76.1 Qwen2.5-Max
  71. 76.0 Phi-4-Reasoning-plus
  72. 75.7 MiniMax-Text-01
  73. 75.5 Grok-2
  74. 74.7 GPT-4o
  75. 74.3 Llama 4 Scout
  76. 74.3 Phi-4-reasoning
  77. 73.6 gpt-oss-20b
  78. 73.3 Llama 3.1-405B
  79. 73.1 Athene-V2-Chat
  80. 72.5 GPT-4o
  81. 71.8 Grok-2 mini
  82. 71.6 Gemini 2.0 Flash-Lite
  83. 71.6 Qwen2.5-72B
  84. 71.0 QwQ-32B-Preview
  85. 70.4 Phi-4
  86. 70.3 Gemini 1.5 Pro
  87. 69.5 ERNIE-4.5-300B-A47B-Base
  88. 69.2 Qwen2.5-32B
  89. 69.1 QwQ-32B
  90. 68.5 Claude 3 Opus
  91. 68.2 Qwen3-235B-A22B
  92. 67.9 Mistral Large 2
  93. 67.5 Gemma 3 27B
  94. 67.3 Hunyuan-A13B-Instruct
  95. 66.8 Mistral Small 3.1
  96. 65.9 Llama 3.3 70B Instruct
  97. 65.9 Mistral Large 2
  98. 65.8 DeepSeek-V2.5
  99. 65.1 Seed-OSS-36B-Base
  100. 65.1 Nemotron 3 Nano 30B-A3B Base
  101. 64.4 Qwen2-72B
  102. 64.1 Gemini 1.5 Flash
  103. 63.7 GPT-4 Turbo
  104. 63.7 Qwen2.5-14B
  105. 63.6 DeepSeek-Coder-V2 236B
  106. 63.1 GPT-4o mini
  107. 62.8 Llama 3.1-70B
  108. 62.8 Llama-3.1-Nemotron-70B-Instruct
  109. 62.4 Yi-Lightning
  110. 62.1 Claude 3.5 Haiku
  111. 61.7 Qwen3-30B-A3B-Base
  112. 60.6 Gemma 3 12B
  113. 60.5 Nemotron-H-56B-Base
  114. 60.4 Seed-OSS-36B-Base-woSyn
  115. 58.9 EXAONE-3.5-32B-Instruct
  116. 58.6 MiMo-7B-RL
  117. 58.1 Yi-Large
  118. 57.6 InternLM3-8B-Instruct
  119. 56.8 Claude 3 Sonnet
  120. 56.7 ERNIE-4.5-21B-A3B-Base
  121. 56.5 Gemma 2 27B
  122. 56.3 Mixtral 8x22B
  123. 56.2 Llama 3-70B
  124. 55.7 phi-3-medium 14B
  125. 55.6 Qwen2.5-Turbo
  126. 55.3 Qwen3.5-2B
  127. 54.8 DeepSeek-V2 (MoE-236B, May 2024)
  128. 52.8 Phi-4 Mini
  129. 52.6 Qwen1.5-72B
  130. 52.3 Yi-1.5-34B
  131. 52.1 Gemma 2 9B
  132. 51.9 Phi-3-medium-128k-instruct
  133. 50.4 MAmmoTH2-8x7B-Plus
  134. 49.9 Qwen1.5-110B
  135. 49.5 Jamba 1.5 Large
  136. 48.4 Mistral Small v24.09
  137. 48.0 GLM-4-9B-Chat
  138. 47.9 GLM-4-9B
  139. 47.9 Phi-3.5-mini
  140. 47.2 Qwen2-7B-Instruct
  141. 46.2 EXAONE-3.5-7.8B-Instruct
  142. 46.0 Yi-1.5-9B-Chat
  143. 45.7 phi-3-mini 3.8B
  144. 45.4 Aya Expanse 32B
  145. 45.0 Qwen2.5-7B
  146. 44.8 Mistral NeMo
  147. 44.3 Llama 3.1-8B
  148. 44.0 Nemotron-H-8B-Base
  149. 43.9 Phi-3-mini-128k-instruct
  150. 43.7 Qwen2.5-3B
  151. 43.6 Gemma 3 4B
  152. 43.4 MAmmoTH2-8B-Plus
  153. 43.3 Mixtral 8x7B
  154. 43.0 Yi-34B
  155. 42.3 Claude 3 Haiku
  156. 42.0 Mathstral-7B-v0.1
  157. 41.9 MiMo-7B-Base
  158. 41.6 DeepSeek-Coder-V2-Lite-Instruct
  159. 41.0 Granite-3.1-8B-Instruct
  160. 41.0 Llama 3-8B
  161. 40.9 MAmmoTH2-7B-Plus
  162. 40.7 Qwen2-7B
  163. 39.8 Mistral-Nemo-Base-2407
  164. 39.2 WizardLM-2 8x22B
  165. 39.1 EXAONE-3.5-2.4B-Instruct
  166. 38.2 Yi-1.5-6B-Chat
  167. 38.0 Qwen1.5-14B-Chat
  168. 37.9 Ministral 8B
  169. 37.9 Command R (03-2024)
  170. 37.5 Llama 2-70B
  171. 35.3 DeepSeekMath-7B-Instruct
  172. 34.4 DeepSeek-Coder-V2-Lite-Base
  173. 33.7 Aya Expanse 8B
  174. 33.7 Gemma 7B
  175. 33.1 Granite-3.1-8B-Base
  176. 33.0 Zephyr 7B Beta
  177. 32.1 Qwen2.5-1.5B
  178. 32.0 Granite-3.1-2B-Instruct
  179. 31.0 Granite-3.0-8B-Base
  180. 30.9 Mistral 7B v0.1
  181. 30.8 Mistral-7B-Instruct-v0.2
  182. 30.4 Mistral 7B v0.2
  183. 29.7 Qwen3.5-0.8B
  184. 29.1 Qwen1.5-7B-Chat
  185. 28.8 Yi-6B-Chat
  186. 26.5 Yi 6B
  187. 25.8 Mistral-7B-Instruct-v0.1
  188. 25.4 Granite-3.1-3B-A800M-Instruct
  189. 25.3 Llama 2-13B
  190. 23.9 Granite-3.1-2B-Base
  191. 23.4 Llemma 7B
  192. 22.6 Qwen2-1.5B-Instruct
  193. 22.6 Qwen2-1.5B
  194. 22.2 Llama 3.2 3B
  195. 21.7 Granite-3.0-2B-Base
  196. 20.4 Granite-3.1-3B-A800M-Base
  197. 20.3 Llama 2-7B
  198. 18.3 SmolLM2-1.7B
  199. 15.9 Qwen2-0.5B-Instruct
  200. 15.8 Gemma 2B
  201. 15.6 Gemma 2 2B IT
  202. 15.0 Qwen2-0.5B
  203. 14.9 Qwen2.5-0.5B
  204. 14.7 Gemma 3 1B
  205. 13.3 Granite-3.1-1B-A400M-Instruct
  206. 12.3 Granite-3.1-1B-A400M-Base
  207. 11.9 Llama 3.2 1B
  208. 11.9 SmolLM-1.7B
  209. 11.4 SmolLM2-360M
  210. 11.2 SmolLM-135M
  211. 10.9 SmolLM-360M
  212. 10.8 SmolLM2-135M
  1. 1 Gemini 3.1 Pro 91.2%
  2. 2 Gemini 3 Pro 90.1%
  3. 3 o1 89.3%
  4. 4 Claude Opus 4.6 thinking 89.1%
  5. 5 Gemini 3 Flash 88.6%
  6. 6 MiniMax-M2.1 88%
  7. 7 Qwen3.5 397B-A17B 87.8%
  8. 8 Seed2.0 Lite (0215) 87.7%
  9. 9 GPT-5.4 87.5%
  10. 10 Claude Sonnet 4.5 thinking 87.4%
  11. 11 GPT-5.2 87.4%
  12. 12 Claude Opus 4 thinking 87.3%
  13. 13 Claude Opus 4.5 thinking 87.3%
  14. 14 Claude Sonnet 4.6 thinking 87.3%
  15. 15 Hunyuan-T1 87.2%
  16. 16 GPT-5 high 87.1%
  17. 17 Seed-Thinking-v1.5 87%
  18. 18 Grok 4 87%
  19. 19 Seed2.0 Pro 87%
  20. 20 Qwen3.5-122B-A10B 86.7%
  21. 21 Seed1.6-Thinking 86.6%
  22. 22 GPT-5.1 86.4%
  23. 23 Gemini 3.1 Flash-Lite 86.2%
  24. 24 GPT-4.5 86.1%
  25. 25 Qwen3.5-27B 86.1%
  26. 26 Gemini 2.5 Pro Jun 2025 86%
  27. 27 GLM-5 86%
  28. 28 Qwen3-Max 85.7%
  29. 29 Qwen3.5-35B-A3B 85.3%
  30. 30 o3 high 85%
  31. 31 DeepSeek-V3.2 85%
  32. 32 DeepSeek-V3.1 84.8%
  33. 33 GLM-4.5 84.6%
  34. 34 Qwen3-235B-A22B-Thinking Jul 2025 84.5%
  35. 35 Grok 4.1 Fast reasoning 84.2%
  36. 36 Claude 3.7 Sonnet thinking 84%
  37. 37 DeepSeek-R1 May 2025 84%
  38. 38 Claude Sonnet 4 83.7%
  39. 39 Seed2.0 Mini 83.6%
  40. 40 Intern-S1 83.5%
  41. 41 Grok-3 mini 83%
  42. 42 Qwen3-235B-A22B-Instruct Jul 2025 83%
  43. 43 Llama 4 Behemoth 82.8%
  44. 44 seed-oss-36b-instruct 82.7%
  45. 45 LongCat-Flash-Chat 82.7%
  46. 46 Qwen3.5-9B 82.5%
  47. 47 MiniMax-M2 82%
  48. 48 GPT-4.1 81.8%
  49. 49 GLM-4.5-Air 81.4%
  50. 50 DeepSeek-V3 Mar 2025 81.3%
  51. 51 MiniMax-M1-80k 81.1%
  52. 52 Kimi K2 Jul 2025 81%
  53. 53 Qwen3-30B-A3B-Thinking Jul 2025 80.9%
  54. 54 gpt-oss-120b high 80.8%
  55. 55 Llama 4 Maverick 80.5%
  56. 56 o1-mini 80.3%
  57. 57 Doubao-1.5-pro 80.1%
  58. 58 MiniMax-M2.5 80.1%
  59. 59 Grok 3 79.9%
  60. 60 o3-mini 79.4%
  61. 61 Gemini 2.0 Pro 79.1%
  62. 62 Qwen3.5-4B 79.1%
  63. 63 Hunyuan-TurboS 79%
  64. 64 ERNIE-4.5-300B-A47B 78.4%
  65. 65 Nemotron 3 Nano 30B-A3B bf16 78.3%
  66. 66 Claude 3.5 Sonnet (October 2024) 78%
  67. 67 GPT-4o Nov 2024 77.9%
  68. 68 Gemini 2.0 Flash Feb 2025 77.6%
  69. 69 Claude 3.5 Sonnet 76.1%
  70. 70 Qwen2.5-Max 76.1%
  71. 71 Phi-4-Reasoning-plus 76%
  72. 72 MiniMax-Text-01 75.7%
  73. 73 Grok-2 Dec 2024 75.5%
  74. 74 GPT-4o Aug 2024 74.7%
  75. 75 Llama 4 Scout 74.3%
  76. 76 Phi-4-reasoning 74.3%
  77. 77 gpt-oss-20b high 73.6%
  78. 78 Llama 3.1-405B 73.3%
  79. 79 Athene-V2-Chat 73.1%
  80. 80 GPT-4o May 2024 72.5%
  81. 81 Grok-2 mini 71.8%
  82. 82 Gemini 2.0 Flash-Lite 71.6%
  83. 83 Qwen2.5-72B 71.6%
  84. 84 QwQ-32B-Preview 71%
  85. 85 Phi-4 70.4%
  86. 86 Gemini 1.5 Pro Sep 2024 70.3%
  87. 87 ERNIE-4.5-300B-A47B-Base 69.5%
  88. 88 Qwen2.5-32B 69.2%
  89. 89 QwQ-32B 69.1%
  90. 90 Claude 3 Opus 68.5%
  91. 91 Qwen3-235B-A22B 68.2%
  92. 92 Mistral Large 2 Nov 2024 67.9%
  93. 93 Gemma 3 27B 67.5%
  94. 94 Hunyuan-A13B-Instruct 67.3%
  95. 95 Mistral Small 3.1 66.8%
  96. 96 Llama 3.3 70B Instruct 65.9%
  97. 97 Mistral Large 2 Jul 2024 65.9%
  98. 98 DeepSeek-V2.5 Sep 2024 65.8%
  99. 99 Seed-OSS-36B-Base 65.1%
  100. 100 Nemotron 3 Nano 30B-A3B Base 65.1%
  101. 101 Qwen2-72B 64.4%
  102. 102 Gemini 1.5 Flash Sep 2024 64.1%
  103. 103 GPT-4 Turbo Apr 2024 63.7%
  104. 104 Qwen2.5-14B 63.7%
  105. 105 DeepSeek-Coder-V2 236B 63.6%
  106. 106 GPT-4o mini 63.1%
  107. 107 Llama 3.1-70B 62.8%
  108. 108 Llama-3.1-Nemotron-70B-Instruct 62.8%
  109. 109 Yi-Lightning 62.4%
  110. 110 Claude 3.5 Haiku 62.1%
  111. 111 Qwen3-30B-A3B-Base 61.7%
  112. 112 Gemma 3 12B 60.6%
  113. 113 Nemotron-H-56B-Base 60.5%
  114. 114 Seed-OSS-36B-Base-woSyn 60.4%
  115. 115 EXAONE-3.5-32B-Instruct 58.9%
  116. 116 MiMo-7B-RL 58.6%
  117. 117 Yi-Large 58.1%
  118. 118 InternLM3-8B-Instruct 57.6%
  119. 119 Claude 3 Sonnet 56.8%
  120. 120 ERNIE-4.5-21B-A3B-Base 56.7%
  121. 121 Gemma 2 27B 56.5%
  122. 122 Mixtral 8x22B 56.3%
  123. 123 Llama 3-70B 56.2%
  124. 124 phi-3-medium 14B 55.7%
  125. 125 Qwen2.5-Turbo 55.6%
  126. 126 Qwen3.5-2B 55.3%
  127. 127 DeepSeek-V2 (MoE-236B, May 2024) 54.8%
  128. 128 Phi-4 Mini 52.8%
  129. 129 Qwen1.5-72B 52.6%
  130. 130 Yi-1.5-34B 52.3%
  131. 131 Gemma 2 9B 52.1%
  132. 132 Phi-3-medium-128k-instruct 51.9%
  133. 133 MAmmoTH2-8x7B-Plus 50.4%
  134. 134 Qwen1.5-110B 49.9%
  135. 135 Jamba 1.5 Large 49.5%
  136. 136 Mistral Small v24.09 48.4%
  137. 137 GLM-4-9B-Chat 48%
  138. 138 GLM-4-9B 47.9%
  139. 139 Phi-3.5-mini 47.9%
  140. 140 Qwen2-7B-Instruct 47.2%
  141. 141 EXAONE-3.5-7.8B-Instruct 46.2%
  142. 142 Yi-1.5-9B-Chat 46%
  143. 143 phi-3-mini 3.8B 45.7%
  144. 144 Aya Expanse 32B 45.4%
  145. 145 Qwen2.5-7B 45%
  146. 146 Mistral NeMo 44.8%
  147. 147 Llama 3.1-8B 44.3%
  148. 148 Nemotron-H-8B-Base 44%
  149. 149 Phi-3-mini-128k-instruct 43.9%
  150. 150 Qwen2.5-3B 43.7%
  151. 151 Gemma 3 4B 43.6%
  152. 152 MAmmoTH2-8B-Plus 43.4%
  153. 153 Mixtral 8x7B 43.3%
  154. 154 Yi-34B 43%
  155. 155 Claude 3 Haiku 42.3%
  156. 156 Mathstral-7B-v0.1 42%
  157. 157 MiMo-7B-Base 41.9%
  158. 158 DeepSeek-Coder-V2-Lite-Instruct 41.6%
  159. 159 Granite-3.1-8B-Instruct 41%
  160. 160 Llama 3-8B 41%
  161. 161 MAmmoTH2-7B-Plus 40.9%
  162. 162 Qwen2-7B 40.7%
  163. 163 Mistral-Nemo-Base-2407 39.8%
  164. 164 WizardLM-2 8x22B 39.2%
  165. 165 EXAONE-3.5-2.4B-Instruct 39.1%
  166. 166 Yi-1.5-6B-Chat 38.2%
  167. 167 Qwen1.5-14B-Chat 38%
  168. 168 Ministral 8B 37.9%
  169. 169 Command R (03-2024) 37.9%
  170. 170 Llama 2-70B 37.5%
  171. 171 DeepSeekMath-7B-Instruct 35.3%
  172. 172 DeepSeek-Coder-V2-Lite-Base 34.4%
  173. 173 Aya Expanse 8B 33.7%
  174. 174 Gemma 7B 33.7%
  175. 175 Granite-3.1-8B-Base 33.1%
  176. 176 Zephyr 7B Beta 33%
  177. 177 Qwen2.5-1.5B 32.1%
  178. 178 Granite-3.1-2B-Instruct 32%
  179. 179 Granite-3.0-8B-Base 31%
  180. 180 Mistral 7B v0.1 30.9%
  181. 181 Mistral-7B-Instruct-v0.2 30.8%
  182. 182 Mistral 7B v0.2 30.4%
  183. 183 Qwen3.5-0.8B 29.7%
  184. 184 Qwen1.5-7B-Chat 29.1%
  185. 185 Yi-6B-Chat 28.8%
  186. 186 Yi 6B 26.5%
  187. 187 Mistral-7B-Instruct-v0.1 25.8%
  188. 188 Granite-3.1-3B-A800M-Instruct 25.4%
  189. 189 Llama 2-13B 25.3%
  190. 190 Granite-3.1-2B-Base 23.9%
  191. 191 Llemma 7B 23.4%
  192. 192 Qwen2-1.5B-Instruct 22.6%
  193. 193 Qwen2-1.5B 22.6%
  194. 194 Llama 3.2 3B 22.2%
  195. 195 Granite-3.0-2B-Base 21.7%
  196. 196 Granite-3.1-3B-A800M-Base 20.4%
  197. 197 Llama 2-7B 20.3%
  198. 198 SmolLM2-1.7B 18.3%
  199. 199 Qwen2-0.5B-Instruct 15.9%
  200. 200 Gemma 2B 15.8%
  201. 201 Gemma 2 2B IT 15.6%
  202. 202 Qwen2-0.5B 15%
  203. 203 Qwen2.5-0.5B 14.9%
  204. 204 Gemma 3 1B 14.7%
  205. 205 Granite-3.1-1B-A400M-Instruct 13.3%
  206. 206 Granite-3.1-1B-A400M-Base 12.3%
  207. 207 Llama 3.2 1B 11.9%
  208. 208 SmolLM-1.7B 11.9%
  209. 209 SmolLM2-360M 11.4%
  210. 210 SmolLM-135M 11.2%
  211. 211 SmolLM-360M 10.9%
  212. 212 SmolLM2-135M 10.8%

Source: Official MMLU-Pro leaderboard

320 models. Scroll sideways for the rest.

  1. 91.2 Gemini 3.1 Pro
  2. 90.1 Gemini 3 Pro
  3. 89.8 Intern-S2-397B
  4. 89.6 Qwen3.7-Max
  5. 89.3 o1
  6. 89.1 Claude Opus 4.6
  7. 88.6 Gemini 3 Flash
  8. 88.5 Qwen 3.6 Plus
  9. 88.5 Qwen3.7-Plus
  10. 88.0 MiniMax-M2.1
  11. 87.8 Qwen3.5 397B-A17B
  12. 87.7 Seed2.0 Lite (0215)
  13. 87.5 GPT-5.4
  14. 87.5 DeepSeek-V4-Pro
  15. 87.4 Claude Sonnet 4.5
  16. 87.4 GPT-5.2
  17. 87.3 Claude Opus 4
  18. 87.3 Claude Opus 4.5
  19. 87.3 Claude Sonnet 4.6
  20. 87.2 Hunyuan-T1
  21. 87.1 GPT-5
  22. 87.1 Kimi K2.5
  23. 87.0 Seed-Thinking-v1.5
  24. 87.0 Grok 4
  25. 87.0 Seed2.0 Pro
  26. 86.8 Nemotron 3 Ultra
  27. 86.7 Qwen3.5-122B-A10B
  28. 86.6 Seed1.6-Thinking
  29. 86.6 Intern-S1-Pro
  30. 86.4 GPT-5.1
  31. 86.3 Solar Pro 4
  32. 86.2 Gemini 3.1 Flash-Lite
  33. 86.2 Qwen3.6 27B
  34. 86.2 DeepSeek-V4-Flash
  35. 86.2 Solar Open 2
  36. 86.1 GPT-4.5
  37. 86.1 Qwen3.5-27B
  38. 86.0 Gemini 2.5 Pro
  39. 86.0 GLM-5
  40. 85.9 Qwen3.5-Omni-Plus
  41. 85.7 Qwen3-Max
  42. 85.7 Qwen3-Max-2026-01-23
  43. 85.4 Cogito v2 Preview 671B MoE
  44. 85.3 Qwen3.5-35B-A3B
  45. 85.2 Gemma 4 31B IT
  46. 85.2 Qwen 3.6 35B-A3B
  47. 85.2 MiniMax-M2.5
  48. 85.0 o3
  49. 85.0 DeepSeek-R1
  50. 85.0 DeepSeek-V3.1-Terminus
  51. 85.0 DeepSeek-V3.2-Exp
  52. 85.0 DeepSeek-V3.2
  53. 85.0 MAI-Thinking-1
  54. 84.9 MiMo-V2-Flash
  55. 84.9 Seed1.8
  56. 84.8 DeepSeek-V3.1
  57. 84.7 Cogito v2.1
  58. 84.6 GLM-4.5
  59. 84.6 Kimi K2 Thinking
  60. 84.5 Qwen3-235B-A22B-Thinking
  61. 84.4 Step 3.5 Flash
  62. 84.3 GLM-4.7
  63. 84.3 ERNIE 5.1
  64. 84.2 Grok 4.1 Fast
  65. 84.0 DeepSeek-R1
  66. 84.0 Claude 3.7 Sonnet
  67. 83.8 Qwen3-VL-235B-A22B
  68. 83.8 K-EXAONE
  69. 83.8 ERNIE 5.0
  70. 83.7 Nemotron 3 Super
  71. 83.7 Claude Sonnet 4
  72. 83.6 Seed2.0 Mini
  73. 83.5 Intern-S1
  74. 83.5 K-EXAONE 2.0
  75. 83.5 MiniCPM-V-4.7-35B-A3B
  76. 83.4 Trinity Large Thinking
  77. 83.3 EXAONE 4.5 33B
  78. 83.0 Grok-3 mini
  79. 83.0 Qwen3-235B-A22B-Instruct
  80. 82.8 Llama 4 Behemoth
  81. 82.7 LongCat-Flash-Omni
  82. 82.7 seed-oss-36b-instruct
  83. 82.7 LongCat-Flash-Chat
  84. 82.7 Qwen3-Next-80B-A3B
  85. 82.6 LongCat-Flash-Thinking
  86. 82.6 Gemma 4 26B A4B
  87. 82.5 Qwen3.5-9B
  88. 82.0 MiniMax-M2
  89. 81.9 Nemotron 3.5 Lightning 30B-A3B
  90. 81.9 INTELLECT-3
  91. 81.8 GPT-4.1
  92. 81.8 EXAONE 4.0 32B
  93. 81.8 MiniMax-M2.7
  94. 81.7 Sarvam 105B
  95. 81.6 Nova 2.0 Pro Preview
  96. 81.4 GLM-4.5-Air
  97. 81.3 DeepSeek-V3
  98. 81.3 InternVL3.5-241B-A28B
  99. 81.2 Solar Pro 3 (March 2026 update)
  100. 81.1 MiniMax-M1-80k
  101. 81.1 Kimi K2
  102. 80.9 Qwen3-30B-A3B-Thinking
  103. 80.9 Nova 2 Lite
  104. 80.8 gpt-oss-120b
  105. 80.7 Nova 2 Omni Preview
  106. 80.7 Hermes 4.3 36B
  107. 80.6 Hermes 4 405B
  108. 80.5 Ring-1T
  109. 80.5 Qwen3-Coder-Next
  110. 80.5 Llama 4 Maverick
  111. 80.4 Solar Open 100B
  112. 80.3 o1-mini
  113. 80.1 Doubao-1.5-pro
  114. 80.0 Sarvam 30B
  115. 79.9 Grok 3
  116. 79.8 Nemotron-Cascade 2 30B-A3B
  117. 79.5 Llama-3.3-Nemotron-Super-49B-v1.5
  118. 79.4 o3-mini
  119. 79.1 Gemini 2.0 Pro
  120. 79.1 Qwen3.5-4B
  121. 79.1 Nemotron 3 Ultra 550B-A55B Base
  122. 79.0 Hunyuan-TurboS
  123. 78.5 Cogito v1 Preview 70B
  124. 78.4 ERNIE-4.5-300B-A47B
  125. 78.3 Nemotron 3 Nano 30B-A3B
  126. 78.0 Claude 3.5 Sonnet (October 2024)
  127. 78.0 Mistral Small 4
  128. 77.9 GPT-4o
  129. 77.6 Gemini 2.0 Flash
  130. 77.6 DiffusionGemma 26B A4B
  131. 77.6 Granite 4.2 30B
  132. 77.3 Nemotron 3 Nano Omni 30B-A3B
  133. 77.2 Mistral Medium 3
  134. 77.2 Gemma 4 12B
  135. 77.1 Nemotron Nano 12B v2 VL
  136. 77.0 Solar Pro 3
  137. 76.5 Solar Pro 2
  138. 76.4 Gemini 2.0 Flash
  139. 76.3 Nemotron 3.5 Super 120B-A12B Base
  140. 76.1 Claude 3.5 Sonnet
  141. 76.1 Qwen2.5-Max
  142. 76.0 STEP3-VL-10B
  143. 76.0 Phi-4-Reasoning-plus
  144. 75.9 DeepSeek-V3
  145. 75.7 MiniMax-Text-01
  146. 75.5 Grok-2
  147. 75.5 Grok-2
  148. 75.3 Trinity Large Preview
  149. 74.7 GPT-4o
  150. 74.3 Llama 4 Scout
  151. 74.3 Phi-4-reasoning
  152. 74.1 NVIDIA Nemotron 3.5 Transcribe 30B-A4B
  153. 74.1 DeepSeek V4.1 Flash
  154. 74.1 DeepSeek-V4.1-Flash Base
  155. 74.0 EXAONE Deep 32B
  156. 73.6 gpt-oss-20b
  157. 73.3 Llama 3.1-405B
  158. 73.1 Athene-V2-Chat
  159. 72.6 Beam 1.0 Base
  160. 72.5 GPT-4o
  161. 72.1 Falcon H1R 7B
  162. 71.8 Grok-2 mini
  163. 71.6 Gemini 2.0 Flash-Lite
  164. 71.6 Qwen2.5-72B
  165. 71.2 Qwen2.5-VL-72B
  166. 71.0 QwQ-32B-Preview
  167. 70.4 Phi-4
  168. 70.3 Gemini 1.5 Pro
  169. 69.6 Cohere Command A
  170. 69.5 ERNIE-4.5-300B-A47B-Base
  171. 69.4 Gemma 4 E4B
  172. 69.2 Qwen2.5-32B
  173. 69.1 QwQ-32B
  174. 69.1 Mistral Small 3.2
  175. 69.0 Mercury
  176. 68.9 Llama 3.3 70B Instruct
  177. 68.5 Kimi-VL-A3B-Thinking-2506
  178. 68.5 Claude 3 Opus
  179. 68.2 Qwen3-235B-A22B
  180. 67.9 Mistral Large 2
  181. 67.5 Gemma 3 27B
  182. 67.5 Nemotron 3.5 Lightning 30B-A3B Base
  183. 67.3 Hunyuan-A13B-Instruct
  184. 66.8 Mistral Small 3.1
  185. 66.5 Qwen3.5-2B
  186. 66.4 Llama 3.1-70B
  187. 66.3 Mistral Small 3
  188. 65.9 Mistral Large 2
  189. 65.8 DeepSeek-V2.5
  190. 65.1 Seed-OSS-36B-Base
  191. 65.1 Nemotron 3 Nano 30B-A3B Base
  192. 65.0 Claude 3.5 Haiku
  193. 65.0 Reka Flash 3
  194. 64.5 Qwen-Turbo
  195. 64.4 Qwen2-72B
  196. 64.1 Gemini 1.5 Flash
  197. 64.1 Granite 4.1 30B
  198. 63.7 GPT-4 Turbo
  199. 63.7 Qwen2.5-14B
  200. 63.6 DeepSeek-Coder-V2 236B
  201. 63.3 Qwen2.5-14B-Instruct-1M
  202. 63.1 GPT-4o mini
  203. 62.8 Llama-3.1-Nemotron-70B-Instruct
  204. 62.4 Yi-Lightning
  205. 61.7 Qwen3-30B-A3B-Base
  206. 60.6 Gemma 3 12B
  207. 60.5 Nemotron-H-56B-Base
  208. 60.4 Seed-OSS-36B-Base-woSyn
  209. 60.2 Code World Model (CWM)
  210. 60.0 Gemma 4 E2B
  211. 58.9 EXAONE-3.5-32B-Instruct
  212. 58.7 Falcon-H1-34B
  213. 58.6 MiMo-7B-RL
  214. 58.1 Yi-Large
  215. 57.6 InternLM3-8B-Instruct
  216. 56.8 Claude 3 Sonnet
  217. 56.7 ERNIE-4.5-21B-A3B-Base
  218. 56.5 Gemma 2 27B
  219. 56.3 Mixtral 8x22B
  220. 56.2 Llama 3-70B
  221. 55.7 phi-3-medium 14B
  222. 55.6 Qwen2.5-Turbo
  223. 55.5 Granite 4.0 H Small
  224. 54.8 DeepSeek-V2 (MoE-236B, May 2024)
  225. 54.3 Phi-3.5-MoE
  226. 52.8 Phi-4 Mini
  227. 52.7 OLMo 2 32B Instruct
  228. 52.6 Qwen1.5-72B
  229. 52.3 Yi-1.5-34B
  230. 52.1 Gemma 2 9B
  231. 51.9 Phi-3-medium-128k-instruct
  232. 50.6 Gemma 3n E4B
  233. 50.4 MAmmoTH2-8x7B-Plus
  234. 49.9 Qwen1.5-110B
  235. 49.6 LFM2.5-1.2B-Thinking
  236. 49.5 Jamba 1.5 Large
  237. 48.4 Mistral Small v24.09
  238. 48.3 Llama 3.1-8B
  239. 48.0 GLM-4-9B-Chat
  240. 47.9 GLM-4-9B
  241. 47.9 Phi-3.5-mini
  242. 47.2 Qwen2-7B-Instruct
  243. 47.0 Qwen2.5-Omni-7B
  244. 46.2 EXAONE-3.5-7.8B-Instruct
  245. 46.0 Yi-1.5-9B-Chat
  246. 45.7 phi-3-mini 3.8B
  247. 45.4 Aya Expanse 32B
  248. 45.0 Qwen2.5-7B
  249. 44.8 Mistral NeMo
  250. 44.4 LFM2.5-1.2B-Instruct
  251. 44.0 Nemotron-H-8B-Base
  252. 43.9 Phi-3-mini-128k-instruct
  253. 43.7 Qwen2.5-3B
  254. 43.6 Gemma 3 4B
  255. 43.4 MAmmoTH2-8B-Plus
  256. 43.3 Mixtral 8x7B
  257. 43.0 Yi-34B
  258. 42.4 Moonlight-16B-A3B
  259. 42.3 Qwen3.5-0.8B
  260. 42.3 Claude 3 Haiku
  261. 42.0 Mathstral-7B-v0.1
  262. 41.9 MiMo-7B-Base
  263. 41.6 DeepSeek-Coder-V2-Lite-Instruct
  264. 41.0 Granite-3.1-8B-Instruct
  265. 41.0 Llama 3-8B
  266. 40.9 MAmmoTH2-7B-Plus
  267. 40.7 Qwen2-7B
  268. 39.8 Mistral-Nemo-Base-2407
  269. 39.2 WizardLM-2 8x22B
  270. 39.1 EXAONE-3.5-2.4B-Instruct
  271. 38.2 Yi-1.5-6B-Chat
  272. 38.0 Qwen1.5-14B-Chat
  273. 37.9 Ministral 8B
  274. 37.9 Command R (03-2024)
  275. 37.5 Llama 2-70B
  276. 37.4 LFM2-8B-A1B
  277. 35.3 DeepSeekMath-7B-Instruct
  278. 34.4 DeepSeek-Coder-V2-Lite-Base
  279. 34.0 granite-4.0-1b
  280. 33.7 Aya Expanse 8B
  281. 33.7 Gemma 7B
  282. 33.1 Granite-3.1-8B-Base
  283. 33.0 Zephyr 7B Beta
  284. 32.1 Qwen2.5-1.5B
  285. 32.0 Granite-3.1-2B-Instruct
  286. 31.0 Granite-3.0-8B-Base
  287. 30.9 Mistral 7B v0.1
  288. 30.8 Mistral-7B-Instruct-v0.2
  289. 30.7 North Micro Vision Instruct
  290. 30.4 Mistral 7B v0.2
  291. 29.1 Qwen1.5-7B-Chat
  292. 28.8 Yi-6B-Chat
  293. 26.5 Yi 6B
  294. 25.8 Mistral-7B-Instruct-v0.1
  295. 25.4 Granite-3.1-3B-A800M-Instruct
  296. 25.3 Llama 2-13B
  297. 23.9 Granite-3.1-2B-Base
  298. 23.4 Llemma 7B
  299. 22.6 Qwen2-1.5B-Instruct
  300. 22.6 Qwen2-1.5B
  301. 22.2 Llama 3.2 3B
  302. 21.7 Granite-3.0-2B-Base
  303. 20.4 Granite-3.1-3B-A800M-Base
  304. 20.3 Llama 2-7B
  305. 19.7 LFM2-1.2B
  306. 18.3 SmolLM2-1.7B
  307. 15.9 Qwen2-0.5B-Instruct
  308. 15.8 Gemma 2B
  309. 15.6 Gemma 2 2B IT
  310. 15.0 Qwen2-0.5B
  311. 14.9 Qwen2.5-0.5B
  312. 14.7 Gemma 3 1B
  313. 13.3 Granite-3.1-1B-A400M-Instruct
  314. 12.3 Granite-3.1-1B-A400M-Base
  315. 11.9 Llama 3.2 1B
  316. 11.9 SmolLM-1.7B
  317. 11.4 SmolLM2-360M
  318. 11.2 SmolLM-135M
  319. 10.9 SmolLM-360M
  320. 10.8 SmolLM2-135M
  1. 1 Gemini 3.1 Pro 91.2%
  2. 2 Gemini 3 Pro 90.1%
  3. 3 Intern-S2-397B 89.8%
  4. 4 Qwen3.7-Max 89.6%
  5. 5 o1 89.3%
  6. 6 Claude Opus 4.6 thinking 89.1%
  7. 7 Gemini 3 Flash 88.6%
  8. 8 Qwen 3.6 Plus 88.5%
  9. 9 Qwen3.7-Plus 88.5%
  10. 10 MiniMax-M2.1 88%
  11. 11 Qwen3.5 397B-A17B 87.8%
  12. 12 Seed2.0 Lite (0215) 87.7%
  13. 13 GPT-5.4 87.5%
  14. 14 DeepSeek-V4-Pro 87.5%
  15. 15 Claude Sonnet 4.5 thinking 87.4%
  16. 16 GPT-5.2 87.4%
  17. 17 Claude Opus 4 thinking 87.3%
  18. 18 Claude Opus 4.5 thinking 87.3%
  19. 19 Claude Sonnet 4.6 thinking 87.3%
  20. 20 Hunyuan-T1 87.2%
  21. 21 GPT-5 high 87.1%
  22. 22 Kimi K2.5 87.1%
  23. 23 Seed-Thinking-v1.5 87%
  24. 24 Grok 4 87%
  25. 25 Seed2.0 Pro 87%
  26. 26 Nemotron 3 Ultra 86.8%
  27. 27 Qwen3.5-122B-A10B 86.7%
  28. 28 Seed1.6-Thinking 86.6%
  29. 29 Intern-S1-Pro 86.6%
  30. 30 GPT-5.1 86.4%
  31. 31 Solar Pro 4 86.3%
  32. 32 Gemini 3.1 Flash-Lite 86.2%
  33. 33 Qwen3.6 27B 86.2%
  34. 34 DeepSeek-V4-Flash 86.2%
  35. 35 Solar Open 2 86.2%
  36. 36 GPT-4.5 86.1%
  37. 37 Qwen3.5-27B 86.1%
  38. 38 Gemini 2.5 Pro Jun 2025 86%
  39. 39 GLM-5 86%
  40. 40 Qwen3.5-Omni-Plus 85.9%
  41. 41 Qwen3-Max 85.7%
  42. 42 Qwen3-Max-2026-01-23 85.7%
  43. 43 Cogito v2 Preview 671B MoE 85.4%
  44. 44 Qwen3.5-35B-A3B 85.3%
  45. 45 Gemma 4 31B IT 85.2%
  46. 46 Qwen 3.6 35B-A3B 85.2%
  47. 47 MiniMax-M2.5 85.2%
  48. 48 o3 high 85%
  49. 49 DeepSeek-R1 May 2025 85%
  50. 50 DeepSeek-V3.1-Terminus 85%
  51. 51 DeepSeek-V3.2-Exp 85%
  52. 52 DeepSeek-V3.2 85%
  53. 53 MAI-Thinking-1 85%
  54. 54 MiMo-V2-Flash 84.9%
  55. 55 Seed1.8 84.9%
  56. 56 DeepSeek-V3.1 84.8%
  57. 57 Cogito v2.1 84.7%
  58. 58 GLM-4.5 84.6%
  59. 59 Kimi K2 Thinking 84.6%
  60. 60 Qwen3-235B-A22B-Thinking Jul 2025 84.5%
  61. 61 Step 3.5 Flash 84.4%
  62. 62 GLM-4.7 84.3%
  63. 63 ERNIE 5.1 84.3%
  64. 64 Grok 4.1 Fast reasoning 84.2%
  65. 65 DeepSeek-R1 84%
  66. 66 Claude 3.7 Sonnet thinking 84%
  67. 67 Qwen3-VL-235B-A22B 83.8%
  68. 68 K-EXAONE 83.8%
  69. 69 ERNIE 5.0 83.8%
  70. 70 Nemotron 3 Super 83.7%
  71. 71 Claude Sonnet 4 83.7%
  72. 72 Seed2.0 Mini 83.6%
  73. 73 Intern-S1 83.5%
  74. 74 K-EXAONE 2.0 83.5%
  75. 75 MiniCPM-V-4.7-35B-A3B 83.5%
  76. 76 Trinity Large Thinking 83.4%
  77. 77 EXAONE 4.5 33B 83.3%
  78. 78 Grok-3 mini 83%
  79. 79 Qwen3-235B-A22B-Instruct Jul 2025 83%
  80. 80 Llama 4 Behemoth 82.8%
  81. 81 LongCat-Flash-Omni 82.7%
  82. 82 seed-oss-36b-instruct 82.7%
  83. 83 LongCat-Flash-Chat 82.7%
  84. 84 Qwen3-Next-80B-A3B 82.7%
  85. 85 LongCat-Flash-Thinking 82.6%
  86. 86 Gemma 4 26B A4B 82.6%
  87. 87 Qwen3.5-9B 82.5%
  88. 88 MiniMax-M2 82%
  89. 89 Nemotron 3.5 Lightning 30B-A3B 81.9%
  90. 90 INTELLECT-3 81.9%
  91. 91 GPT-4.1 81.8%
  92. 92 EXAONE 4.0 32B 81.8%
  93. 93 MiniMax-M2.7 81.8%
  94. 94 Sarvam 105B 81.7%
  95. 95 Nova 2.0 Pro Preview 81.6%
  96. 96 GLM-4.5-Air 81.4%
  97. 97 DeepSeek-V3 Mar 2025 81.3%
  98. 98 InternVL3.5-241B-A28B 81.3%
  99. 99 Solar Pro 3 (March 2026 update) 81.2%
  100. 100 MiniMax-M1-80k 81.1%
  101. 101 Kimi K2 Jul 2025 81.1%
  102. 102 Qwen3-30B-A3B-Thinking Jul 2025 80.9%
  103. 103 Nova 2 Lite 80.9%
  104. 104 gpt-oss-120b high 80.8%
  105. 105 Nova 2 Omni Preview 80.7%
  106. 106 Hermes 4.3 36B 80.7%
  107. 107 Hermes 4 405B 80.6%
  108. 108 Ring-1T 80.5%
  109. 109 Qwen3-Coder-Next 80.5%
  110. 110 Llama 4 Maverick 80.5%
  111. 111 Solar Open 100B 80.4%
  112. 112 o1-mini 80.3%
  113. 113 Doubao-1.5-pro 80.1%
  114. 114 Sarvam 30B 80%
  115. 115 Grok 3 79.9%
  116. 116 Nemotron-Cascade 2 30B-A3B 79.8%
  117. 117 Llama-3.3-Nemotron-Super-49B-v1.5 79.5%
  118. 118 o3-mini 79.4%
  119. 119 Gemini 2.0 Pro 79.1%
  120. 120 Qwen3.5-4B 79.1%
  121. 121 Nemotron 3 Ultra 550B-A55B Base 79.1%
  122. 122 Hunyuan-TurboS 79%
  123. 123 Cogito v1 Preview 70B 78.5%
  124. 124 ERNIE-4.5-300B-A47B 78.4%
  125. 125 Nemotron 3 Nano 30B-A3B bf16 78.3%
  126. 126 Claude 3.5 Sonnet (October 2024) 78%
  127. 127 Mistral Small 4 78%
  128. 128 GPT-4o Nov 2024 77.9%
  129. 129 Gemini 2.0 Flash Feb 2025 77.6%
  130. 130 DiffusionGemma 26B A4B 77.6%
  131. 131 Granite 4.2 30B 77.6%
  132. 132 Nemotron 3 Nano Omni 30B-A3B 77.3%
  133. 133 Mistral Medium 3 77.2%
  134. 134 Gemma 4 12B 77.2%
  135. 135 Nemotron Nano 12B v2 VL 77.1%
  136. 136 Solar Pro 3 77%
  137. 137 Solar Pro 2 76.5%
  138. 138 Gemini 2.0 Flash Dec 2024 76.4%
  139. 139 Nemotron 3.5 Super 120B-A12B Base 76.3%
  140. 140 Claude 3.5 Sonnet 76.1%
  141. 141 Qwen2.5-Max 76.1%
  142. 142 STEP3-VL-10B 76%
  143. 143 Phi-4-Reasoning-plus 76%
  144. 144 DeepSeek-V3 75.9%
  145. 145 MiniMax-Text-01 75.7%
  146. 146 Grok-2 Aug 2024 75.5%
  147. 147 Grok-2 Dec 2024 75.5%
  148. 148 Trinity Large Preview 75.3%
  149. 149 GPT-4o Aug 2024 74.7%
  150. 150 Llama 4 Scout 74.3%
  151. 151 Phi-4-reasoning 74.3%
  152. 152 NVIDIA Nemotron 3.5 Transcribe 30B-A4B 74.1%
  153. 153 DeepSeek V4.1 Flash 74.1%
  154. 154 DeepSeek-V4.1-Flash Base 74.1%
  155. 155 EXAONE Deep 32B 74%
  156. 156 gpt-oss-20b high 73.6%
  157. 157 Llama 3.1-405B 73.3%
  158. 158 Athene-V2-Chat 73.1%
  159. 159 Beam 1.0 Base 72.6%
  160. 160 GPT-4o May 2024 72.5%
  161. 161 Falcon H1R 7B 72.1%
  162. 162 Grok-2 mini 71.8%
  163. 163 Gemini 2.0 Flash-Lite 71.6%
  164. 164 Qwen2.5-72B 71.6%
  165. 165 Qwen2.5-VL-72B 71.2%
  166. 166 QwQ-32B-Preview 71%
  167. 167 Phi-4 70.4%
  168. 168 Gemini 1.5 Pro Sep 2024 70.3%
  169. 169 Cohere Command A 69.6%
  170. 170 ERNIE-4.5-300B-A47B-Base 69.5%
  171. 171 Gemma 4 E4B 69.4%
  172. 172 Qwen2.5-32B 69.2%
  173. 173 QwQ-32B 69.1%
  174. 174 Mistral Small 3.2 69.1%
  175. 175 Mercury 69%
  176. 176 Llama 3.3 70B Instruct 68.9%
  177. 177 Kimi-VL-A3B-Thinking-2506 68.5%
  178. 178 Claude 3 Opus 68.5%
  179. 179 Qwen3-235B-A22B 68.2%
  180. 180 Mistral Large 2 Nov 2024 67.9%
  181. 181 Gemma 3 27B 67.5%
  182. 182 Nemotron 3.5 Lightning 30B-A3B Base 67.5%
  183. 183 Hunyuan-A13B-Instruct 67.3%
  184. 184 Mistral Small 3.1 66.8%
  185. 185 Qwen3.5-2B 66.5%
  186. 186 Llama 3.1-70B 66.4%
  187. 187 Mistral Small 3 66.3%
  188. 188 Mistral Large 2 Jul 2024 65.9%
  189. 189 DeepSeek-V2.5 Sep 2024 65.8%
  190. 190 Seed-OSS-36B-Base 65.1%
  191. 191 Nemotron 3 Nano 30B-A3B Base 65.1%
  192. 192 Claude 3.5 Haiku 65%
  193. 193 Reka Flash 3 65%
  194. 194 Qwen-Turbo 64.5%
  195. 195 Qwen2-72B 64.4%
  196. 196 Gemini 1.5 Flash Sep 2024 64.1%
  197. 197 Granite 4.1 30B 64.1%
  198. 198 GPT-4 Turbo Apr 2024 63.7%
  199. 199 Qwen2.5-14B 63.7%
  200. 200 DeepSeek-Coder-V2 236B 63.6%
  201. 201 Qwen2.5-14B-Instruct-1M 63.3%
  202. 202 GPT-4o mini 63.1%
  203. 203 Llama-3.1-Nemotron-70B-Instruct 62.8%
  204. 204 Yi-Lightning 62.4%
  205. 205 Qwen3-30B-A3B-Base 61.7%
  206. 206 Gemma 3 12B 60.6%
  207. 207 Nemotron-H-56B-Base 60.5%
  208. 208 Seed-OSS-36B-Base-woSyn 60.4%
  209. 209 Code World Model (CWM) 60.2%
  210. 210 Gemma 4 E2B 60%
  211. 211 EXAONE-3.5-32B-Instruct 58.9%
  212. 212 Falcon-H1-34B 58.7%
  213. 213 MiMo-7B-RL 58.6%
  214. 214 Yi-Large 58.1%
  215. 215 InternLM3-8B-Instruct 57.6%
  216. 216 Claude 3 Sonnet 56.8%
  217. 217 ERNIE-4.5-21B-A3B-Base 56.7%
  218. 218 Gemma 2 27B 56.5%
  219. 219 Mixtral 8x22B 56.3%
  220. 220 Llama 3-70B 56.2%
  221. 221 phi-3-medium 14B 55.7%
  222. 222 Qwen2.5-Turbo 55.6%
  223. 223 Granite 4.0 H Small 55.5%
  224. 224 DeepSeek-V2 (MoE-236B, May 2024) 54.8%
  225. 225 Phi-3.5-MoE 54.3%
  226. 226 Phi-4 Mini 52.8%
  227. 227 OLMo 2 32B Instruct 52.7%
  228. 228 Qwen1.5-72B 52.6%
  229. 229 Yi-1.5-34B 52.3%
  230. 230 Gemma 2 9B 52.1%
  231. 231 Phi-3-medium-128k-instruct 51.9%
  232. 232 Gemma 3n E4B 50.6%
  233. 233 MAmmoTH2-8x7B-Plus 50.4%
  234. 234 Qwen1.5-110B 49.9%
  235. 235 LFM2.5-1.2B-Thinking 49.6%
  236. 236 Jamba 1.5 Large 49.5%
  237. 237 Mistral Small v24.09 48.4%
  238. 238 Llama 3.1-8B 48.3%
  239. 239 GLM-4-9B-Chat 48%
  240. 240 GLM-4-9B 47.9%
  241. 241 Phi-3.5-mini 47.9%
  242. 242 Qwen2-7B-Instruct 47.2%
  243. 243 Qwen2.5-Omni-7B 47%
  244. 244 EXAONE-3.5-7.8B-Instruct 46.2%
  245. 245 Yi-1.5-9B-Chat 46%
  246. 246 phi-3-mini 3.8B 45.7%
  247. 247 Aya Expanse 32B 45.4%
  248. 248 Qwen2.5-7B 45%
  249. 249 Mistral NeMo 44.8%
  250. 250 LFM2.5-1.2B-Instruct 44.4%
  251. 251 Nemotron-H-8B-Base 44%
  252. 252 Phi-3-mini-128k-instruct 43.9%
  253. 253 Qwen2.5-3B 43.7%
  254. 254 Gemma 3 4B 43.6%
  255. 255 MAmmoTH2-8B-Plus 43.4%
  256. 256 Mixtral 8x7B 43.3%
  257. 257 Yi-34B 43%
  258. 258 Moonlight-16B-A3B 42.4%
  259. 259 Qwen3.5-0.8B 42.3%
  260. 260 Claude 3 Haiku 42.3%
  261. 261 Mathstral-7B-v0.1 42%
  262. 262 MiMo-7B-Base 41.9%
  263. 263 DeepSeek-Coder-V2-Lite-Instruct 41.6%
  264. 264 Granite-3.1-8B-Instruct 41%
  265. 265 Llama 3-8B 41%
  266. 266 MAmmoTH2-7B-Plus 40.9%
  267. 267 Qwen2-7B 40.7%
  268. 268 Mistral-Nemo-Base-2407 39.8%
  269. 269 WizardLM-2 8x22B 39.2%
  270. 270 EXAONE-3.5-2.4B-Instruct 39.1%
  271. 271 Yi-1.5-6B-Chat 38.2%
  272. 272 Qwen1.5-14B-Chat 38%
  273. 273 Ministral 8B 37.9%
  274. 274 Command R (03-2024) 37.9%
  275. 275 Llama 2-70B 37.5%
  276. 276 LFM2-8B-A1B 37.4%
  277. 277 DeepSeekMath-7B-Instruct 35.3%
  278. 278 DeepSeek-Coder-V2-Lite-Base 34.4%
  279. 279 granite-4.0-1b 34%
  280. 280 Aya Expanse 8B 33.7%
  281. 281 Gemma 7B 33.7%
  282. 282 Granite-3.1-8B-Base 33.1%
  283. 283 Zephyr 7B Beta 33%
  284. 284 Qwen2.5-1.5B 32.1%
  285. 285 Granite-3.1-2B-Instruct 32%
  286. 286 Granite-3.0-8B-Base 31%
  287. 287 Mistral 7B v0.1 30.9%
  288. 288 Mistral-7B-Instruct-v0.2 30.8%
  289. 289 North Micro Vision Instruct 30.7%
  290. 290 Mistral 7B v0.2 30.4%
  291. 291 Qwen1.5-7B-Chat 29.1%
  292. 292 Yi-6B-Chat 28.8%
  293. 293 Yi 6B 26.5%
  294. 294 Mistral-7B-Instruct-v0.1 25.8%
  295. 295 Granite-3.1-3B-A800M-Instruct 25.4%
  296. 296 Llama 2-13B 25.3%
  297. 297 Granite-3.1-2B-Base 23.9%
  298. 298 Llemma 7B 23.4%
  299. 299 Qwen2-1.5B-Instruct 22.6%
  300. 300 Qwen2-1.5B 22.6%
  301. 301 Llama 3.2 3B 22.2%
  302. 302 Granite-3.0-2B-Base 21.7%
  303. 303 Granite-3.1-3B-A800M-Base 20.4%
  304. 304 Llama 2-7B 20.3%
  305. 305 LFM2-1.2B 19.7%
  306. 306 SmolLM2-1.7B 18.3%
  307. 307 Qwen2-0.5B-Instruct 15.9%
  308. 308 Gemma 2B 15.8%
  309. 309 Gemma 2 2B IT 15.6%
  310. 310 Qwen2-0.5B 15%
  311. 311 Qwen2.5-0.5B 14.9%
  312. 312 Gemma 3 1B 14.7%
  313. 313 Granite-3.1-1B-A400M-Instruct 13.3%
  314. 314 Granite-3.1-1B-A400M-Base 12.3%
  315. 315 Llama 3.2 1B 11.9%
  316. 316 SmolLM-1.7B 11.9%
  317. 317 SmolLM2-360M 11.4%
  318. 318 SmolLM-135M 11.2%
  319. 319 SmolLM-360M 10.9%
  320. 320 SmolLM2-135M 10.8%

Sources: Official MMLU-Pro leaderboard and labs’ model cards

#ModelScoreSettingDateSource
1 Intern-S2-397B 89.8% Sep 2026 Lab model card
2 Qwen3.7-Max 89.6% May 2026 Lab model card
3 Qwen 3.6 Plus 88.5% Apr 2026 Lab model card
4 Qwen3.7-Plus 88.5% Jun 2026 Lab model card
5 MiniMax-M2.1 88% Dec 2025 Lab model card
6 Qwen3.5 397B-A17B 87.8% Feb 2026 Lab model card
7 DeepSeek-V4-Pro 87.5% Apr 2026 Lab model card
8 Hunyuan-T1 87.2% Mar 2025 Lab model card
9 Kimi K2.5 87.1% Jan 2026 Lab model card
10 Seed-Thinking-v1.5 87% Apr 2025 Lab model card
11 Seed2.0 Pro 87% Feb 2026 Lab model card
12 Nemotron 3 Ultra 86.8% Jun 2026 Lab model card
13 Qwen3.5-122B-A10B 86.7% Feb 2026 Lab model card
14 Intern-S1-Pro 86.6% Feb 2026 Lab model card
15 Solar Pro 4 86.3% Aug 2026 Lab model card
16 Qwen3.6 27B 86.2% Apr 2026 Lab model card
17 DeepSeek-V4-Flash 86.2% Apr 2026 Lab model card
18 Solar Open 2 86.2% Jul 2026 Lab model card
19 Qwen3.5-27B 86.1% Feb 2026 Lab model card
20 Qwen3.5-Omni-Plus 85.9% Mar 2026 Lab model card
21 Qwen3-Max-2026-01-23 85.7% Jan 2026 Lab model card
22 Cogito v2 Preview 671B MoE 85.4% Jul 2025 Lab model card
23 Qwen3.5-35B-A3B 85.3% Feb 2026 Lab model card
24 Gemma 4 31B IT 85.2% Apr 2026 Lab model card
25 Qwen 3.6 35B-A3B 85.2% Apr 2026 Lab model card
26 MiniMax-M2.5 85.2% May 2026 Lab model card
27 DeepSeek-R1 May 2025 85% May 2025 Lab model card
28 DeepSeek-V3.1-Terminus 85% Sep 2025 Lab model card
29 DeepSeek-V3.2-Exp 85% Sep 2025 Lab model card
30 DeepSeek-V3.2 85% Dec 2025 Lab model card
31 MAI-Thinking-1 85% Jun 2026 Lab model card
32 MiMo-V2-Flash 84.9% Dec 2025 Lab model card
33 Seed1.8 84.9% Dec 2025 Lab model card
34 DeepSeek-V3.1 84.8% Aug 2025 Lab model card
35 Cogito v2.1 84.7% Nov 2025 Lab model card
36 GLM-4.5 84.6% Jul 2025 Lab model card
37 Kimi K2 Thinking 84.6% Nov 2025 Lab model card
38 Qwen3-235B-A22B-Thinking Jul 2025 84.4% Jul 2025 Lab model card
39 Step 3.5 Flash 84.4% Feb 2026 Lab model card
40 GLM-4.7 84.3% Dec 2025 Lab model card
41 ERNIE 5.1 84.3% May 2026 Lab model card
42 DeepSeek-R1 84% Jan 2025 Lab model card
43 Qwen3-VL-235B-A22B 83.8% Sep 2025 Lab model card
44 K-EXAONE 83.8% Dec 2025 Lab model card
45 ERNIE 5.0 83.8% Feb 2026 Lab model card
46 Nemotron 3 Super 83.7% Mar 2026 Lab model card
47 Intern-S1 83.5% Jul 2025 Lab model card
48 K-EXAONE 2.0 83.5% Jul 2026 Lab model card
49 MiniCPM-V-4.7-35B-A3B 83.5% Oct 2026 Lab model card
50 Trinity Large Thinking 83.4% Apr 2026 Lab model card
51 EXAONE 4.5 33B 83.3% Apr 2026 Lab model card
52 LongCat-Flash-Omni 82.7% Oct 2025 Lab model card
53 seed-oss-36b-instruct 82.7% Aug 2025 Lab model card
54 Qwen3-Next-80B-A3B 82.7% Sep 2025 Lab model card
55 LongCat-Flash-Chat 82.7% Aug 2025 Lab model card
56 LongCat-Flash-Thinking 82.6% Sep 2025 Lab model card
57 Gemma 4 26B A4B 82.6% Apr 2026 Lab model card
58 Qwen3.5-9B 82.5% Mar 2026 Lab model card
59 Llama 4 Behemoth 82.2% Apr 2025 Lab model card
60 MiniMax-M2 82% Oct 2025 Lab model card
61 Nemotron 3.5 Lightning 30B-A3B 81.9% Aug 2026 Lab model card
62 INTELLECT-3 81.9% Nov 2025 Lab model card
63 EXAONE 4.0 32B 81.8% Jul 2025 Lab model card
64 MiniMax-M2.7 81.8% May 2026 Lab model card
65 Sarvam 105B 81.7% Mar 2026 Lab model card
66 Nova 2.0 Pro Preview 81.6% Dec 2025 Lab model card
67 GLM-4.5-Air 81.4% Aug 2025 Lab model card
68 InternVL3.5-241B-A28B 81.3% Aug 2025 Lab model card
69 Solar Pro 3 (March 2026 update) 81.2% Mar 2026 Lab model card
70 DeepSeek-V3 Mar 2025 81.2% Mar 2025 Lab model card
71 MiniMax-M1-80k 81.1% Jun 2025 Lab model card
72 Kimi K2 Jul 2025 81.1% Jul 2025 Lab model card
73 Nova 2 Lite 80.9% Dec 2025 Lab model card
74 Nova 2 Omni Preview 80.7% Dec 2025 Lab model card
75 Hermes 4.3 36B 80.7% Dec 2025 Lab model card
76 Hermes 4 405B 80.6% Aug 2025 Lab model card
77 Ring-1T 80.5% Oct 2025 Lab model card
78 Qwen3-Coder-Next 80.5% Feb 2026 Lab model card
79 Llama 4 Maverick 80.5% Apr 2025 Lab model card
80 Solar Open 100B 80.4% Jan 2026 Lab model card
81 Sarvam 30B 80% Mar 2026 Lab model card
82 Grok 3 79.9% Feb 2025 Lab model card
83 Nemotron-Cascade 2 30B-A3B 79.8% Mar 2026 Lab model card
84 Llama-3.3-Nemotron-Super-49B-v1.5 79.5% Jul 2025 Lab model card
85 Gemini 2.0 Pro 79.1% Feb 2025 Lab model card
86 Qwen3.5-4B 79.1% Mar 2026 Lab model card
87 Nemotron 3 Ultra 550B-A55B Base 79.1% Jun 2026 Lab model card
88 Hunyuan-TurboS 79% Feb 2025 Lab model card
89 Grok-3 mini 78.9% Feb 2025 Lab model card
90 Cogito v1 Preview 70B 78.5% Apr 2025 Lab model card
91 ERNIE-4.5-300B-A47B 78.4% Jun 2025 Lab model card
92 Nemotron 3 Nano 30B-A3B 78.3% Dec 2025 Lab model card
93 Claude 3.5 Sonnet (October 2024) 78% Oct 2024 Lab model card
94 Mistral Small 4 78% Mar 2026 Lab model card
95 Gemini 2.0 Flash Feb 2025 77.6% Feb 2025 Lab model card
96 DiffusionGemma 26B A4B 77.6% Jun 2026 Lab model card
97 Granite 4.2 30B 77.6% Aug 2026 Lab model card
98 Nemotron 3 Nano Omni 30B-A3B 77.3% Apr 2026 Lab model card
99 Mistral Medium 3 77.2% May 2025 Lab model card
100 Gemma 4 12B 77.2% Jun 2026 Lab model card
101 Nemotron Nano 12B v2 VL 77.1% Nov 2025 Lab model card
102 Solar Pro 3 77% Jan 2026 Lab model card
103 Solar Pro 2 76.5% Jul 2025 Lab model card
104 Gemini 2.0 Flash Dec 2024 76.4% Dec 2024 Lab model card
105 Nemotron 3.5 Super 120B-A12B Base 76.3% Oct 2026 Lab model card
106 STEP3-VL-10B 76% Jan 2026 Lab model card
107 Phi-4-Reasoning-plus 76% Apr 2025 Lab model card
108 DeepSeek-V3 75.9% Dec 2024 Lab model card
109 MiniMax-Text-01 75.7% Jan 2025 Lab model card
110 Grok-2 Aug 2024 75.5% Aug 2024 Lab model card
111 Trinity Large Preview 75.3% Feb 2026 Lab model card
112 Llama 4 Scout 74.3% Apr 2025 Lab model card
113 Phi-4-reasoning 74.3% Apr 2025 Lab model card
114 NVIDIA Nemotron 3.5 Transcribe 30B-A4B 74.1% Oct 2026 Lab model card
115 DeepSeek V4.1 Flash 74.1% Sep 2026 Lab model card
116 DeepSeek-V4.1-Flash Base 74.1% Sep 2026 Lab model card
117 EXAONE Deep 32B 74% Mar 2025 Lab model card
118 Llama 3.1-405B 73.3% Jul 2024 Lab model card
119 Beam 1.0 Base 72.6% Oct 2026 Lab model card
120 Falcon H1R 7B 72.1% Jan 2026 Lab model card
121 Gemini 2.0 Flash-Lite 71.6% Feb 2025 Lab model card
122 Qwen2.5-VL-72B 71.2% Feb 2025 Lab model card
123 Qwen2.5-72B 71.1% Sep 2024 Lab model card
124 Phi-4 70.4% Dec 2024 Lab model card
125 Cohere Command A 69.6% Apr 2025 Lab model card
126 Gemma 4 E4B 69.4% Apr 2026 Lab model card
127 Mistral Small 3.2 69.1% Jun 2025 Lab model card
128 Qwen2.5-32B 69% Dec 2024 Lab model card
129 Mercury 69% Jun 2025 Lab model card
130 Llama 3.3 70B Instruct 68.9% Dec 2024 Lab model card
131 Kimi-VL-A3B-Thinking-2506 68.5% Jun 2025 Lab model card
132 Gemma 3 27B 67.5% Mar 2025 Lab model card
133 Nemotron 3.5 Lightning 30B-A3B Base 67.5% Oct 2026 Lab model card
134 Mistral Small 3.1 66.8% Mar 2025 Lab model card
135 Qwen3.5-2B 66.5% Mar 2026 Lab model card
136 Llama 3.1-70B 66.4% Jul 2024 Lab model card
137 Mistral Small 3 66.3% Jan 2025 Lab model card
138 Nemotron 3 Nano 30B-A3B Base 65% Dec 2025 Lab model card
139 Claude 3.5 Haiku 65% Oct 2024 Lab model card
140 Reka Flash 3 65% Mar 2025 Lab model card
141 Qwen-Turbo 64.5% Dec 2024 Lab model card
142 Qwen2-72B 64.4% Jun 2024 Lab model card
143 Granite 4.1 30B 64.1% Apr 2026 Lab model card
144 Qwen2.5-14B 63.7% Dec 2024 Lab model card
145 Qwen2.5-14B-Instruct-1M 63.3% Jan 2025 Lab model card
146 Gemma 3 12B 60.6% Mar 2025 Lab model card
147 Code World Model (CWM) 60.2% Sep 2025 Lab model card
148 Gemma 4 E2B 60% Apr 2026 Lab model card
149 Falcon-H1-34B 58.7% May 2025 Lab model card
150 MiMo-7B-RL 58.6% Apr 2025 Lab model card
151 InternLM3-8B-Instruct 57.6% Jan 2025 Lab model card
152 Granite 4.0 H Small 55.5% Oct 2025 Lab model card
153 Phi-3.5-MoE 54.3% Aug 2024 Lab model card
154 Phi-4 Mini 52.8% Feb 2025 Lab model card
155 OLMo 2 32B Instruct 52.7% Dec 2024 Lab model card
156 Gemma 3n E4B 50.6% Jun 2025 Lab model card
157 LFM2.5-1.2B-Thinking 49.6% Jan 2026 Lab model card
158 Llama 3.1-8B 48.3% Jul 2024 Lab model card
159 Phi-3.5-mini 47.5% Aug 2024 Lab model card
160 Qwen2.5-Omni-7B 47% Mar 2025 Lab model card
161 Qwen2.5-7B 45% Dec 2024 Lab model card
162 LFM2.5-1.2B-Instruct 44.4% Jan 2026 Lab model card
163 Gemma 3 4B 43.6% Mar 2025 Lab model card
164 Moonlight-16B-A3B 42.4% Feb 2025 Lab model card
165 Qwen3.5-0.8B 42.3% Mar 2026 Lab model card
166 LFM2-8B-A1B 37.4% Oct 2025 Lab model card
167 granite-4.0-1b 34% Oct 2025 Lab model card
168 North Micro Vision Instruct 30.7% Aug 2026 Lab model card
169 LFM2-1.2B 19.7% Nov 2025 Lab model card
170 Gemma 3 1B 14.7% Mar 2025 Lab model card
#ModelScoreSettingDateSource
1 Gemini 3.1 Pro 91.2% Feb 2026 Official leaderboard
2 Gemini 3 Pro 90.1% Nov 2025 Official leaderboard
3 o1 89.3% Dec 2024 Official leaderboard
4 Claude Opus 4.6 89.1% thinking Feb 2026 Official leaderboard
5 Gemini 3 Flash 88.6% Dec 2025 Official leaderboard
6 MiniMax-M2.1 88% Dec 2025 Official leaderboard
7 Qwen3.5 397B-A17B 87.8% Feb 2026 Official leaderboard
8 Seed2.0 Lite (0215) 87.7% Feb 2026 Official leaderboard
9 GPT-5.4 87.5% Mar 2026 Official leaderboard
10 Claude Sonnet 4.5 87.4% thinking Sep 2025 Official leaderboard
11 GPT-5.2 87.4% Dec 2025 Official leaderboard
12 Claude Opus 4 87.3% thinking May 2025 Official leaderboard
13 Claude Opus 4.5 87.3% thinking Nov 2025 Official leaderboard
14 Claude Sonnet 4.6 87.3% thinking Feb 2026 Official leaderboard
15 Hunyuan-T1 87.2% Mar 2025 Official leaderboard
16 GPT-5 87.1% high Aug 2025 Official leaderboard
17 Seed-Thinking-v1.5 87% Apr 2025 Official leaderboard
18 Grok 4 87% Jul 2025 Official leaderboard
19 Seed2.0 Pro 87% Feb 2026 Official leaderboard
20 Qwen3.5-122B-A10B 86.7% Feb 2026 Official leaderboard
21 Seed1.6-Thinking 86.6% Jun 2025 Official leaderboard
22 GPT-5.1 86.4% Nov 2025 Official leaderboard
23 Gemini 3.1 Flash-Lite 86.2% Mar 2026 Official leaderboard
24 GPT-4.5 86.1% Feb 2025 Official leaderboard
25 Qwen3.5-27B 86.1% Feb 2026 Official leaderboard
26 Gemini 2.5 Pro Jun 2025 86% Jun 2025 Official leaderboard
27 GLM-5 86% Feb 2026 Official leaderboard
28 Qwen3-Max 85.7% Sep 2025 Official leaderboard
29 Qwen3.5-35B-A3B 85.3% Feb 2026 Official leaderboard
30 o3 85% high Apr 2025 Official leaderboard
31 DeepSeek-V3.2 85% Dec 2025 Official leaderboard
32 DeepSeek-V3.1 84.8% Aug 2025 Official leaderboard
33 GLM-4.5 84.6% Aug 2025 Official leaderboard
34 Qwen3-235B-A22B-Thinking Jul 2025 84.5% Jul 2025 Official leaderboard
35 Grok 4.1 Fast 84.2% reasoning Nov 2025 Official leaderboard
36 Claude 3.7 Sonnet 84% thinking Feb 2025 Official leaderboard
37 DeepSeek-R1 May 2025 84% May 2025 Official leaderboard
38 Claude Sonnet 4 83.7% May 2025 Official leaderboard
39 Seed2.0 Mini 83.6% Feb 2026 Official leaderboard
40 Intern-S1 83.5% Jul 2025 Official leaderboard
41 Grok-3 mini 83% Apr 2025 Official leaderboard
42 Qwen3-235B-A22B-Instruct Jul 2025 83% Jul 2025 Official leaderboard
43 Llama 4 Behemoth 82.8% Apr 2025 Official leaderboard
44 seed-oss-36b-instruct 82.7% Aug 2025 Official leaderboard
45 LongCat-Flash-Chat 82.7% Aug 2025 Official leaderboard
46 Qwen3.5-9B 82.5% Feb 2026 Official leaderboard
47 MiniMax-M2 82% Oct 2025 Official leaderboard
48 GPT-4.1 81.8% Apr 2025 Official leaderboard
49 GLM-4.5-Air 81.4% Jul 2025 Official leaderboard
50 DeepSeek-V3 Mar 2025 81.3% Mar 2025 Official leaderboard
51 MiniMax-M1-80k 81.1% Jun 2025 Official leaderboard
52 Kimi K2 Jul 2025 81% Jul 2025 Official leaderboard
53 Qwen3-30B-A3B-Thinking Jul 2025 80.9% Jul 2025 Official leaderboard
54 gpt-oss-120b 80.8% high Aug 2025 Official leaderboard
55 Llama 4 Maverick 80.5% Apr 2025 Official leaderboard
56 o1-mini 80.3% Sep 2024 Official leaderboard
57 Doubao-1.5-pro 80.1% Jan 2025 Official leaderboard
58 MiniMax-M2.5 80.1% Feb 2026 Official leaderboard
59 Grok 3 79.9% Apr 2025 Official leaderboard
60 o3-mini 79.4% Jan 2025 Official leaderboard
61 Gemini 2.0 Pro 79.1% Feb 2025 Official leaderboard
62 Qwen3.5-4B 79.1% Feb 2026 Official leaderboard
63 Hunyuan-TurboS 79% Feb 2025 Official leaderboard
64 ERNIE-4.5-300B-A47B 78.4% Jun 2025 Official leaderboard
65 Nemotron 3 Nano 30B-A3B 78.3% bf16 Dec 2025 Official leaderboard
66 Claude 3.5 Sonnet (October 2024) 78% Oct 2024 Official leaderboard
67 GPT-4o Nov 2024 77.9% Nov 2024 Official leaderboard
68 Gemini 2.0 Flash Feb 2025 77.6% Feb 2025 Official leaderboard
69 Claude 3.5 Sonnet 76.1% Jun 2024 Official leaderboard
70 Qwen2.5-Max 76.1% Jan 2025 Official leaderboard
71 Phi-4-Reasoning-plus 76% Apr 2025 Official leaderboard
72 MiniMax-Text-01 75.7% Jan 2025 Official leaderboard
73 Grok-2 Dec 2024 75.5% Dec 2024 Official leaderboard
74 GPT-4o Aug 2024 74.7% Aug 2024 Official leaderboard
75 Llama 4 Scout 74.3% Apr 2025 Official leaderboard
76 Phi-4-reasoning 74.3% Apr 2025 Official leaderboard
77 gpt-oss-20b 73.6% high Aug 2025 Official leaderboard
78 Llama 3.1-405B 73.3% Jul 2024 Official leaderboard
79 Athene-V2-Chat 73.1% Nov 2024 Official leaderboard
80 GPT-4o May 2024 72.5% May 2024 Official leaderboard
81 Grok-2 mini 71.8% Aug 2024 Official leaderboard
82 Gemini 2.0 Flash-Lite 71.6% Feb 2025 Official leaderboard
83 Qwen2.5-72B 71.6% Sep 2024 Official leaderboard
84 QwQ-32B-Preview 71% Nov 2024 Official leaderboard
85 Phi-4 70.4% Dec 2024 Official leaderboard
86 Gemini 1.5 Pro Sep 2024 70.3% Sep 2024 Official leaderboard
87 ERNIE-4.5-300B-A47B-Base 69.5% Jun 2025 Official leaderboard
88 Qwen2.5-32B 69.2% Sep 2024 Official leaderboard
89 QwQ-32B 69.1% Mar 2025 Official leaderboard
90 Claude 3 Opus 68.5% Feb 2024 Official leaderboard
91 Qwen3-235B-A22B 68.2% Apr 2025 Official leaderboard
92 Mistral Large 2 Nov 2024 67.9% Nov 2024 Official leaderboard
93 Gemma 3 27B 67.5% Mar 2025 Official leaderboard
94 Hunyuan-A13B-Instruct 67.3% Jun 2025 Official leaderboard
95 Mistral Small 3.1 66.8% Mar 2025 Official leaderboard
96 Llama 3.3 70B Instruct 65.9% Dec 2024 Official leaderboard
97 Mistral Large 2 Jul 2024 65.9% Jul 2024 Official leaderboard
98 DeepSeek-V2.5 Sep 2024 65.8% Sep 2024 Official leaderboard
99 Seed-OSS-36B-Base 65.1% Aug 2025 Official leaderboard
100 Nemotron 3 Nano 30B-A3B Base 65.1% Dec 2025 Official leaderboard
101 Qwen2-72B 64.4% Jun 2024 Official leaderboard
102 Gemini 1.5 Flash Sep 2024 64.1% Aug 2024 Official leaderboard
103 GPT-4 Turbo Apr 2024 63.7% Apr 2024 Official leaderboard
104 Qwen2.5-14B 63.7% Sep 2024 Official leaderboard
105 DeepSeek-Coder-V2 236B 63.6% Jun 2024 Official leaderboard
106 GPT-4o mini 63.1% Jul 2024 Official leaderboard
107 Llama 3.1-70B 62.8% Jul 2024 Official leaderboard
108 Llama-3.1-Nemotron-70B-Instruct 62.8% Oct 2024 Official leaderboard
109 Yi-Lightning 62.4% Dec 2024 Official leaderboard
110 Claude 3.5 Haiku 62.1% Oct 2024 Official leaderboard
111 Qwen3-30B-A3B-Base 61.7% Apr 2025 Official leaderboard
112 Gemma 3 12B 60.6% Mar 2025 Official leaderboard
113 Nemotron-H-56B-Base 60.5% Apr 2025 Official leaderboard
114 Seed-OSS-36B-Base-woSyn 60.4% Aug 2025 Official leaderboard
115 EXAONE-3.5-32B-Instruct 58.9% Dec 2024 Official leaderboard
116 MiMo-7B-RL 58.6% Apr 2025 Official leaderboard
117 Yi-Large 58.1% May 2024 Official leaderboard
118 InternLM3-8B-Instruct 57.6% Jan 2025 Official leaderboard
119 Claude 3 Sonnet 56.8% Feb 2024 Official leaderboard
120 ERNIE-4.5-21B-A3B-Base 56.7% Jun 2025 Official leaderboard
121 Gemma 2 27B 56.5% Jun 2024 Official leaderboard
122 Mixtral 8x22B 56.3% Apr 2024 Official leaderboard
123 Llama 3-70B 56.2% Apr 2024 Official leaderboard
124 phi-3-medium 14B 55.7% Apr 2024 Official leaderboard
125 Qwen2.5-Turbo 55.6% Nov 2024 Official leaderboard
126 Qwen3.5-2B 55.3% Feb 2026 Official leaderboard
127 DeepSeek-V2 (MoE-236B, May 2024) 54.8% May 2024 Official leaderboard
128 Phi-4 Mini 52.8% Feb 2025 Official leaderboard
129 Qwen1.5-72B 52.6% Feb 2024 Official leaderboard
130 Yi-1.5-34B 52.3% May 2024 Official leaderboard
131 Gemma 2 9B 52.1% Jun 2024 Official leaderboard
132 Phi-3-medium-128k-instruct 51.9% May 2024 Official leaderboard
133 MAmmoTH2-8x7B-Plus 50.4% May 2024 Official leaderboard
134 Qwen1.5-110B 49.9% Apr 2024 Official leaderboard
135 Jamba 1.5 Large 49.5% Aug 2024 Official leaderboard
136 Mistral Small v24.09 48.4% Sep 2024 Official leaderboard
137 GLM-4-9B-Chat 48% Jun 2024 Official leaderboard
138 GLM-4-9B 47.9% Jun 2024 Official leaderboard
139 Phi-3.5-mini 47.9% Aug 2024 Official leaderboard
140 Qwen2-7B-Instruct 47.2% Jun 2024 Official leaderboard
141 EXAONE-3.5-7.8B-Instruct 46.2% Dec 2024 Official leaderboard
142 Yi-1.5-9B-Chat 46% May 2024 Official leaderboard
143 phi-3-mini 3.8B 45.7% Apr 2024 Official leaderboard
144 Aya Expanse 32B 45.4% Oct 2024 Official leaderboard
145 Qwen2.5-7B 45% Sep 2024 Official leaderboard
146 Mistral NeMo 44.8% Jul 2024 Official leaderboard
147 Llama 3.1-8B 44.3% Jul 2024 Official leaderboard
148 Nemotron-H-8B-Base 44% Mar 2025 Official leaderboard
149 Phi-3-mini-128k-instruct 43.9% Apr 2024 Official leaderboard
150 Qwen2.5-3B 43.7% Sep 2024 Official leaderboard
151 Gemma 3 4B 43.6% Mar 2025 Official leaderboard
152 MAmmoTH2-8B-Plus 43.4% May 2024 Official leaderboard
153 Mixtral 8x7B 43.3% Dec 2023 Official leaderboard
154 Yi-34B 43% Nov 2023 Official leaderboard
155 Claude 3 Haiku 42.3% Mar 2024 Official leaderboard
156 Mathstral-7B-v0.1 42% Jul 2024 Official leaderboard
157 MiMo-7B-Base 41.9% Apr 2025 Official leaderboard
158 DeepSeek-Coder-V2-Lite-Instruct 41.6% Jun 2024 Official leaderboard
159 Granite-3.1-8B-Instruct 41% Dec 2024 Official leaderboard
160 Llama 3-8B 41% Apr 2024 Official leaderboard
161 MAmmoTH2-7B-Plus 40.9% May 2024 Official leaderboard
162 Qwen2-7B 40.7% Jun 2024 Official leaderboard
163 Mistral-Nemo-Base-2407 39.8% Jul 2024 Official leaderboard
164 WizardLM-2 8x22B 39.2% Apr 2024 Official leaderboard
165 EXAONE-3.5-2.4B-Instruct 39.1% Dec 2024 Official leaderboard
166 Yi-1.5-6B-Chat 38.2% May 2024 Official leaderboard
167 Qwen1.5-14B-Chat 38% Feb 2024 Official leaderboard
168 Ministral 8B 37.9% Oct 2024 Official leaderboard
169 Command R (03-2024) 37.9% Mar 2024 Official leaderboard
170 Llama 2-70B 37.5% Jul 2023 Official leaderboard
171 DeepSeekMath-7B-Instruct 35.3% Feb 2024 Official leaderboard
172 DeepSeek-Coder-V2-Lite-Base 34.4% Jun 2024 Official leaderboard
173 Aya Expanse 8B 33.7% Oct 2024 Official leaderboard
174 Gemma 7B 33.7% Feb 2024 Official leaderboard
175 Granite-3.1-8B-Base 33.1% Dec 2024 Official leaderboard
176 Zephyr 7B Beta 33% Oct 2023 Official leaderboard
177 Qwen2.5-1.5B 32.1% Sep 2024 Official leaderboard
178 Granite-3.1-2B-Instruct 32% Dec 2024 Official leaderboard
179 Granite-3.0-8B-Base 31% Oct 2024 Official leaderboard
180 Mistral 7B v0.1 30.9% Sep 2023 Official leaderboard
181 Mistral-7B-Instruct-v0.2 30.8% Dec 2023 Official leaderboard
182 Mistral 7B v0.2 30.4% Dec 2023 Official leaderboard
183 Qwen3.5-0.8B 29.7% Feb 2026 Official leaderboard
184 Qwen1.5-7B-Chat 29.1% Feb 2024 Official leaderboard
185 Yi-6B-Chat 28.8% Nov 2023 Official leaderboard
186 Yi 6B 26.5% Nov 2023 Official leaderboard
187 Mistral-7B-Instruct-v0.1 25.8% Sep 2023 Official leaderboard
188 Granite-3.1-3B-A800M-Instruct 25.4% Dec 2024 Official leaderboard
189 Llama 2-13B 25.3% Jul 2023 Official leaderboard
190 Granite-3.1-2B-Base 23.9% Dec 2024 Official leaderboard
191 Llemma 7B 23.4% Oct 2023 Official leaderboard
192 Qwen2-1.5B-Instruct 22.6% Jun 2024 Official leaderboard
193 Qwen2-1.5B 22.6% Jun 2024 Official leaderboard
194 Llama 3.2 3B 22.2% Sep 2024 Official leaderboard
195 Granite-3.0-2B-Base 21.7% Oct 2024 Official leaderboard
196 Granite-3.1-3B-A800M-Base 20.4% Dec 2024 Official leaderboard
197 Llama 2-7B 20.3% Jul 2023 Official leaderboard
198 SmolLM2-1.7B 18.3% Oct 2024 Official leaderboard
199 Qwen2-0.5B-Instruct 15.9% Jun 2024 Official leaderboard
200 Gemma 2B 15.8% Feb 2024 Official leaderboard
201 Gemma 2 2B IT 15.6% Jul 2024 Official leaderboard
202 Qwen2-0.5B 15% Jun 2024 Official leaderboard
203 Qwen2.5-0.5B 14.9% Sep 2024 Official leaderboard
204 Gemma 3 1B 14.7% Mar 2025 Official leaderboard
205 Granite-3.1-1B-A400M-Instruct 13.3% Dec 2024 Official leaderboard
206 Granite-3.1-1B-A400M-Base 12.3% Dec 2024 Official leaderboard
207 Llama 3.2 1B 11.9% Sep 2024 Official leaderboard
208 SmolLM-1.7B 11.9% Jul 2024 Official leaderboard
209 SmolLM2-360M 11.4% Oct 2024 Official leaderboard
210 SmolLM-135M 11.2% Jul 2024 Official leaderboard
211 SmolLM-360M 10.9% Jul 2024 Official leaderboard
212 SmolLM2-135M 10.8% Oct 2024 Official leaderboard
#ModelScoreSettingDateSource
1 Gemini 3.1 Pro 91.2% Feb 2026 Official leaderboard
2 Gemini 3 Pro 90.1% Nov 2025 Official leaderboard
3 Intern-S2-397B 89.8% Sep 2026 Lab model card
4 Qwen3.7-Max 89.6% May 2026 Lab model card
5 o1 89.3% Dec 2024 Official leaderboard
6 Claude Opus 4.6 89.1% thinking Feb 2026 Official leaderboard
7 Gemini 3 Flash 88.6% Dec 2025 Official leaderboard
8 Qwen 3.6 Plus 88.5% Apr 2026 Lab model card
9 Qwen3.7-Plus 88.5% Jun 2026 Lab model card
10 MiniMax-M2.1 88% Dec 2025 Official leaderboard
11 Qwen3.5 397B-A17B 87.8% Feb 2026 Official leaderboard
12 Seed2.0 Lite (0215) 87.7% Feb 2026 Official leaderboard
13 GPT-5.4 87.5% Mar 2026 Official leaderboard
14 DeepSeek-V4-Pro 87.5% Apr 2026 Lab model card
15 Claude Sonnet 4.5 87.4% thinking Sep 2025 Official leaderboard
16 GPT-5.2 87.4% Dec 2025 Official leaderboard
17 Claude Opus 4 87.3% thinking May 2025 Official leaderboard
18 Claude Opus 4.5 87.3% thinking Nov 2025 Official leaderboard
19 Claude Sonnet 4.6 87.3% thinking Feb 2026 Official leaderboard
20 Hunyuan-T1 87.2% Mar 2025 Official leaderboard
21 GPT-5 87.1% high Aug 2025 Official leaderboard
22 Kimi K2.5 87.1% Jan 2026 Lab model card
23 Seed-Thinking-v1.5 87% Apr 2025 Official leaderboard
24 Grok 4 87% Jul 2025 Official leaderboard
25 Seed2.0 Pro 87% Feb 2026 Official leaderboard
26 Nemotron 3 Ultra 86.8% Jun 2026 Lab model card
27 Qwen3.5-122B-A10B 86.7% Feb 2026 Official leaderboard
28 Seed1.6-Thinking 86.6% Jun 2025 Official leaderboard
29 Intern-S1-Pro 86.6% Feb 2026 Lab model card
30 GPT-5.1 86.4% Nov 2025 Official leaderboard
31 Solar Pro 4 86.3% Aug 2026 Lab model card
32 Gemini 3.1 Flash-Lite 86.2% Mar 2026 Official leaderboard
33 Qwen3.6 27B 86.2% Apr 2026 Lab model card
34 DeepSeek-V4-Flash 86.2% Apr 2026 Lab model card
35 Solar Open 2 86.2% Jul 2026 Lab model card
36 GPT-4.5 86.1% Feb 2025 Official leaderboard
37 Qwen3.5-27B 86.1% Feb 2026 Official leaderboard
38 Gemini 2.5 Pro Jun 2025 86% Jun 2025 Official leaderboard
39 GLM-5 86% Feb 2026 Official leaderboard
40 Qwen3.5-Omni-Plus 85.9% Mar 2026 Lab model card
41 Qwen3-Max 85.7% Sep 2025 Official leaderboard
42 Qwen3-Max-2026-01-23 85.7% Jan 2026 Lab model card
43 Cogito v2 Preview 671B MoE 85.4% Jul 2025 Lab model card
44 Qwen3.5-35B-A3B 85.3% Feb 2026 Official leaderboard
45 Gemma 4 31B IT 85.2% Apr 2026 Lab model card
46 Qwen 3.6 35B-A3B 85.2% Apr 2026 Lab model card
47 MiniMax-M2.5 85.2% May 2026 Lab model card
48 o3 85% high Apr 2025 Official leaderboard
49 DeepSeek-R1 May 2025 85% May 2025 Lab model card
50 DeepSeek-V3.1-Terminus 85% Sep 2025 Lab model card
51 DeepSeek-V3.2-Exp 85% Sep 2025 Lab model card
52 DeepSeek-V3.2 85% Dec 2025 Official leaderboard
53 MAI-Thinking-1 85% Jun 2026 Lab model card
54 MiMo-V2-Flash 84.9% Dec 2025 Lab model card
55 Seed1.8 84.9% Dec 2025 Lab model card
56 DeepSeek-V3.1 84.8% Aug 2025 Official leaderboard
57 Cogito v2.1 84.7% Nov 2025 Lab model card
58 GLM-4.5 84.6% Aug 2025 Official leaderboard
59 Kimi K2 Thinking 84.6% Nov 2025 Lab model card
60 Qwen3-235B-A22B-Thinking Jul 2025 84.5% Jul 2025 Official leaderboard
61 Step 3.5 Flash 84.4% Feb 2026 Lab model card
62 GLM-4.7 84.3% Dec 2025 Lab model card
63 ERNIE 5.1 84.3% May 2026 Lab model card
64 Grok 4.1 Fast 84.2% reasoning Nov 2025 Official leaderboard
65 DeepSeek-R1 84% Jan 2025 Lab model card
66 Claude 3.7 Sonnet 84% thinking Feb 2025 Official leaderboard
67 Qwen3-VL-235B-A22B 83.8% Sep 2025 Lab model card
68 K-EXAONE 83.8% Dec 2025 Lab model card
69 ERNIE 5.0 83.8% Feb 2026 Lab model card
70 Nemotron 3 Super 83.7% Mar 2026 Lab model card
71 Claude Sonnet 4 83.7% May 2025 Official leaderboard
72 Seed2.0 Mini 83.6% Feb 2026 Official leaderboard
73 Intern-S1 83.5% Jul 2025 Official leaderboard
74 K-EXAONE 2.0 83.5% Jul 2026 Lab model card
75 MiniCPM-V-4.7-35B-A3B 83.5% Oct 2026 Lab model card
76 Trinity Large Thinking 83.4% Apr 2026 Lab model card
77 EXAONE 4.5 33B 83.3% Apr 2026 Lab model card
78 Grok-3 mini 83% Apr 2025 Official leaderboard
79 Qwen3-235B-A22B-Instruct Jul 2025 83% Jul 2025 Official leaderboard
80 Llama 4 Behemoth 82.8% Apr 2025 Official leaderboard
81 LongCat-Flash-Omni 82.7% Oct 2025 Lab model card
82 seed-oss-36b-instruct 82.7% Aug 2025 Official leaderboard
83 LongCat-Flash-Chat 82.7% Aug 2025 Official leaderboard
84 Qwen3-Next-80B-A3B 82.7% Sep 2025 Lab model card
85 LongCat-Flash-Thinking 82.6% Sep 2025 Lab model card
86 Gemma 4 26B A4B 82.6% Apr 2026 Lab model card
87 Qwen3.5-9B 82.5% Feb 2026 Official leaderboard
88 MiniMax-M2 82% Oct 2025 Official leaderboard
89 Nemotron 3.5 Lightning 30B-A3B 81.9% Aug 2026 Lab model card
90 INTELLECT-3 81.9% Nov 2025 Lab model card
91 GPT-4.1 81.8% Apr 2025 Official leaderboard
92 EXAONE 4.0 32B 81.8% Jul 2025 Lab model card
93 MiniMax-M2.7 81.8% May 2026 Lab model card
94 Sarvam 105B 81.7% Mar 2026 Lab model card
95 Nova 2.0 Pro Preview 81.6% Dec 2025 Lab model card
96 GLM-4.5-Air 81.4% Jul 2025 Official leaderboard
97 DeepSeek-V3 Mar 2025 81.3% Mar 2025 Official leaderboard
98 InternVL3.5-241B-A28B 81.3% Aug 2025 Lab model card
99 Solar Pro 3 (March 2026 update) 81.2% Mar 2026 Lab model card
100 MiniMax-M1-80k 81.1% Jun 2025 Official leaderboard
101 Kimi K2 Jul 2025 81.1% Jul 2025 Lab model card
102 Qwen3-30B-A3B-Thinking Jul 2025 80.9% Jul 2025 Official leaderboard
103 Nova 2 Lite 80.9% Dec 2025 Lab model card
104 gpt-oss-120b 80.8% high Aug 2025 Official leaderboard
105 Nova 2 Omni Preview 80.7% Dec 2025 Lab model card
106 Hermes 4.3 36B 80.7% Dec 2025 Lab model card
107 Hermes 4 405B 80.6% Aug 2025 Lab model card
108 Ring-1T 80.5% Oct 2025 Lab model card
109 Qwen3-Coder-Next 80.5% Feb 2026 Lab model card
110 Llama 4 Maverick 80.5% Apr 2025 Official leaderboard
111 Solar Open 100B 80.4% Jan 2026 Lab model card
112 o1-mini 80.3% Sep 2024 Official leaderboard
113 Doubao-1.5-pro 80.1% Jan 2025 Official leaderboard
114 Sarvam 30B 80% Mar 2026 Lab model card
115 Grok 3 79.9% Apr 2025 Official leaderboard
116 Nemotron-Cascade 2 30B-A3B 79.8% Mar 2026 Lab model card
117 Llama-3.3-Nemotron-Super-49B-v1.5 79.5% Jul 2025 Lab model card
118 o3-mini 79.4% Jan 2025 Official leaderboard
119 Gemini 2.0 Pro 79.1% Feb 2025 Official leaderboard
120 Qwen3.5-4B 79.1% Feb 2026 Official leaderboard
121 Nemotron 3 Ultra 550B-A55B Base 79.1% Jun 2026 Lab model card
122 Hunyuan-TurboS 79% Feb 2025 Official leaderboard
123 Cogito v1 Preview 70B 78.5% Apr 2025 Lab model card
124 ERNIE-4.5-300B-A47B 78.4% Jun 2025 Official leaderboard
125 Nemotron 3 Nano 30B-A3B 78.3% bf16 Dec 2025 Official leaderboard
126 Claude 3.5 Sonnet (October 2024) 78% Oct 2024 Official leaderboard
127 Mistral Small 4 78% Mar 2026 Lab model card
128 GPT-4o Nov 2024 77.9% Nov 2024 Official leaderboard
129 Gemini 2.0 Flash Feb 2025 77.6% Feb 2025 Official leaderboard
130 DiffusionGemma 26B A4B 77.6% Jun 2026 Lab model card
131 Granite 4.2 30B 77.6% Aug 2026 Lab model card
132 Nemotron 3 Nano Omni 30B-A3B 77.3% Apr 2026 Lab model card
133 Mistral Medium 3 77.2% May 2025 Lab model card
134 Gemma 4 12B 77.2% Jun 2026 Lab model card
135 Nemotron Nano 12B v2 VL 77.1% Nov 2025 Lab model card
136 Solar Pro 3 77% Jan 2026 Lab model card
137 Solar Pro 2 76.5% Jul 2025 Lab model card
138 Gemini 2.0 Flash Dec 2024 76.4% Dec 2024 Lab model card
139 Nemotron 3.5 Super 120B-A12B Base 76.3% Oct 2026 Lab model card
140 Claude 3.5 Sonnet 76.1% Jun 2024 Official leaderboard
141 Qwen2.5-Max 76.1% Jan 2025 Official leaderboard
142 STEP3-VL-10B 76% Jan 2026 Lab model card
143 Phi-4-Reasoning-plus 76% Apr 2025 Official leaderboard
144 DeepSeek-V3 75.9% Dec 2024 Lab model card
145 MiniMax-Text-01 75.7% Jan 2025 Official leaderboard
146 Grok-2 Aug 2024 75.5% Aug 2024 Lab model card
147 Grok-2 Dec 2024 75.5% Dec 2024 Official leaderboard
148 Trinity Large Preview 75.3% Feb 2026 Lab model card
149 GPT-4o Aug 2024 74.7% Aug 2024 Official leaderboard
150 Llama 4 Scout 74.3% Apr 2025 Official leaderboard
151 Phi-4-reasoning 74.3% Apr 2025 Official leaderboard
152 NVIDIA Nemotron 3.5 Transcribe 30B-A4B 74.1% Oct 2026 Lab model card
153 DeepSeek V4.1 Flash 74.1% Sep 2026 Lab model card
154 DeepSeek-V4.1-Flash Base 74.1% Sep 2026 Lab model card
155 EXAONE Deep 32B 74% Mar 2025 Lab model card
156 gpt-oss-20b 73.6% high Aug 2025 Official leaderboard
157 Llama 3.1-405B 73.3% Jul 2024 Official leaderboard
158 Athene-V2-Chat 73.1% Nov 2024 Official leaderboard
159 Beam 1.0 Base 72.6% Oct 2026 Lab model card
160 GPT-4o May 2024 72.5% May 2024 Official leaderboard
161 Falcon H1R 7B 72.1% Jan 2026 Lab model card
162 Grok-2 mini 71.8% Aug 2024 Official leaderboard
163 Gemini 2.0 Flash-Lite 71.6% Feb 2025 Official leaderboard
164 Qwen2.5-72B 71.6% Sep 2024 Official leaderboard
165 Qwen2.5-VL-72B 71.2% Feb 2025 Lab model card
166 QwQ-32B-Preview 71% Nov 2024 Official leaderboard
167 Phi-4 70.4% Dec 2024 Official leaderboard
168 Gemini 1.5 Pro Sep 2024 70.3% Sep 2024 Official leaderboard
169 Cohere Command A 69.6% Apr 2025 Lab model card
170 ERNIE-4.5-300B-A47B-Base 69.5% Jun 2025 Official leaderboard
171 Gemma 4 E4B 69.4% Apr 2026 Lab model card
172 Qwen2.5-32B 69.2% Sep 2024 Official leaderboard
173 QwQ-32B 69.1% Mar 2025 Official leaderboard
174 Mistral Small 3.2 69.1% Jun 2025 Lab model card
175 Mercury 69% Jun 2025 Lab model card
176 Llama 3.3 70B Instruct 68.9% Dec 2024 Lab model card
177 Kimi-VL-A3B-Thinking-2506 68.5% Jun 2025 Lab model card
178 Claude 3 Opus 68.5% Feb 2024 Official leaderboard
179 Qwen3-235B-A22B 68.2% Apr 2025 Official leaderboard
180 Mistral Large 2 Nov 2024 67.9% Nov 2024 Official leaderboard
181 Gemma 3 27B 67.5% Mar 2025 Official leaderboard
182 Nemotron 3.5 Lightning 30B-A3B Base 67.5% Oct 2026 Lab model card
183 Hunyuan-A13B-Instruct 67.3% Jun 2025 Official leaderboard
184 Mistral Small 3.1 66.8% Mar 2025 Official leaderboard
185 Qwen3.5-2B 66.5% Mar 2026 Lab model card
186 Llama 3.1-70B 66.4% Jul 2024 Lab model card
187 Mistral Small 3 66.3% Jan 2025 Lab model card
188 Mistral Large 2 Jul 2024 65.9% Jul 2024 Official leaderboard
189 DeepSeek-V2.5 Sep 2024 65.8% Sep 2024 Official leaderboard
190 Seed-OSS-36B-Base 65.1% Aug 2025 Official leaderboard
191 Nemotron 3 Nano 30B-A3B Base 65.1% Dec 2025 Official leaderboard
192 Claude 3.5 Haiku 65% Oct 2024 Lab model card
193 Reka Flash 3 65% Mar 2025 Lab model card
194 Qwen-Turbo 64.5% Dec 2024 Lab model card
195 Qwen2-72B 64.4% Jun 2024 Lab model card
196 Gemini 1.5 Flash Sep 2024 64.1% Aug 2024 Official leaderboard
197 Granite 4.1 30B 64.1% Apr 2026 Lab model card
198 GPT-4 Turbo Apr 2024 63.7% Apr 2024 Official leaderboard
199 Qwen2.5-14B 63.7% Dec 2024 Lab model card
200 DeepSeek-Coder-V2 236B 63.6% Jun 2024 Official leaderboard
201 Qwen2.5-14B-Instruct-1M 63.3% Jan 2025 Lab model card
202 GPT-4o mini 63.1% Jul 2024 Official leaderboard
203 Llama-3.1-Nemotron-70B-Instruct 62.8% Oct 2024 Official leaderboard
204 Yi-Lightning 62.4% Dec 2024 Official leaderboard
205 Qwen3-30B-A3B-Base 61.7% Apr 2025 Official leaderboard
206 Gemma 3 12B 60.6% Mar 2025 Official leaderboard
207 Nemotron-H-56B-Base 60.5% Apr 2025 Official leaderboard
208 Seed-OSS-36B-Base-woSyn 60.4% Aug 2025 Official leaderboard
209 Code World Model (CWM) 60.2% Sep 2025 Lab model card
210 Gemma 4 E2B 60% Apr 2026 Lab model card
211 EXAONE-3.5-32B-Instruct 58.9% Dec 2024 Official leaderboard
212 Falcon-H1-34B 58.7% May 2025 Lab model card
213 MiMo-7B-RL 58.6% Apr 2025 Official leaderboard
214 Yi-Large 58.1% May 2024 Official leaderboard
215 InternLM3-8B-Instruct 57.6% Jan 2025 Official leaderboard
216 Claude 3 Sonnet 56.8% Feb 2024 Official leaderboard
217 ERNIE-4.5-21B-A3B-Base 56.7% Jun 2025 Official leaderboard
218 Gemma 2 27B 56.5% Jun 2024 Official leaderboard
219 Mixtral 8x22B 56.3% Apr 2024 Official leaderboard
220 Llama 3-70B 56.2% Apr 2024 Official leaderboard
221 phi-3-medium 14B 55.7% Apr 2024 Official leaderboard
222 Qwen2.5-Turbo 55.6% Nov 2024 Official leaderboard
223 Granite 4.0 H Small 55.5% Oct 2025 Lab model card
224 DeepSeek-V2 (MoE-236B, May 2024) 54.8% May 2024 Official leaderboard
225 Phi-3.5-MoE 54.3% Aug 2024 Lab model card
226 Phi-4 Mini 52.8% Feb 2025 Official leaderboard
227 OLMo 2 32B Instruct 52.7% Dec 2024 Lab model card
228 Qwen1.5-72B 52.6% Feb 2024 Official leaderboard
229 Yi-1.5-34B 52.3% May 2024 Official leaderboard
230 Gemma 2 9B 52.1% Jun 2024 Official leaderboard
231 Phi-3-medium-128k-instruct 51.9% May 2024 Official leaderboard
232 Gemma 3n E4B 50.6% Jun 2025 Lab model card
233 MAmmoTH2-8x7B-Plus 50.4% May 2024 Official leaderboard
234 Qwen1.5-110B 49.9% Apr 2024 Official leaderboard
235 LFM2.5-1.2B-Thinking 49.6% Jan 2026 Lab model card
236 Jamba 1.5 Large 49.5% Aug 2024 Official leaderboard
237 Mistral Small v24.09 48.4% Sep 2024 Official leaderboard
238 Llama 3.1-8B 48.3% Jul 2024 Lab model card
239 GLM-4-9B-Chat 48% Jun 2024 Official leaderboard
240 GLM-4-9B 47.9% Jun 2024 Official leaderboard
241 Phi-3.5-mini 47.9% Aug 2024 Official leaderboard
242 Qwen2-7B-Instruct 47.2% Jun 2024 Official leaderboard
243 Qwen2.5-Omni-7B 47% Mar 2025 Lab model card
244 EXAONE-3.5-7.8B-Instruct 46.2% Dec 2024 Official leaderboard
245 Yi-1.5-9B-Chat 46% May 2024 Official leaderboard
246 phi-3-mini 3.8B 45.7% Apr 2024 Official leaderboard
247 Aya Expanse 32B 45.4% Oct 2024 Official leaderboard
248 Qwen2.5-7B 45% Sep 2024 Official leaderboard
249 Mistral NeMo 44.8% Jul 2024 Official leaderboard
250 LFM2.5-1.2B-Instruct 44.4% Jan 2026 Lab model card
251 Nemotron-H-8B-Base 44% Mar 2025 Official leaderboard
252 Phi-3-mini-128k-instruct 43.9% Apr 2024 Official leaderboard
253 Qwen2.5-3B 43.7% Sep 2024 Official leaderboard
254 Gemma 3 4B 43.6% Mar 2025 Official leaderboard
255 MAmmoTH2-8B-Plus 43.4% May 2024 Official leaderboard
256 Mixtral 8x7B 43.3% Dec 2023 Official leaderboard
257 Yi-34B 43% Nov 2023 Official leaderboard
258 Moonlight-16B-A3B 42.4% Feb 2025 Lab model card
259 Qwen3.5-0.8B 42.3% Mar 2026 Lab model card
260 Claude 3 Haiku 42.3% Mar 2024 Official leaderboard
261 Mathstral-7B-v0.1 42% Jul 2024 Official leaderboard
262 MiMo-7B-Base 41.9% Apr 2025 Official leaderboard
263 DeepSeek-Coder-V2-Lite-Instruct 41.6% Jun 2024 Official leaderboard
264 Granite-3.1-8B-Instruct 41% Dec 2024 Official leaderboard
265 Llama 3-8B 41% Apr 2024 Official leaderboard
266 MAmmoTH2-7B-Plus 40.9% May 2024 Official leaderboard
267 Qwen2-7B 40.7% Jun 2024 Official leaderboard
268 Mistral-Nemo-Base-2407 39.8% Jul 2024 Official leaderboard
269 WizardLM-2 8x22B 39.2% Apr 2024 Official leaderboard
270 EXAONE-3.5-2.4B-Instruct 39.1% Dec 2024 Official leaderboard
271 Yi-1.5-6B-Chat 38.2% May 2024 Official leaderboard
272 Qwen1.5-14B-Chat 38% Feb 2024 Official leaderboard
273 Ministral 8B 37.9% Oct 2024 Official leaderboard
274 Command R (03-2024) 37.9% Mar 2024 Official leaderboard
275 Llama 2-70B 37.5% Jul 2023 Official leaderboard
276 LFM2-8B-A1B 37.4% Oct 2025 Lab model card
277 DeepSeekMath-7B-Instruct 35.3% Feb 2024 Official leaderboard
278 DeepSeek-Coder-V2-Lite-Base 34.4% Jun 2024 Official leaderboard
279 granite-4.0-1b 34% Oct 2025 Lab model card
280 Aya Expanse 8B 33.7% Oct 2024 Official leaderboard
281 Gemma 7B 33.7% Feb 2024 Official leaderboard
282 Granite-3.1-8B-Base 33.1% Dec 2024 Official leaderboard
283 Zephyr 7B Beta 33% Oct 2023 Official leaderboard
284 Qwen2.5-1.5B 32.1% Sep 2024 Official leaderboard
285 Granite-3.1-2B-Instruct 32% Dec 2024 Official leaderboard
286 Granite-3.0-8B-Base 31% Oct 2024 Official leaderboard
287 Mistral 7B v0.1 30.9% Sep 2023 Official leaderboard
288 Mistral-7B-Instruct-v0.2 30.8% Dec 2023 Official leaderboard
289 North Micro Vision Instruct 30.7% Aug 2026 Lab model card
290 Mistral 7B v0.2 30.4% Dec 2023 Official leaderboard
291 Qwen1.5-7B-Chat 29.1% Feb 2024 Official leaderboard
292 Yi-6B-Chat 28.8% Nov 2023 Official leaderboard
293 Yi 6B 26.5% Nov 2023 Official leaderboard
294 Mistral-7B-Instruct-v0.1 25.8% Sep 2023 Official leaderboard
295 Granite-3.1-3B-A800M-Instruct 25.4% Dec 2024 Official leaderboard
296 Llama 2-13B 25.3% Jul 2023 Official leaderboard
297 Granite-3.1-2B-Base 23.9% Dec 2024 Official leaderboard
298 Llemma 7B 23.4% Oct 2023 Official leaderboard
299 Qwen2-1.5B-Instruct 22.6% Jun 2024 Official leaderboard
300 Qwen2-1.5B 22.6% Jun 2024 Official leaderboard
301 Llama 3.2 3B 22.2% Sep 2024 Official leaderboard
302 Granite-3.0-2B-Base 21.7% Oct 2024 Official leaderboard
303 Granite-3.1-3B-A800M-Base 20.4% Dec 2024 Official leaderboard
304 Llama 2-7B 20.3% Jul 2023 Official leaderboard
305 LFM2-1.2B 19.7% Nov 2025 Lab model card
306 SmolLM2-1.7B 18.3% Oct 2024 Official leaderboard
307 Qwen2-0.5B-Instruct 15.9% Jun 2024 Official leaderboard
308 Gemma 2B 15.8% Feb 2024 Official leaderboard
309 Gemma 2 2B IT 15.6% Jul 2024 Official leaderboard
310 Qwen2-0.5B 15% Jun 2024 Official leaderboard
311 Qwen2.5-0.5B 14.9% Sep 2024 Official leaderboard
312 Gemma 3 1B 14.7% Mar 2025 Official leaderboard
313 Granite-3.1-1B-A400M-Instruct 13.3% Dec 2024 Official leaderboard
314 Granite-3.1-1B-A400M-Base 12.3% Dec 2024 Official leaderboard
315 Llama 3.2 1B 11.9% Sep 2024 Official leaderboard
316 SmolLM-1.7B 11.9% Jul 2024 Official leaderboard
317 SmolLM2-360M 11.4% Oct 2024 Official leaderboard
318 SmolLM-135M 11.2% Jul 2024 Official leaderboard
319 SmolLM-360M 10.9% Jul 2024 Official leaderboard
320 SmolLM2-135M 10.8% Oct 2024 Official leaderboard

The benchmark's own leaderboard has 240 entries, 212 of them models we track. Switch the scores above to rank them together.

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
DeepSeek-V3.1
non-thinking
84.8%
Grok 4.1 Fast
non-reasoni… reasoning
84.2%
Nemotron 3 Nano 30B-A3B
fp8
78.3%
gpt-oss-20b
medium
73.6%

Source: Official MMLU-Pro leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Gemini 3.1 Pro 91.2%
Gemini 3 Pro 90.1%
o1 89.3%
Claude Opus 4.6 thinking 89.1%
Gemini 3 Flash 88.6%
MiniMax-M2.1 88%
Qwen3.5 397B-A17B 87.8%
Seed2.0 Lite (0215) 87.7%
GPT-5.4 87.5%
Claude Sonnet 4.5 thinking 87.4%
GPT-5.2 87.4%
Claude Opus 4 thinking 87.3%
Claude Opus 4.5 thinking 87.3%
Claude Sonnet 4.6 thinking 87.3%
Hunyuan-T1 87.2%
GPT-5 high 87.1%
Seed-Thinking-v1.5 87%
Grok 4 87%
Seed2.0 Pro 87%
Qwen3.5-122B-A10B 86.7%
Seed1.6-Thinking 86.6%
GPT-5.1 86.4%
Gemini 3.1 Flash-Lite 86.2%
GPT-4.5 86.1%
Qwen3.5-27B 86.1%
Gemini 2.5 Pro Jun 2025 86%
GLM-5 86%
Qwen3-Max 85.7%
Qwen3.5-35B-A3B 85.3%
o3 high 85%
DeepSeek-V3.2 85%
DeepSeek-V3.1 84.8%
GLM-4.5 84.6%
Gemini 2.5 Pro Jun 2025 84.5%
Qwen3-235B-A22B-Thinking Jul 2025 84.5%
Grok 4.1 Fast reasoning 84.2%
Claude 3.7 Sonnet thinking 84%
DeepSeek-R1 May 2025 84%
Claude Sonnet 4 83.7%
DeepSeek-V3.1 non-thinking 83.7%
Seed2.0 Mini 83.6%
Intern-S1 83.5%
DeepSeek-R1 May 2025 83.4%
Grok-3 mini 83%
Qwen3-235B-A22B-Instruct Jul 2025 83%
Llama 4 Behemoth 82.8%
seed-oss-36b-instruct 82.7%
LongCat-Flash-Chat 82.7%
Qwen3.5-9B 82.5%
MiniMax-M2 82%
GPT-4.1 81.8%
GLM-4.5-Air 81.4%
DeepSeek-V3 Mar 2025 81.3%
MiniMax-M1-80k 81.1%
Kimi K2 Jul 2025 81%
Qwen3-30B-A3B-Thinking Jul 2025 80.9%
gpt-oss-120b high 80.8%
Llama 4 Maverick 80.5%
o1-mini 80.3%
Doubao-1.5-pro 80.1%
MiniMax-M2.5 80.1%
Grok 3 79.9%
o3-mini 79.4%
Gemini 2.0 Pro 79.1%
Qwen3.5-4B 79.1%
Hunyuan-TurboS 79%
Grok-3 mini 78.9%
Qwen3-30B-A3B-Thinking Jul 2025 78.5%
ERNIE-4.5-300B-A47B 78.4%
Nemotron 3 Nano 30B-A3B bf16 78.3%
Nemotron 3 Nano 30B-A3B fp8 78.1%
Claude 3.5 Sonnet (October 2024) 78%
GPT-4o Nov 2024 77.9%
Claude 3.5 Sonnet (October 2024) 77.6%
Gemini 2.0 Flash Feb 2025 77.6%
Gemini 2.0 Flash Feb 2025 76.2%
Claude 3.5 Sonnet 76.1%
Qwen2.5-Max 76.1%
Phi-4-Reasoning-plus 76%
DeepSeek-V3 Mar 2025 75.9%
MiniMax-Text-01 75.7%
Grok-2 Dec 2024 75.5%
Grok 4.1 Fast non-reasoning 75.2%
GPT-4o Aug 2024 74.7%
Llama 4 Scout 74.3%
Phi-4-reasoning 74.3%
gpt-oss-20b high 73.6%
Llama 3.1-405B 73.3%
gpt-oss-20b medium 73.1%
Athene-V2-Chat 73.1%
GPT-4o May 2024 72.5%
Grok-2 mini 71.8%
Gemini 2.0 Flash-Lite 71.6%
Qwen2.5-72B 71.6%
QwQ-32B-Preview 71%
Phi-4 70.4%
Gemini 1.5 Pro Sep 2024 70.3%
Athene-V2-Chat 70.2%
ERNIE-4.5-300B-A47B-Base 69.5%
Qwen2.5-32B 69.2%
QwQ-32B 69.1%
Gemini 1.5 Pro Sep 2024 69%
Claude 3 Opus 68.5%
Qwen3-235B-A22B 68.2%
Mistral Large 2 Nov 2024 67.9%
Gemma 3 27B 67.5%
Hunyuan-A13B-Instruct 67.3%
Mistral Small 3.1 66.8%
Llama 3.3 70B Instruct 65.9%
Mistral Large 2 Jul 2024 65.9%
DeepSeek-V2.5 Sep 2024 65.8%
Seed-OSS-36B-Base 65.1%
Nemotron 3 Nano 30B-A3B Base 65.1%
Qwen2-72B 64.4%
Gemini 1.5 Flash Sep 2024 64.1%
GPT-4 Turbo Apr 2024 63.7%
Qwen2.5-14B 63.7%
DeepSeek-Coder-V2 236B 63.6%
GPT-4o mini 63.1%
Llama 3.1-70B 62.8%
Llama-3.1-Nemotron-70B-Instruct 62.8%
Yi-Lightning 62.4%
Claude 3.5 Haiku 62.1%
Qwen3-30B-A3B-Base 61.7%
Llama 3.1-405B 61.6%
Gemma 3 12B 60.6%
Nemotron-H-56B-Base 60.5%
Seed-OSS-36B-Base-woSyn 60.4%
Gemini 1.5 Flash Sep 2024 59.1%
EXAONE-3.5-32B-Instruct 58.9%
MiMo-7B-RL 58.6%
Yi-Large 58.1%
InternLM3-8B-Instruct 57.6%
Claude 3 Sonnet 56.8%
ERNIE-4.5-21B-A3B-Base 56.7%
Gemma 2 27B 56.5%
Mixtral 8x22B 56.3%
Llama 3-70B 56.2%
phi-3-medium 14B 55.7%
Qwen2.5-Turbo 55.6%
Qwen3.5-2B 55.3%
DeepSeek-V2 (MoE-236B, May 2024) 54.8%
Phi-4 Mini 52.8%
Llama 3-70B 52.8%
Qwen1.5-72B 52.6%
Llama 3.1-70B 52.5%
Yi-1.5-34B 52.3%
Gemma 2 9B 52.1%
Phi-3-medium-128k-instruct 51.9%
MAmmoTH2-8x7B-Plus 50.4%
Qwen1.5-110B 49.9%
Jamba 1.5 Large 49.5%
Mistral Small v24.09 48.4%
GLM-4-9B-Chat 48%
GLM-4-9B 47.9%
Phi-3.5-mini 47.9%
Qwen2-7B-Instruct 47.2%
EXAONE-3.5-7.8B-Instruct 46.2%
Yi-1.5-9B-Chat 46%
phi-3-mini 3.8B 45.7%
Aya Expanse 32B 45.4%
Gemma 2 9B 45.1%
Qwen2.5-7B 45%
Mistral NeMo 44.8%
Llama 3.1-8B 44.3%
Nemotron-H-8B-Base 44%
Phi-3-mini-128k-instruct 43.9%
Qwen2.5-3B 43.7%
Gemma 3 4B 43.6%
MAmmoTH2-8B-Plus 43.4%
Mixtral 8x7B 43.3%
Yi-34B 43%
Claude 3 Haiku 42.3%
Mathstral-7B-v0.1 42%
MiMo-7B-Base 41.9%
DeepSeek-Coder-V2-Lite-Instruct 41.6%
Mixtral 8x7B 41%
Granite-3.1-8B-Instruct 41%
Llama 3-8B 41%
MAmmoTH2-7B-Plus 40.9%
Qwen2-7B 40.7%
Mistral-Nemo-Base-2407 39.8%
WizardLM-2 8x22B 39.2%
EXAONE-3.5-2.4B-Instruct 39.1%
Yi-1.5-6B-Chat 38.2%
Qwen1.5-14B-Chat 38%
Ministral 8B 37.9%
Command R (03-2024) 37.9%
Llama 2-70B 37.5%
Llama 3.1-8B 36.6%
Llama 3-8B 35.4%
DeepSeekMath-7B-Instruct 35.3%
DeepSeek-Coder-V2-Lite-Base 34.4%
Aya Expanse 8B 33.7%
Gemma 7B 33.7%
Granite-3.1-8B-Base 33.1%
Zephyr 7B Beta 33%
Qwen2.5-1.5B 32.1%
Granite-3.1-2B-Instruct 32%
Granite-3.0-8B-Base 31%
Mistral 7B v0.1 30.9%
Mistral-7B-Instruct-v0.2 30.8%
Mistral 7B v0.2 30.4%
Qwen3.5-0.8B 29.7%
Qwen1.5-7B-Chat 29.1%
Yi-6B-Chat 28.8%
Yi 6B 26.5%
Mistral-7B-Instruct-v0.1 25.8%
Granite-3.1-3B-A800M-Instruct 25.4%
Llama 2-13B 25.3%
Granite-3.1-2B-Base 23.9%
Llemma 7B 23.4%
Qwen2-1.5B-Instruct 22.6%
Qwen2-1.5B 22.6%
Llama 3.2 3B 22.2%
Granite-3.0-2B-Base 21.7%
Granite-3.1-3B-A800M-Base 20.4%
Llama 2-7B 20.3%
SmolLM2-1.7B 18.3%
Qwen2-0.5B-Instruct 15.9%
Gemma 2B 15.8%
Gemma 2 2B IT 15.6%
Qwen2-0.5B 15%
Qwen2.5-0.5B 14.9%
Gemma 3 1B 14.7%
Granite-3.1-1B-A400M-Instruct 13.3%
Granite-3.1-1B-A400M-Base 12.3%
Llama 3.2 1B 11.9%
SmolLM-1.7B 11.9%
SmolLM2-360M 11.4%
SmolLM-135M 11.2%
SmolLM-360M 10.9%
SmolLM2-135M 10.8%

10 models have held number one in labs’ own numbers since its release in June 2024 6 models have held number one across all independent runs since its release in June 2024 6 models have held number one with claims included since its release in June 2024

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

OpenAI Anthropic Google DeepMind Qwen Meta xAI DeepSeek Tencent MiniMax Shanghai AI Lab
Own leaderboard Passed 85% in Dec 2024, by o1
Lab claims Passed 85% in Mar 2025, by Hunyuan-T1

Sources: Official MMLU-Pro leaderboard and labs’ model cards

O1 on the official leaderboard passed 85% in December 2024

That was 6 months after its release. By the same age, 15% of the benchmarks from 2024 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) Gemini 3.1 Pro o1 o1-mini Claude 3.5 Sonnet Claude 3 Opus Qwen1.5-72B Mixtral 8x7B 0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) Gemini 3.1 Pro o1-mini

Sources: Official MMLU-Pro leaderboard and labs’ model cards

11 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Qwen
Meta
xAI
Microsoft
Anthropic
Google DeepMind
DeepSeek
Shanghai AI Lab
MiniMax
Mistral AI
Moonshot AI
Tencent
Reka
Baidu
Deep Cogito
ByteDance Seed
Xiaomi Corp
Technology Innovation Institute
Inception Labs
Upstage
LG AI Research
NVIDIA
Z.ai
Nous Research
Meituan
IBM
Liquid AI
Prime Intellect
Amazon
Arcee AI
Sarvam AI
Cohere
OpenBMB (ModelBest)
DateLaunch postScoreAs the post writes it
Oct 28, 2026 NVIDIA Nemotron 3.5 Transcribe 30B-A4B 74.1 MMLU-Pro — Accuracy
Oct 7, 2026 MiniCPM-V-4.7-35B-A3B 83.5 MMLU-Pro
Sep 13, 2026 Intern-S2-397B 89.8 MMLU Pro
Sep 10, 2026 DeepSeek V4.1 Flash 74.1 MMLU-Pro (EM)
Aug 25, 2026 Granite 4.2 30B 77.6 MMLU-Pro
Aug 12, 2026 North Micro Vision Instruct 30.7 MMLU-Pro test
Aug 11, 2026 Nemotron 3.5 Lightning 30B-A3B 81.9 MMLU Pro
Aug 11, 2026 Solar Pro 4 86.3 MMLU-Pro* (in-house)
Jul 31, 2026 K-EXAONE 2.0 83.5 MMLU-Pro
Jul 22, 2026 Solar Open 2 86.2 MMLU-Pro
Jun 10, 2026 DiffusionGemma 26B A4B 77.6 MMLU Pro
Jun 4, 2026 Nemotron 3 Ultra 86.8 MMLU-Pro
Jun 3, 2026 Gemma 4 12B 77.2 MMLU Pro
Jun 2, 2026 MAI-Thinking-1 85 MMLU Pro
Jun 1, 2026 Qwen3.7-Plus 88.5 MMLU-Pro
May 20, 2026 Qwen3.7-Max 89.6 MMLU-Pro
May 9, 2026 ERNIE 5.1 84.3 MMLU-Pro (acc)
Apr 29, 2026 Granite 4.1 30B 64.1 MMLU-Pro (5-shot, CoT)
Apr 24, 2026 DeepSeek-V4-Flash 86.2 MMLU-Pro (EM), V4-Flash Max
Apr 24, 2026 DeepSeek-V4-Pro 87.5 MMLU-Pro (EM)
Apr 22, 2026 Qwen3.6 27B 86.2 MMLU-Pro
Apr 15, 2026 Qwen 3.6 35B-A3B 85.2 MMLU-Pro
Apr 9, 2026 EXAONE 4.5 33B 83.3 MMLU-Pro
Apr 2, 2026 Gemma 4 E4B 69.4 MMLU Pro
Apr 2, 2026 Gemma 4 E2B 60 MMLU Pro
Apr 2, 2026 Gemma 4 26B A4B 82.6 MMLU Pro
Apr 2, 2026 Qwen 3.6 Plus 88.5 MMLU-Pro
Apr 2, 2026 Gemma 4 31B IT 85.2 MMLU Pro
Apr 1, 2026 Trinity Large Thinking 83.4 MMLU-Pro
Mar 30, 2026 Qwen3.5-Omni-Plus 85.9 MMLU-Pro
Mar 24, 2026 Solar Pro 3 (March 2026 update) 81.2 MMLU PRO (0-shot, CoT)
Mar 19, 2026 Nemotron-Cascade 2 30B-A3B 79.8 MMLU-Pro
Mar 16, 2026 Mistral Small 4 78 MMLU Pro (reasoning; instruct = 73.5)
Mar 11, 2026 Nemotron 3 Super 83.7 MMLU-Pro
Mar 6, 2026 Sarvam 30B 80 MMLU Pro
Mar 6, 2026 Sarvam 105B 81.7 MMLU Pro
Mar 2, 2026 Qwen3.5-0.8B 42.3 MMLU-Pro (Thinking)
Mar 2, 2026 Qwen3.5-2B 66.5 MMLU-Pro (Thinking)
Mar 2, 2026 Qwen3.5-4B 79.1 MMLU-Pro
Mar 2, 2026 Qwen3.5-9B 82.5 MMLU-Pro
Feb 24, 2026 Qwen3.5-35B-A3B 85.3 MMLU-Pro
Feb 24, 2026 Qwen3.5-27B 86.1 MMLU-Pro
Feb 24, 2026 Qwen3.5-122B-A10B 86.7 MMLU-Pro
Feb 16, 2026 Qwen3.5 397B-A17B 87.8 MMLU-Pro
Feb 14, 2026 Seed2.0 Pro 87 MMLU-Pro
Feb 6, 2026 ERNIE 5.0 83.8 MMLU-Pro
Feb 2, 2026 Intern-S1-Pro 86.6 MMLU-Pro
Jan 27, 2026 Trinity Large Preview 75.2 MMLU-Pro
Jan 27, 2026 Kimi K2.5 87.1 MMLU-Pro
Jan 26, 2026 Solar Pro 3 77 MMLU-Pro
Jan 23, 2026 Qwen3-Max-2026-01-23 85.7 MMLU-Pro
Jan 20, 2026 LFM2.5-1.2B-Thinking 49.6 MMLU-Pro (± 0.18)
Jan 5, 2026 Falcon H1R 7B 72.1 MMLU-Pro
Jan 5, 2026 LFM2.5-1.2B-Instruct 44.4 MMLU-Pro
Jan 5, 2026 Solar Open 100B 80.4 MMLU-Pro
Dec 31, 2025 K-EXAONE 83.8 MMLU-Pro
Dec 23, 2025 MiniMax-M2.1 88 MMLU-Pro
Dec 22, 2025 GLM-4.7 84.3 MMLU-Pro
Dec 18, 2025 Seed1.8 84.9 MMLU-pro
Dec 16, 2025 MiMo-V2-Flash 84.9 MMLU-Pro
Dec 15, 2025 Nemotron 3 Nano 30B-A3B 78.3 MMLU-Pro
Dec 3, 2025 Hermes 4.3 36B 80.7 MMLU-Pro
Dec 2, 2025 Nova 2 Omni Preview 80.7 MMLU-Pro acc
Dec 2, 2025 Nova 2 Lite 80.9 MMLU-Pro acc
Dec 2, 2025 Nova 2.0 Pro Preview 81.6 MMLU-Pro acc
Dec 1, 2025 DeepSeek-V3.2 85 MMLU-Pro (EM)
Nov 26, 2025 INTELLECT-3 81.9 MMLU-Pro
Nov 19, 2025 Cogito v2.1 84.7 MMLU Pro
Nov 6, 2025 Kimi K2 Thinking 84.6 MMLU-Pro, no tools
Oct 31, 2025 LongCat-Flash-Omni 82.7 MMLU-Pro(acc)
Oct 28, 2025 granite-4.0-1b 34 MMLU-Pro (5-shot, CoT)
Oct 27, 2025 MiniMax-M2 82 MMLU-Pro (AA)
Oct 7, 2025 LFM2-8B-A1B 37.4 MMLU-Pro
Oct 2, 2025 Granite 4.0 H Small 55.5 MMLU-Pro (5-shot, CoT)
Sep 29, 2025 DeepSeek-V3.2-Exp 85 MMLU-Pro
Sep 23, 2025 Qwen3-VL-235B-A22B 83.8 MMLU-Pro
Sep 22, 2025 DeepSeek-V3.1-Terminus 85 MMLU-Pro
Sep 21, 2025 LongCat-Flash-Thinking 82.6 MMLU-Pro(acc)
Sep 11, 2025 Qwen3-Next-80B-A3B 82.7 MMLU-Pro
Aug 29, 2025 LongCat-Flash-Chat 82.7 MMLU-Pro(acc)
Aug 26, 2025 Hermes 4 405B 80.5 MMLU-Pro (reasoning)
Aug 25, 2025 InternVL3.5-241B-A28B 81.3 MMLU-Pro
Aug 21, 2025 DeepSeek-V3.1 84.8 MMLU-Pro (EM)
Aug 20, 2025 seed-oss-36b-instruct 82.7 MMLU-Pro
Jul 31, 2025 Cogito v2 Preview 671B MoE 85.4 MMLU-Pro (thinking)
Jul 28, 2025 GLM-4.5 84.6 MMLU Pro
Jul 25, 2025 Llama-3.3-Nemotron-Super-49B-v1.5 79.5 MMLU Pro (CoT)
Jul 25, 2025 Qwen3-235B-A22B-Thinking 84.4 MMLU-Pro
Jul 24, 2025 Intern-S1 83.5 MMLU-Pro
Jul 15, 2025 EXAONE 4.0 32B 81.8 MMLU-Pro (reasoning)
Jul 11, 2025 Kimi K2 81.1 MMLU-Pro, EM
Jul 10, 2025 Solar Pro 2 76.5 MMLU-Pro
Jun 30, 2025 ERNIE-4.5-300B-A47B 78.4 MMLU-Pro
Jun 26, 2025 Gemma 3n E4B 50.6 MMLU (Pro) 0-shot (IT)
Jun 26, 2025 Mercury 69 MMLU-Pro (% accuracy)
Jun 21, 2025 Kimi-VL-A3B-Thinking-2506 68.5 MMLU-Pro
Jun 20, 2025 Mistral Small 3.2 69.1 MMLU Pro (5-shot CoT)
Jun 16, 2025 MiniMax-M1-80k 81.1 MMLU-Pro
May 28, 2025 DeepSeek-R1 85 MMLU-Pro (EM)
May 20, 2025 Falcon-H1-34B 58.7 MMLU-Pro
May 7, 2025 Mistral Medium 3 77.2 MMLU Pro 5-shot CoT
Apr 30, 2025 Phi-4-reasoning 74.3 MMLUPro
Apr 30, 2025 Phi-4-Reasoning-plus 76 MMLUPro
Apr 29, 2025 MiMo-7B-RL 58.6 MMLU-Pro (EM)
Apr 10, 2025 Seed-Thinking-v1.5 87 MMLU-PRO
Apr 8, 2025 Cogito v1 Preview 70B 78.5 MMLU-Pro (thinking)
Apr 5, 2025 Llama 4 Scout 74.3 MMLU Pro, 0-shot, macro_avg/acc
Apr 5, 2025 Llama 4 Maverick 80.5 MMLU Pro, 0-shot, macro_avg/acc
Apr 5, 2025 Llama 4 Behemoth 82.2 MMLU Pro
Mar 27, 2025 Qwen2.5-Omni-7B 47 MMLU-Pro
Mar 24, 2025 DeepSeek-V3 81.2 MMLU-Pro
Mar 21, 2025 Hunyuan-T1 87.2 MMLU PRO
Mar 17, 2025 Mistral Small 3.1 66.8 MMLU Pro (5-shot CoT)
Mar 16, 2025 ERNIE 4.5 MMLU-Pro
Mar 12, 2025 Gemma 3 27B 67.5 MMLU (Pro) 0-shot (IT); PT MMLU (Pro COT) 5-shot = 52.2
Mar 10, 2025 Reka Flash 3 65 MMLU-Pro
Feb 27, 2025 Hunyuan-TurboS 79 MMLU-pro
Feb 26, 2025 Phi-4 Mini 52.8 MMLU-Pro (0-shot, CoT)
Feb 22, 2025 Moonlight-16B-A3B 42.4 MMLU-pro
Feb 19, 2025 Grok-3 mini 78.9 MMLU-pro [Grok 3 mini, non-reasoning table]
Feb 19, 2025 Grok 3 79.9 MMLU-pro [Grok 3 Beta, non-reasoning table]
Feb 5, 2025 Gemini 2.0 Flash-Lite 71.6 MMLU-Pro
Feb 5, 2025 Gemini 2.0 Flash 77.6 MMLU-Pro
Feb 5, 2025 Gemini 2.0 Pro 79.1 MMLU-Pro
Jan 30, 2025 Mistral Small 3 66.3 mmlu_pro_5shot_cot_instruct
Jan 28, 2025 Qwen2.5-Max MMLU-Pro
Jan 20, 2025 DeepSeek-R1 84 MMLU-Pro (EM)
Jan 15, 2025 MiniMax-Text-01 75.7 MMLU-Pro*
Jan 13, 2025 InternLM3-8B-Instruct 57.6 MMLU-Pro(0-shot)
Dec 26, 2024 DeepSeek-V3 75.9 MMLU-Pro (EM)
Dec 12, 2024 Phi-4 70.4 MMLUPro
Dec 11, 2024 Gemini 2.0 Flash 76.4 MMLU-Pro
Dec 6, 2024 Llama 3.3 70B Instruct 68.9 MMLU Pro (CoT), 5-shot, macro_avg/acc
Oct 22, 2024 Claude 3.5 Sonnet (October 2024) 78 MMLU Pro (0-shot CoT)
Sep 19, 2024 Qwen2.5-72B 71.1 MMLU-Pro
Aug 20, 2024 Phi-3.5-MoE 54.3 MMLU-Pro (0-shot, CoT)
Aug 13, 2024 Grok-2 75.5 MMLU-Pro (0-shot CoT)
Jul 23, 2024 Llama 3.1-405B 73.3 MMLU-Pro (5-shot, CoT)
Jun 7, 2024 Qwen2-72B 64.4 MMLU-Pro

103 system cards report it, from 28 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 15, 2026 Nemotron 3.5 Super 120B-A12B Base technical report 76.3 MMLU-Pro, exact match, Table 6 Capabilities
Oct 15, 2026 Nemotron 3.5 Lightning 30B-A3B Base technical report 67.5 MMLU-Pro, exact match, Table 6 Capabilities
Oct 5, 2026 Beam 1.0 Base technical report 72.6 MMLU-Pro, 5-shot CoT greedy, Table 23 Capabilities
Sep 10, 2026 DeepSeek-V4.1-Flash Base model card 74.1 MMLU-Pro, EM, 5-shot Capabilities
Aug 5, 2026 K-EXAONE 2.0 technical report 83.5 MMLU-Pro (Table 6, Reasoning mode) Capabilities
Jul 31, 2026 DiffusionGemma 26B A4B technical report 77.6 MMLU-Pro, TD, 77.9 no-think (Table 3) Capabilities
Jul 24, 2026 Gemma 4 E4B technical report 69.4 MMLU Pro, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 E2B technical report 60 MMLU Pro, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 31B IT technical report 85.2 MMLU Pro, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 26B A4B technical report 82.6 MMLU Pro, thinking (Table 5) Capabilities
Jul 22, 2026 Solar Open 2 technical report 86.2 MMLU-Pro (Table 3) Capabilities
Jun 9, 2026 Nemotron 3 Ultra technical report 86.8 MMLU-Pro (Table 10) Capabilities
Jun 4, 2026 Nemotron 3 Ultra 550B-A55B Base model card 79.1 MMLU-Pro, 5-shot CoT EM Capabilities
Jun 2, 2026 MAI-Thinking-1 technical report 85 MMLU Pro (Table 12, rounded) Capabilities
May 26, 2026 MiniMax-M2.7 technical report 81.8 MMLU-Pro, AA Index v4.0 protocol (Table 4) Capabilities
May 26, 2026 MiniMax-M2.5 technical report 85.2 MMLU-Pro, AA Index v4.0 protocol (Table 4) Capabilities
Apr 27, 2026 Nemotron 3 Nano Omni 30B-A3B technical report 77.3 MMLU-Pro (Table 10) Capabilities
Apr 26, 2026 DeepSeek-V4-Pro technical report 87.5 MMLU-Pro (EM), DS-V4-Pro Max (Table 6). High 87.1, Non-Think 82.9 (Table 7) Capabilities
Apr 26, 2026 DeepSeek-V4-Flash technical report 86.2 MMLU-Pro (EM), DeepSeek-V4-Flash Max (Table 7). High 86.4, Non-Think 83.0 Capabilities
Apr 17, 2026 Qwen3.5-Omni-Plus technical report 85.9 MMLU-Pro (Table 4) Capabilities
Apr 9, 2026 EXAONE 4.5 33B technical report 83.3 MMLU-Pro (Table 3, Reasoning mode) Capabilities
Mar 26, 2026 Intern-S1-Pro technical report 86.6 MMLU-Pro, thinking (Table 2) Capabilities
Mar 21, 2026 Seed1.8 model card 84.9 MMLU-pro (Table 1) Capabilities
Mar 19, 2026 Nemotron-Cascade 2 30B-A3B technical report 79.8 MMLU-Pro (Table 1) Capabilities
Mar 11, 2026 Nemotron 3 Super technical report 83.7 MMLU-Pro (Table 5) Capabilities
Feb 28, 2026 Qwen3-Coder-Next technical report 80.5 MMLU-Pro (Table 8) Capabilities
Feb 19, 2026 Trinity Large Preview technical report 75.3 MMLU-Pro, Trinity Large Preview (Table 4) Capabilities
Feb 14, 2026 Seed2.0 Pro model card 87 MMLU-Pro (Table 3) Capabilities
Feb 11, 2026 Step 3.5 Flash technical report 84.4 MMLU-Pro (Table 5). PaCoRe 84.8 Capabilities
Feb 4, 2026 ERNIE 5.0 technical report 83.8 MMLU-Pro (Table 2) Capabilities
Jan 30, 2026 Kimi K2.5 technical report 87.1 MMLU-Pro (Table 4) Capabilities
Jan 14, 2026 STEP3-VL-10B technical report 76 MMLU-Pro, SeRe (Tables 2 and 3). PaCoRe 77.09 Capabilities
Jan 11, 2026 Solar Open 100B technical report 80.4 MMLU-Pro (Table 6) Capabilities
Jan 6, 2026 MiMo-V2-Flash technical report 84.9 MMLU-Pro (Table 9) Capabilities
Jan 5, 2026 Falcon H1R 7B technical report 72.1 MMLU-Pro (Table 6) Capabilities
Jan 5, 2026 K-EXAONE technical report 83.8 MMLU-Pro (Table 3, Reasoning mode). Non-reasoning 81.0 (Table 4) Capabilities
Dec 23, 2025 Nemotron 3 Nano 30B-A3B technical report 78.3 MMLU-Pro (Table 3) Capabilities
Dec 23, 2025 Nemotron 3 Nano 30B-A3B Base technical report 65 MMLU-Pro, 5-shot CoT EM, base model Table 2 Capabilities
Dec 18, 2025 INTELLECT-3 technical report 81.9 MMLU-Pro, Avg@1 (Table 2, Appendix A.1) Capabilities
Dec 2, 2025 Nova 2 Omni Preview technical report 80.7 MMLU-Pro acc (Table 5) Capabilities
Dec 2, 2025 Nova 2 Lite technical report 80.9 MMLU-Pro acc (Table 1) Capabilities
Dec 2, 2025 Nova 2.0 Pro Preview technical report 81.6 MMLU-Pro acc (Table 1) Capabilities
Dec 1, 2025 DeepSeek-V3.2 technical report 85 MMLU-Pro (EM), DeepSeek-V3.2 Thinking (Table 2) Capabilities
Nov 28, 2025 LFM2-8B-A1B technical report 37.4 MMLU-Pro, LFM2-8B-A1B (Table 7) Capabilities
Nov 28, 2025 LFM2-1.2B technical report 19.7 MMLU-Pro, LFM2-1.2B (Table 6) Capabilities
Nov 26, 2025 Qwen3-VL-235B-A22B technical report 83.8 MMLU-Pro (Table 6) Capabilities
Nov 6, 2025 Nemotron Nano 12B v2 VL technical report 77.1 MMLU-Pro-1000, a 1,000-sample subset, reasoning on, SFT stage 4 (Table 6) Capabilities
Oct 31, 2025 LongCat-Flash-Omni technical report 82.7 MMLU-Pro (acc) (Table 13) Capabilities
Oct 21, 2025 Ring-1T technical report 80.5 MMLU-Pro (Table 1) Capabilities
Sep 30, 2025 Code World Model (CWM) technical report 60.2 MMLU-Pro, without reasoning (Table 13) Capabilities
Sep 29, 2025 DeepSeek-V3.2-Exp technical report 85 MMLU-Pro (EM), DeepSeek-V3.2-Exp (Table 1) Capabilities
Sep 23, 2025 LongCat-Flash-Thinking technical report 82.6 MMLU-Pro(acc), model-judged answer matching (Table 2) Capabilities
Sep 1, 2025 LongCat-Flash-Chat technical report 82.7 MMLU-Pro (acc) (Table 3) Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 81.3 MMLU-Pro (Table 2) Capabilities
Aug 25, 2025 Hermes 4 405B technical report 80.6 MMLU-Pro, reasoning mode (Table 3). Non-reasoning 58.3. v1 printed 80.5 Capabilities
Aug 21, 2025 Intern-S1 technical report 83.5 MMLU-Pro (Table 2) Capabilities
Aug 8, 2025 GLM-4.5 technical report 84.6 MMLU Pro (Table 4) Capabilities
Aug 8, 2025 GLM-4.5-Air technical report 81.4 MMLU Pro (Table 4) Capabilities
Jul 30, 2025 Falcon-H1-34B technical report 58.7 MMLU-Pro, logprobs 5-shot (Table 24) Capabilities
Jul 21, 2025 Kimi K2 technical report 81.1 MMLU-Pro (EM) (Table 3) Capabilities
Jul 15, 2025 EXAONE 4.0 32B technical report 81.8 MMLU-Pro (Table 3, Reasoning). Non-reasoning 77.6 (Table 4) Capabilities
Jun 29, 2025 ERNIE-4.5-300B-A47B technical report 78.4 MMLU-Pro (Table 5) Capabilities
Jun 16, 2025 MiniMax-M1-80k technical report 81.1 MMLU-Pro (Table 2) Capabilities
May 12, 2025 MiMo-7B-RL technical report 58.6 MMLU-Pro (EM) (Table 4) Capabilities
Apr 30, 2025 Phi-4-reasoning technical report 74.3 MMLUPro (Table 2) Capabilities
Apr 30, 2025 Phi-4-Reasoning-plus technical report 76 MMLUPro (Table 2) Capabilities
Apr 15, 2025 Gemini 2.0 Flash model card 77.6 MMLU-Pro (Table, p.3, column Gemini 2.0 Flash (GA)) Capabilities
Apr 10, 2025 Gemini 2.0 Flash-Lite model card 71.6 MMLU-Pro (Table, p.3, column Gemini 2.0 Flash-Lite (Public Preview)) Capabilities
Apr 10, 2025 Seed-Thinking-v1.5 technical report 87 MMLU-PRO (Table 2) Capabilities
Apr 5, 2025 Llama 4 Scout model card 74.3 MMLU Pro, 0-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) Capabilities
Apr 5, 2025 Llama 4 Maverick model card 80.5 MMLU Pro, 0-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) Capabilities
Apr 1, 2025 Cohere Command A technical report 69.6 MMLU-Pro, 5-shot CoT (Table 3) Capabilities
Mar 26, 2025 Qwen2.5-Omni-7B technical report 47 MMLU-Pro (Table 1) Capabilities
Mar 25, 2025 Gemma 3 4B technical report 43.6 MMLU-Pro, IT, zero-shot (Table 6) Capabilities
Mar 25, 2025 Gemma 3 27B technical report 67.5 MMLU-Pro, IT, zero-shot (Table 6) Capabilities
Mar 25, 2025 Gemma 3 1B technical report 14.7 MMLU-Pro, IT, zero-shot (Table 6) Capabilities
Mar 25, 2025 Gemma 3 12B technical report 60.6 MMLU-Pro, IT, zero-shot (Table 6) Capabilities
Mar 16, 2025 EXAONE Deep 32B technical report 74 MMLU-Pro, accuracy (Table 3) Capabilities
Mar 3, 2025 Phi-4 Mini technical report 52.8 MMLU-Pro (0-Shot, CoT) (Table 7) Capabilities
Feb 19, 2025 Qwen2.5-VL-72B technical report 71.2 MMLU-Pro (Table 4) Capabilities
Jan 26, 2025 Qwen2.5-14B-Instruct-1M technical report 63.3 MMLU-Pro (Table 6) Capabilities
Jan 20, 2025 DeepSeek-R1 technical report 84 MMLU-Pro (EM), DeepSeek-R1 (Table 8, same as final column of Table 3) Capabilities
Jan 14, 2025 MiniMax-Text-01 technical report 75.7 MMLU-Pro, 0-shot CoT (Table 8) Capabilities
Dec 31, 2024 OLMo 2 32B Instruct technical report 52.7 MMLU Pro, 0-shot CoT, unseen suite (Table 24) Capabilities
Dec 26, 2024 DeepSeek-V3 technical report 75.9 MMLU-Pro (EM), DeepSeek-V3 (Table 6) Capabilities
Dec 19, 2024 Qwen2.5-7B technical report 45 MMLU-Pro 45.0 (base model, Table 8. 56.3 is the instruction-tuned model) Capabilities
Dec 19, 2024 Qwen2.5-72B technical report 71.1 MMLU-Pro, Qwen2.5-72B-Instruct (Table 6) Capabilities
Dec 19, 2024 Qwen2.5-32B technical report 69 MMLU-Pro, Qwen2.5-32B-Instruct (Table 7) Capabilities
Dec 19, 2024 Qwen2.5-14B technical report 63.7 MMLU-Pro, Qwen2.5-14B-Instruct (Table 7) Capabilities
Dec 19, 2024 Qwen-Turbo technical report 64.5 MMLU-Pro, Qwen2.5-Turbo (Table 7) Capabilities
Dec 12, 2024 Phi-4 technical report 70.4 MMLUPro, internal framework (Table 1) Capabilities
Dec 6, 2024 Llama 3.3 70B Instruct model card 68.9 MMLU Pro (CoT), 5-shot, macro_avg/acc (Instruction tuned models table) Capabilities
Nov 23, 2024 Llama 3.1-8B technical report 48.3 MMLU-Pro (5-shot, CoT), post-trained (Table 2) Capabilities
Nov 23, 2024 Llama 3.1-70B technical report 66.4 MMLU-Pro (5-shot, CoT), post-trained (Table 2) Capabilities
Nov 23, 2024 Llama 3.1-405B technical report 73.3 MMLU-Pro (5-shot, CoT), post-trained (Table 2) Capabilities
Oct 22, 2024 Claude 3.5 Sonnet (October 2024) model card 78 MMLU Pro, 0-shot CoT (Table 7) Capabilities
Oct 22, 2024 Claude 3.5 Haiku model card 65 MMLU Pro, 0-shot CoT (Table 8) Capabilities
Aug 30, 2024 Phi-3.5-MoE technical report 54.3 MMLU-Pro (0-shot, CoT) (Table 3) Capabilities
Aug 30, 2024 Phi-3.5-mini technical report 47.5 MMLU-Pro (0-shot, CoT) (Table 3) Capabilities
Jul 23, 2024 Llama 3.1-8B model card 48.3 MMLU-Pro (CoT), 5-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) Capabilities
Jul 23, 2024 Llama 3.1-70B model card 66.4 MMLU-Pro (CoT), 5-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) Capabilities
Jul 23, 2024 Llama 3.1-405B model card 73.3 MMLU-Pro (CoT), 5-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) Capabilities
Jul 15, 2024 Qwen2-72B technical report 64.4 MMLU-Pro, Qwen2-72B-Instruct (Table 6) Capabilities

The lab's number comes out ahead in 8 of 13 pairs that differ

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 49 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap What differs
Qwen3.5-0.8B 42.3% MMLU-Pro (Thinking) 29.7% Leaderboard +13 pts Qwen reports 42.3 with thinking enabled. The board's 29.7 matches the card's nonthinking result.
Qwen3.5-2B 66.5% MMLU-Pro (Thinking) 55.3% Leaderboard +11 pts Qwen reports 66.5% with thinking enabled. The board lists the self-reported 55.3% result from the model card's non-thinking mode.
MiniMax-M2.5 85.2% MMLU-Pro, AA Index v4.0 protocol (Table 4) 80.1% Leaderboard +5.1 pts
Grok-3 mini 78.9% MMLU-pro [Grok 3 mini, non-reasoning table] 83% Leaderboard −4.1 pts
Llama 3.1-8B 48.3% MMLU-Pro (CoT), 5-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) 44.3% Leaderboard +4.0 pts
Llama 3.1-70B 66.4% MMLU-Pro (CoT), 5-shot, macro_avg/acc, instruction tuned (Instruction tuned models table) 62.8% Leaderboard +3.6 pts
Llama 3.3 70B Instruct 68.9% MMLU Pro (CoT), 5-shot, macro_avg/acc 65.9% Leaderboard +3.0 pts
Claude 3.5 Haiku 65% MMLU Pro, 0-shot CoT (Table 8) 62.1% Leaderboard +2.9 pts
DeepSeek-R1 May 2025 85% MMLU-Pro (EM) 84% Leaderboard +1.0 pts
Llama 4 Behemoth 82.2% MMLU Pro 82.8% Leaderboard −0.6 pts
Qwen2.5-72B 71.1% MMLU-Pro 71.6% Leaderboard −0.5 pts
Phi-3.5-mini 47.5% MMLU-Pro (0-shot, CoT) (Table 3) 47.9% Leaderboard −0.4 pts
Qwen2.5-32B 69% MMLU-Pro, Qwen2.5-32B-Instruct (Table 7) 69.2% Leaderboard −0.2 pts
DeepSeek-V3 Mar 2025 81.2% MMLU-Pro 81.3% Leaderboard Same
Kimi K2 Jul 2025 81.1% MMLU-Pro, EM 81% Leaderboard Same
Qwen3-235B-A22B-Thinking Jul 2025 84.4% MMLU-Pro 84.5% Leaderboard Same
Nemotron 3 Nano 30B-A3B Base 65% MMLU-Pro, 5-shot CoT EM, base model Table 2 65.1% Leaderboard Same
Mistral Small 3.1 66.8% MMLU Pro (5-shot CoT) 66.8% Leaderboard Same
Qwen2-72B 64.4% MMLU-Pro 64.4% Leaderboard Same
LongCat-Flash-Chat 82.7% MMLU-Pro(acc) 82.7% Leaderboard Same
Qwen2.5-14B 63.7% MMLU-Pro, Qwen2.5-14B-Instruct (Table 7) 63.7% Leaderboard Same
Llama 3.1-405B 73.3% MMLU-Pro (5-shot, CoT) 73.3% Leaderboard Same
Claude 3.5 Sonnet (October 2024) 78% MMLU Pro (0-shot CoT) 78% Leaderboard Same
Phi-4 70.4% MMLUPro 70.4% Leaderboard Same
InternLM3-8B-Instruct 57.6% MMLU-Pro(0-shot) 57.6% Leaderboard Same
MiniMax-Text-01 75.7% MMLU-Pro* 75.7% Leaderboard Same
Gemini 2.0 Pro 79.1% MMLU-Pro 79.1% Leaderboard Same
Gemini 2.0 Flash Feb 2025 77.6% MMLU-Pro 77.6% Leaderboard Same
Gemini 2.0 Flash-Lite 71.6% MMLU-Pro 71.6% Leaderboard Same
Grok 3 79.9% MMLU-pro [Grok 3 Beta, non-reasoning table] 79.9% Leaderboard Same
Phi-4 Mini 52.8% MMLU-Pro (0-shot, CoT) 52.8% Leaderboard Same
Hunyuan-TurboS 79% MMLU-pro 79% Leaderboard Same
Gemma 3 27B 67.5% MMLU (Pro) 0-shot (IT); PT MMLU (Pro COT) 5-shot = 52.2 67.5% Leaderboard Same
Hunyuan-T1 87.2% MMLU PRO 87.2% Leaderboard Same
Llama 4 Maverick 80.5% MMLU Pro, 0-shot, macro_avg/acc 80.5% Leaderboard Same
Llama 4 Scout 74.3% MMLU Pro, 0-shot, macro_avg/acc 74.3% Leaderboard Same
Seed-Thinking-v1.5 87% MMLU-PRO 87% Leaderboard Same
MiMo-7B-RL 58.6% MMLU-Pro (EM) 58.6% Leaderboard Same
Phi-4-Reasoning-plus 76% MMLUPro 76% Leaderboard Same
Phi-4-reasoning 74.3% MMLUPro 74.3% Leaderboard Same
MiniMax-M1-80k 81.1% MMLU-Pro 81.1% Leaderboard Same
ERNIE-4.5-300B-A47B 78.4% MMLU-Pro 78.4% Leaderboard Same
Intern-S1 83.5% MMLU-Pro 83.5% Leaderboard Same
GLM-4.5 84.6% MMLU Pro 84.6% Leaderboard Same
seed-oss-36b-instruct 82.7% MMLU-Pro 82.7% Leaderboard Same
DeepSeek-V3.1 84.8% MMLU-Pro (EM) 84.8% Leaderboard Same
MiniMax-M2 82% MMLU-Pro (AA) 82% Leaderboard Same
DeepSeek-V3.2 85% MMLU-Pro (EM) 85% Leaderboard Same
Nemotron 3 Nano 30B-A3B 78.3% MMLU-Pro 78.3% Leaderboard, bf16 Same
MiniMax-M2.1 88% MMLU-Pro 88% Leaderboard Same
Seed2.0 Pro 87% MMLU-Pro 87% Leaderboard Same
Qwen3.5 397B-A17B 87.8% MMLU-Pro 87.8% Leaderboard Same
Qwen3.5-122B-A10B 86.7% MMLU-Pro 86.7% Leaderboard Same
Qwen3.5-27B 86.1% MMLU-Pro 86.1% Leaderboard Same
Qwen3.5-35B-A3B 85.3% MMLU-Pro 85.3% Leaderboard Same
Qwen3.5-9B 82.5% MMLU-Pro 82.5% Leaderboard Same
Qwen3.5-4B 79.1% MMLU-Pro 79.1% Leaderboard Same
Qwen2.5-7B 45% MMLU-Pro 45.0 (base model, Table 8. 56.3 is the instruction-tuned model) 45% Leaderboard Same
Gemma 3 12B 60.6% MMLU-Pro, IT, zero-shot (Table 6) 60.6% Leaderboard Same
Gemma 3 1B 14.7% MMLU-Pro, IT, zero-shot (Table 6) 14.7% Leaderboard Same
Gemma 3 4B 43.6% MMLU-Pro, IT, zero-shot (Table 6) 43.6% Leaderboard Same
GLM-4.5-Air 81.4% MMLU Pro (Table 4) 81.4% Leaderboard Same

One logged change that moves scores

Changes by the maintainers, so a score from before a change may not compare with one from after it. Newest first, each with the maintainers' own source. The list may be incomplete. What we log.

  1. Questions corrected January 18th, 2026

    The maintainers removed inconsistent leading spaces from answer options in STEM subjects. This correction removed a formatting shortcut that could inflate earlier scores. Source

Versions

More in Knowledge

All 332 in Knowledge
  • MMLU In use 57-subject multiple-choice exam questions from elementary to professional level. Scored by accuracy. Seed1.8 92.3%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • HLE-Diamond In use Models answer 1,000 cleaned expert questions from Humanity's Last Exam, 500 on reasoning and 500 on knowledge, closed book. Graded on exact answers. Claude Fable 5.2 77.9%
  • HLE-Verified In use Verified and revised version of Humanity's Last Exam (1,811 items) with uncertain items removed. Gemini 3 Pro 48.2%

Data audit

All 9 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (240 entries), launch posts (137 scores) and system cards (103 scores).
  • Pass Models scored 320 models we track, plus 28 other entries on its leaderboard.
  • Pass Newest score The newest is from October 2026, for NVIDIA Nemotron 3.5 Transcribe 30B-A4B.
  • Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28).
  • Note Headroom Saturated. The best is 91.2%, past 85% since December 2024.
Leaderboard import 3 pass
  • Pass Importer We read the board on huggingface.co: 233 rows for models we track and 28 other entries at the last read.
  • Pass Last read November 19th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims 2 large gaps of 62 pairs, each explained in our notes.
  • Pass Known issues No published audit found problems.

How we check