Benchmark Almanac

MMLU

Seed1.8 92.3% leads since December 2025
Gopher (280B) 60% leads
Seed1.8 92.3% leads since December 2025
In use 1 known issue

The official leaderboard hasn’t scored a model released after Gopher (280B) (December 2021). Labs report their own numbers for 189 newer models, such as Seed1.8 at 92.3%.

57-subject multiple-choice exam questions from elementary to professional level. Scored by accuracy.

Saturated and contaminated. Reported only as a base-model metric. Google reports a medical subject aggregate called MMLU Med in the MedGemma model card.

Released
September 2020
Kind
Exam questions
Level
University
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Text
Interaction
Single turn
Items from
Exams and contests
Run by
Anyone
Strongest model tested
DeepSeek-V4-Pro #70 on the capability index
Last in a launch post
October 2026
Last model added
October 2026
Scores refreshed
November 19th, 2026

In the authors’ words

We propose a new test to measure a text model's multitask accuracy. The test covers 57 tasks including elementary mathematics, US history, computer science, law, and more. To attain high accuracy on this test, models must possess extensive world knowledge and problem solving ability. We find that while most recent models have near random-chance accuracy, the very largest GPT-3 model improves over random chance by almost 20 percentage points on average. However, on every one of the 57 tasks, the best models still need substantial improvements before they can reach expert-level accuracy. Models also have lopsided performance and frequently do not know when they are wrong. Worse, they still have near-random accuracy on some socially important subjects such as morality and law. By comprehensively evaluating the breadth and depth of a model's academic and professional understanding, our test can be used to analyze models across many tasks and to identify important shortcomings.

Abstract of Measuring Massive Multitask Language Understanding (arXiv:2009.03300, CC0)

Known issues

A published audit found a problem in MMLU. We summarize each finding in our words and link its source. The scores on this page don't include any fixes. See all audits

  1. Wrong labels Unclear tasks Fixed in a copy

    Gema et al. report numerous ground-truth errors in MMLU, with Virology the worst subject they analyzed. They re-annotated 5,700 questions across all 57 subjects as MMLU-Redux and estimate the error rate for all of MMLU.

    57% of the analyzed Virology questions contain errors, and the authors estimate that 6.49% of all MMLU questions contain errors.

    For example, we find that 57% of the analysed questions in the Virology subset contain errors.

    Are We Done with MMLU? by Gema et al., June 6th, 2024

132
Models scored in lab cards
100
Launch posts list it, from 30 labs
91
System cards report it, from 27 labs

Seed1.8 leads with 92.3%

Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.

Gopher (280B) leads with 60% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.

Seed1.8 leads with 92.3%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.

132 models. Scroll sideways for the rest.

  1. 92.3 Seed1.8
  2. 91.8 o1
  3. 91.7 Gemini 1.5 Pro
  4. 91.3 Cogito v2 Preview 671B MoE
  5. 91.0 Cogito v1 Preview 70B
  6. 90.8 o1-preview
  7. 90.8 DeepSeek-R1
  8. 90.6 Qwen3-VL-235B-A22B
  9. 90.6 Sarvam 105B
  10. 90.5 Claude 3.5 Sonnet (October 2024)
  11. 90.4 Claude 3.5 Sonnet
  12. 90.3 LongCat-Flash-Omni
  13. 90.1 DeepSeek-V4-Pro
  14. 90.0 Gemini 1.0 Ultra
  15. 90.0 gpt-oss-120b
  16. 90.0 GLM-4.5
  17. 89.7 LongCat-Flash-Chat
  18. 89.5 Hunyuan-TurboS
  19. 89.5 Kimi K2
  20. 89.1 InternVL3.5-241B-A28B
  21. 89.1 Nemotron 3 Ultra 550B-A55B Base
  22. 88.7 GPT-4o
  23. 88.6 Llama 3.1-405B
  24. 88.5 DeepSeek-V3
  25. 88.5 MiniMax-Text-01
  26. 88.3 GLM-5
  27. 88.2 Claude 3 Opus
  28. 88.2 Solar Open 100B
  29. 88.2 Solar Pro 2
  30. 87.7 Qwen3-Coder-Next
  31. 87.7 Hermes 4.3 36B
  32. 87.5 Grok-2
  33. 87.5 GPT-4.1 mini
  34. 87.4 Kimi k1.5
  35. 87.4 Amazon Nova Premier
  36. 87.4 GLM-4.5-Air
  37. 87.4 seed-oss-36b-instruct
  38. 87.2 Trinity Large Preview
  39. 87.2 Hermes 4 405B
  40. 87.1 sonar
  41. 87.0 Tulu 3 405B
  42. 87.0 Sarvam-M
  43. 86.9 o3-mini
  44. 86.7 GPT-4 Turbo
  45. 86.5 ERNIE-4.5-300B-A47B
  46. 86.4 GPT-4
  47. 86.4 Olmo 3.1 32B Think
  48. 86.1 Qwen2.5-72B
  49. 86.0 Llama 3.1-70B
  50. 86.0 Llama 3.3 70B Instruct
  51. 85.5 Cohere Command A
  52. 85.5 Llama 4 Maverick
  53. 85.4 Olmo 3 32B Think
  54. 85.3 gpt-oss-20b
  55. 85.1 Sarvam 30B
  56. 85.0 Trinity Mini
  57. 84.8 Phi-4
  58. 84.1 Nemotron 3.5 Super 120B-A12B Base
  59. 84.0 Falcon-H1-34B
  60. 84.0 Mistral Large 2
  61. 84.0 NVIDIA Nemotron 3.5 Transcribe 30B-A4B
  62. 83.0 EXAONE Deep 32B
  63. 82.3 Qwen2-72B
  64. 82.0 Llama 3-70B
  65. 82.0 GPT-4o mini
  66. 82.0 Kimi-VL-A3B-Thinking-2506
  67. 81.9 Gemini 1.5 Pro
  68. 81.5 GLM-4
  69. 81.5 Claude 3 Sonnet
  70. 81.3 Grok-1.5
  71. 81.2 Mistral Large
  72. 80.9 Claude 3.5 Haiku
  73. 80.7 Mistral Small 3
  74. 80.6 Mistral Small 3.1
  75. 80.5 Mistral Small 3.2
  76. 80.2 Apple Foundation Model Server (2025)
  77. 80.2 Granite 4.1 30B
  78. 80.1 GPT-4.1 nano
  79. 79.6 Llama 4 Scout
  80. 78.9 Phi-3.5-MoE
  81. 78.6 Nemotron 3.5 Lightning 30B-A3B Base
  82. 78.6 Gemma 3 27B
  83. 78.6 Nemotron 3 Nano 30B-A3B Base
  84. 78.5 Claude 2
  85. 78.4 Granite 4.0 H Small
  86. 78.2 phi-3-medium 14B
  87. 77.8 DeepSeek-V2 (MoE-236B, May 2024)
  88. 77.7 Code World Model (CWM)
  89. 77.5 Qwen1.5-72B
  90. 77.3 OLMo 2 32B Instruct
  91. 76.7 Claude 3 Haiku
  92. 76.6 InternLM3-8B-Instruct
  93. 76.2 Gemma 2 27B
  94. 75.7 phi-3-small 7.4B
  95. 73.0 Grok-1
  96. 73.0 Llama 3.1-8B
  97. 72.5 Mixtral 8x7B
  98. 72.3 Gemma 2 9B
  99. 71.9 Gemma 3 12B
  100. 71.1 DeepSeek LLM 67B
  101. 70.0 Moonlight-16B-A3B
  102. 69.6 Apertus 70B
  103. 69.0 Phi-3.5-mini
  104. 68.9 Llama 2-70B
  105. 68.8 phi-3-mini 3.8B
  106. 68.4 Llama 3-8B
  107. 67.3 Phi-4 Mini
  108. 66.8 Granite 3.2 8B Instruct
  109. 66.5 Qwen-14B
  110. 65.9 AFM-4.5B
  111. 65.5 Granite 3.3 8B Instruct
  112. 64.9 Gemma 3n E4B
  113. 64.8 LFM2-8B-A1B
  114. 64.3 Gemma 7B
  115. 63.3 Falcon-Arabic-7B
  116. 62.6 Llama 2-34B
  117. 61.4 ChatGLM3-6B
  118. 60.1 Mistral 7B v0.1
  119. 59.4 granite-4.0-1b
  120. 58.1 Gemma 3 4B
  121. 57.0 Qwen-7B
  122. 56.7 Phi-2
  123. 55.2 LFM2-1.2B
  124. 54.8 Llama 2-13B
  125. 53.2 BitNet b1.58 2B4T
  126. 51.0 LFM2-VL-1.6B
  127. 50.4 North Micro Vision Instruct
  128. 45.3 Llama 2-7B
  129. 43.9 Qwen-1_8B
  130. 42.3 Gemma 2B
  131. 40.0 OLMo 2 1B Instruct
  132. 38.8 Gemma 3 1B
  1. 1 Seed1.8 92.3%
  2. 2 o1 91.8%
  3. 3 Gemini 1.5 Pro May 2024 91.7%
  4. 4 Cogito v2 Preview 671B MoE 91.3%
  5. 5 Cogito v1 Preview 70B 91%
  6. 6 o1-preview 90.8%
  7. 7 DeepSeek-R1 90.8%
  8. 8 Qwen3-VL-235B-A22B 90.6%
  9. 9 Sarvam 105B 90.6%
  10. 10 Claude 3.5 Sonnet (October 2024) 90.5%
  11. 11 Claude 3.5 Sonnet 90.4%
  12. 12 LongCat-Flash-Omni 90.3%
  13. 13 DeepSeek-V4-Pro 90.1%
  14. 14 Gemini 1.0 Ultra 90%
  15. 15 gpt-oss-120b 90%
  16. 16 GLM-4.5 90%
  17. 17 LongCat-Flash-Chat 89.7%
  18. 18 Hunyuan-TurboS 89.5%
  19. 19 Kimi K2 Jul 2025 89.5%
  20. 20 InternVL3.5-241B-A28B 89.1%
  21. 21 Nemotron 3 Ultra 550B-A55B Base 89.1%
  22. 22 GPT-4o May 2024 88.7%
  23. 23 Llama 3.1-405B 88.6%
  24. 24 DeepSeek-V3 88.5%
  25. 25 MiniMax-Text-01 88.5%
  26. 26 GLM-5 88.3%
  27. 27 Claude 3 Opus 88.2%
  28. 28 Solar Open 100B 88.2%
  29. 29 Solar Pro 2 88.2%
  30. 30 Qwen3-Coder-Next 87.7%
  31. 31 Hermes 4.3 36B 87.7%
  32. 32 Grok-2 Aug 2024 87.5%
  33. 33 GPT-4.1 mini 87.5%
  34. 34 Kimi k1.5 87.4%
  35. 35 Amazon Nova Premier 87.4%
  36. 36 GLM-4.5-Air 87.4%
  37. 37 seed-oss-36b-instruct 87.4%
  38. 38 Trinity Large Preview 87.2%
  39. 39 Hermes 4 405B 87.2%
  40. 40 sonar 87.1%
  41. 41 Tulu 3 405B 87%
  42. 42 Sarvam-M 87%
  43. 43 o3-mini 86.9%
  44. 44 GPT-4 Turbo Apr 2024 86.7%
  45. 45 ERNIE-4.5-300B-A47B 86.5%
  46. 46 GPT-4 Mar 2023 86.4%
  47. 47 Olmo 3.1 32B Think 86.4%
  48. 48 Qwen2.5-72B 86.1%
  49. 49 Llama 3.1-70B 86%
  50. 50 Llama 3.3 70B Instruct 86%
  51. 51 Cohere Command A 85.5%
  52. 52 Llama 4 Maverick 85.5%
  53. 53 Olmo 3 32B Think 85.4%
  54. 54 gpt-oss-20b 85.3%
  55. 55 Sarvam 30B 85.1%
  56. 56 Trinity Mini 85%
  57. 57 Phi-4 84.8%
  58. 58 Nemotron 3.5 Super 120B-A12B Base 84.1%
  59. 59 Falcon-H1-34B 84%
  60. 60 Mistral Large 2 Jul 2024 84%
  61. 61 NVIDIA Nemotron 3.5 Transcribe 30B-A4B 84%
  62. 62 EXAONE Deep 32B 83%
  63. 63 Qwen2-72B 82.3%
  64. 64 Llama 3-70B 82%
  65. 65 GPT-4o mini 82%
  66. 66 Kimi-VL-A3B-Thinking-2506 82%
  67. 67 Gemini 1.5 Pro Feb 2024 81.9%
  68. 68 GLM-4 81.5%
  69. 69 Claude 3 Sonnet 81.5%
  70. 70 Grok-1.5 81.3%
  71. 71 Mistral Large 81.2%
  72. 72 Claude 3.5 Haiku 80.9%
  73. 73 Mistral Small 3 80.7%
  74. 74 Mistral Small 3.1 80.6%
  75. 75 Mistral Small 3.2 80.5%
  76. 76 Apple Foundation Model Server (2025) 80.2%
  77. 77 Granite 4.1 30B 80.2%
  78. 78 GPT-4.1 nano 80.1%
  79. 79 Llama 4 Scout 79.6%
  80. 80 Phi-3.5-MoE 78.9%
  81. 81 Nemotron 3.5 Lightning 30B-A3B Base 78.6%
  82. 82 Gemma 3 27B 78.6%
  83. 83 Nemotron 3 Nano 30B-A3B Base 78.6%
  84. 84 Claude 2 78.5%
  85. 85 Granite 4.0 H Small 78.4%
  86. 86 phi-3-medium 14B 78.2%
  87. 87 DeepSeek-V2 (MoE-236B, May 2024) 77.8%
  88. 88 Code World Model (CWM) 77.7%
  89. 89 Qwen1.5-72B 77.5%
  90. 90 OLMo 2 32B Instruct 77.3%
  91. 91 Claude 3 Haiku 76.7%
  92. 92 InternLM3-8B-Instruct 76.6%
  93. 93 Gemma 2 27B 76.2%
  94. 94 phi-3-small 7.4B 75.7%
  95. 95 Grok-1 73%
  96. 96 Llama 3.1-8B 73%
  97. 97 Mixtral 8x7B 72.5%
  98. 98 Gemma 2 9B 72.3%
  99. 99 Gemma 3 12B 71.9%
  100. 100 DeepSeek LLM 67B 71.1%
  101. 101 Moonlight-16B-A3B 70%
  102. 102 Apertus 70B 69.6%
  103. 103 Phi-3.5-mini 69%
  104. 104 Llama 2-70B 68.9%
  105. 105 phi-3-mini 3.8B 68.8%
  106. 106 Llama 3-8B 68.4%
  107. 107 Phi-4 Mini 67.3%
  108. 108 Granite 3.2 8B Instruct 66.8%
  109. 109 Qwen-14B 66.5%
  110. 110 AFM-4.5B 65.9%
  111. 111 Granite 3.3 8B Instruct 65.5%
  112. 112 Gemma 3n E4B 64.9%
  113. 113 LFM2-8B-A1B 64.8%
  114. 114 Gemma 7B 64.3%
  115. 115 Falcon-Arabic-7B 63.3%
  116. 116 Llama 2-34B 62.6%
  117. 117 ChatGLM3-6B 61.4%
  118. 118 Mistral 7B v0.1 60.1%
  119. 119 granite-4.0-1b 59.4%
  120. 120 Gemma 3 4B 58.1%
  121. 121 Qwen-7B 57%
  122. 122 Phi-2 56.7%
  123. 123 LFM2-1.2B 55.2%
  124. 124 Llama 2-13B 54.8%
  125. 125 BitNet b1.58 2B4T 53.2%
  126. 126 LFM2-VL-1.6B 51%
  127. 127 North Micro Vision Instruct 50.4%
  128. 128 Llama 2-7B 45.3%
  129. 129 Qwen-1_8B 43.9%
  130. 130 Gemma 2B 42.3%
  131. 131 OLMo 2 1B Instruct 40%
  132. 132 Gemma 3 1B 38.8%

Source: Labs’ model cards

Source: Official MMLU leaderboard

192 models. Scroll sideways for the rest.

  1. 92.3 Seed1.8
  2. 91.8 o1
  3. 91.7 Gemini 1.5 Pro
  4. 91.3 Cogito v2 Preview 671B MoE
  5. 91.0 Cogito v1 Preview 70B
  6. 90.8 o1-preview
  7. 90.8 DeepSeek-R1
  8. 90.6 Qwen3-VL-235B-A22B
  9. 90.6 Sarvam 105B
  10. 90.5 Claude 3.5 Sonnet (October 2024)
  11. 90.4 Claude 3.5 Sonnet
  12. 90.3 LongCat-Flash-Omni
  13. 90.1 DeepSeek-V4-Pro
  14. 90.0 Gemini 1.0 Ultra
  15. 90.0 gpt-oss-120b
  16. 90.0 GLM-4.5
  17. 89.7 LongCat-Flash-Chat
  18. 89.5 Hunyuan-TurboS
  19. 89.5 Kimi K2
  20. 89.1 InternVL3.5-241B-A28B
  21. 89.1 Nemotron 3 Ultra 550B-A55B Base
  22. 88.7 GPT-4o
  23. 88.6 Llama 3.1-405B
  24. 88.5 DeepSeek-V3
  25. 88.5 MiniMax-Text-01
  26. 88.3 GLM-5
  27. 88.2 Claude 3 Opus
  28. 88.2 Solar Open 100B
  29. 88.2 Solar Pro 2
  30. 88.1 GPT-4o
  31. 87.7 Qwen3-Coder-Next
  32. 87.7 Hermes 4.3 36B
  33. 87.5 Grok-2
  34. 87.5 GPT-4.1 mini
  35. 87.4 Kimi k1.5
  36. 87.4 Amazon Nova Premier
  37. 87.4 GLM-4.5-Air
  38. 87.4 seed-oss-36b-instruct
  39. 87.2 Trinity Large Preview
  40. 87.2 Hermes 4 405B
  41. 87.1 sonar
  42. 87.0 Tulu 3 405B
  43. 87.0 Sarvam-M
  44. 86.9 o3-mini
  45. 86.7 GPT-4 Turbo
  46. 86.5 ERNIE-4.5-300B-A47B
  47. 86.4 GPT-4
  48. 86.4 Olmo 3.1 32B Think
  49. 86.3 Llama 3.3 70B
  50. 86.1 Qwen2.5-72B
  51. 86.0 Llama 3.1-70B
  52. 86.0 Llama 3.3 70B Instruct
  53. 85.5 Cohere Command A
  54. 85.5 Llama 4 Maverick
  55. 85.4 Olmo 3 32B Think
  56. 85.3 gpt-oss-20b
  57. 85.1 Sarvam 30B
  58. 85.0 Trinity Mini
  59. 84.8 Phi-4
  60. 84.1 Nemotron 3.5 Super 120B-A12B Base
  61. 84.0 Falcon-H1-34B
  62. 84.0 Mistral Large 2
  63. 84.0 NVIDIA Nemotron 3.5 Transcribe 30B-A4B
  64. 83.0 EXAONE Deep 32B
  65. 82.3 Qwen2-72B
  66. 82.0 Llama 3-70B
  67. 82.0 GPT-4o mini
  68. 82.0 Kimi-VL-A3B-Thinking-2506
  69. 81.9 Gemini 1.5 Pro
  70. 81.5 GLM-4
  71. 81.5 Claude 3 Sonnet
  72. 81.3 Grok-1.5
  73. 81.2 Mistral Large
  74. 80.9 Claude 3.5 Haiku
  75. 80.7 Mistral Small 3
  76. 80.6 Mistral Small 3.1
  77. 80.5 Mistral Small 3.2
  78. 80.2 Apple Foundation Model Server (2025)
  79. 80.2 Granite 4.1 30B
  80. 80.1 GPT-4.1 nano
  81. 79.9 Qwen2.5-14B
  82. 79.6 Llama 4 Scout
  83. 79.1 Qwen2.5-Coder-32B
  84. 78.9 Phi-3.5-MoE
  85. 78.6 Nemotron 3.5 Lightning 30B-A3B Base
  86. 78.6 Gemma 3 27B
  87. 78.6 Nemotron 3 Nano 30B-A3B Base
  88. 78.5 Claude 2
  89. 78.4 Granite 4.0 H Small
  90. 78.4 DeepSeek-V2 (MoE-236B, May 2024)
  91. 78.2 phi-3-medium 14B
  92. 77.7 Code World Model (CWM)
  93. 77.5 Qwen1.5-72B
  94. 77.3 OLMo 2 32B Instruct
  95. 77.0 Claude 1.3
  96. 76.7 Claude 3 Haiku
  97. 76.6 InternLM3-8B-Instruct
  98. 76.3 Yi-34B
  99. 76.2 Gemma 2 27B
  100. 75.7 phi-3-small 7.4B
  101. 75.2 Qwen2.5-Coder-14B
  102. 73.4 Claude Instant
  103. 73.0 Grok-1
  104. 73.0 Llama 3.1-8B
  105. 72.7 Inflection-1
  106. 72.5 Mixtral 8x7B
  107. 72.3 Gemma 2 9B
  108. 71.9 Gemma 3 12B
  109. 71.4 GPT-3.5 Turbo
  110. 71.1 DeepSeek LLM 67B
  111. 70.6 Falcon-180B
  112. 70.0 davinci-002
  113. 70.0 Moonlight-16B-A3B
  114. 69.9 Llama 2-70B
  115. 69.6 Apertus 70B
  116. 69.3 PaLM (540B)
  117. 69.0 Phi-3.5-mini
  118. 68.8 phi-3-mini 3.8B
  119. 68.6 Stable Beluga 2
  120. 68.4 Yi-9B
  121. 68.4 Llama 3-8B
  122. 68.0 Qwen2.5-Coder (7B)
  123. 67.5 Chinchilla
  124. 67.3 Phi-4 Mini
  125. 66.8 Granite 3.2 8B Instruct
  126. 66.5 Qwen-14B
  127. 65.9 AFM-4.5B
  128. 65.5 Granite 3.3 8B Instruct
  129. 64.9 Gemma 3n E4B
  130. 64.8 LFM2-8B-A1B
  131. 64.3 Gemma 7B
  132. 64.1 StarCoder 2 15B
  133. 63.4 LLaMA-65B
  134. 63.3 Falcon-Arabic-7B
  135. 63.2 Yi 6B
  136. 62.6 Llama 2-34B
  137. 62.5 Mistral 7B v0.1
  138. 62.5 Mistral 7B v0.2
  139. 61.4 ChatGLM3-6B
  140. 60.5 DeepSeek-Coder-V2-Lite-Base
  141. 60.0 Gopher (280B)
  142. 59.4 granite-4.0-1b
  143. 59.2 Baichuan2-13B
  144. 58.7 LLaMA-33B
  145. 58.7 Nemotron-4 15B
  146. 58.4 Falcon 2 11B
  147. 58.1 Gemma 3 4B
  148. 57.4 internlm-chat-20b
  149. 57.0 Qwen-7B
  150. 56.9 Falcon-40B
  151. 56.7 Phi-2
  152. 55.6 Llama 2-13B
  153. 55.2 LFM2-1.2B
  154. 55.1 Baichuan2-13B-Chat
  155. 54.2 Baichuan 2-7B
  156. 53.7 PaLM 62B
  157. 53.6 Qwen2.5-Coder (1.5B)
  158. 53.2 BitNet b1.58 2B4T
  159. 51.6 Baichuan 1-13B
  160. 51.0 internlm-7b
  161. 51.0 LFM2-VL-1.6B
  162. 50.4 North Micro Vision Instruct
  163. 49.9 INTELLECT-1
  164. 47.9 MPT-30B
  165. 47.9 chatglm2-6b
  166. 47.7 LLaMA-13B
  167. 45.7 Llama 2-7B
  168. 43.9 InstructGPT 175B
  169. 43.9 Qwen-1_8B
  170. 43.2 GPT-3 6.7B
  171. 42.3 Baichuan1-7B
  172. 42.3 Gemma 2B
  173. 42.0 Qwen2.5-Coder-0.5B
  174. 40.5 CodeQwen1.5-7B
  175. 40.0 OLMo 2 1B Instruct
  176. 39.4 DeepSeek Coder 33B
  177. 38.8 StarCoder 2 7B
  178. 38.8 Gemma 3 1B
  179. 37.6 Phi-1.5
  180. 36.6 StarCoder 2 3B
  181. 36.4 DeepSeek Coder 6.7B
  182. 36.3 XGen-7B
  183. 35.6 LLaMA-7B
  184. 35.0 Falcon-7B
  185. 30.8 MPT-7B
  186. 29.9 open_llama_7b
  187. 26.3 RedPajama-INCITE-7B-Base
  188. 26.2 Cerebras-GPT-13B
  189. 26.2 Dolly 2.0-12b
  190. 25.8 DeepSeek Coder 1.3B
  191. 25.7 GPT-J-6B
  192. 25.1 OPT-1.3B
  1. 1 Seed1.8 92.3%
  2. 2 o1 91.8%
  3. 3 Gemini 1.5 Pro May 2024 91.7%
  4. 4 Cogito v2 Preview 671B MoE 91.3%
  5. 5 Cogito v1 Preview 70B 91%
  6. 6 o1-preview 90.8%
  7. 7 DeepSeek-R1 90.8%
  8. 8 Qwen3-VL-235B-A22B 90.6%
  9. 9 Sarvam 105B 90.6%
  10. 10 Claude 3.5 Sonnet (October 2024) 90.5%
  11. 11 Claude 3.5 Sonnet 90.4%
  12. 12 LongCat-Flash-Omni 90.3%
  13. 13 DeepSeek-V4-Pro 90.1%
  14. 14 Gemini 1.0 Ultra 90%
  15. 15 gpt-oss-120b 90%
  16. 16 GLM-4.5 90%
  17. 17 LongCat-Flash-Chat 89.7%
  18. 18 Hunyuan-TurboS 89.5%
  19. 19 Kimi K2 Jul 2025 89.5%
  20. 20 InternVL3.5-241B-A28B 89.1%
  21. 21 Nemotron 3 Ultra 550B-A55B Base 89.1%
  22. 22 GPT-4o May 2024 88.7%
  23. 23 Llama 3.1-405B 88.6%
  24. 24 DeepSeek-V3 88.5%
  25. 25 MiniMax-Text-01 88.5%
  26. 26 GLM-5 88.3%
  27. 27 Claude 3 Opus 88.2%
  28. 28 Solar Open 100B 88.2%
  29. 29 Solar Pro 2 88.2%
  30. 30 GPT-4o Nov 2024 88.1%
  31. 31 Qwen3-Coder-Next 87.7%
  32. 32 Hermes 4.3 36B 87.7%
  33. 33 Grok-2 Aug 2024 87.5%
  34. 34 GPT-4.1 mini 87.5%
  35. 35 Kimi k1.5 87.4%
  36. 36 Amazon Nova Premier 87.4%
  37. 37 GLM-4.5-Air 87.4%
  38. 38 seed-oss-36b-instruct 87.4%
  39. 39 Trinity Large Preview 87.2%
  40. 40 Hermes 4 405B 87.2%
  41. 41 sonar 87.1%
  42. 42 Tulu 3 405B 87%
  43. 43 Sarvam-M 87%
  44. 44 o3-mini 86.9%
  45. 45 GPT-4 Turbo Apr 2024 86.7%
  46. 46 ERNIE-4.5-300B-A47B 86.5%
  47. 47 GPT-4 Mar 2023 86.4%
  48. 48 Olmo 3.1 32B Think 86.4%
  49. 49 Llama 3.3 70B 86.3%
  50. 50 Qwen2.5-72B 86.1%
  51. 51 Llama 3.1-70B 86%
  52. 52 Llama 3.3 70B Instruct 86%
  53. 53 Cohere Command A 85.5%
  54. 54 Llama 4 Maverick 85.5%
  55. 55 Olmo 3 32B Think 85.4%
  56. 56 gpt-oss-20b 85.3%
  57. 57 Sarvam 30B 85.1%
  58. 58 Trinity Mini 85%
  59. 59 Phi-4 84.8%
  60. 60 Nemotron 3.5 Super 120B-A12B Base 84.1%
  61. 61 Falcon-H1-34B 84%
  62. 62 Mistral Large 2 Jul 2024 84%
  63. 63 NVIDIA Nemotron 3.5 Transcribe 30B-A4B 84%
  64. 64 EXAONE Deep 32B 83%
  65. 65 Qwen2-72B 82.3%
  66. 66 Llama 3-70B 82%
  67. 67 GPT-4o mini 82%
  68. 68 Kimi-VL-A3B-Thinking-2506 82%
  69. 69 Gemini 1.5 Pro Feb 2024 81.9%
  70. 70 GLM-4 81.5%
  71. 71 Claude 3 Sonnet 81.5%
  72. 72 Grok-1.5 81.3%
  73. 73 Mistral Large 81.2%
  74. 74 Claude 3.5 Haiku 80.9%
  75. 75 Mistral Small 3 80.7%
  76. 76 Mistral Small 3.1 80.6%
  77. 77 Mistral Small 3.2 80.5%
  78. 78 Apple Foundation Model Server (2025) 80.2%
  79. 79 Granite 4.1 30B 80.2%
  80. 80 GPT-4.1 nano 80.1%
  81. 81 Qwen2.5-14B 79.9%
  82. 82 Llama 4 Scout 79.6%
  83. 83 Qwen2.5-Coder-32B 79.1%
  84. 84 Phi-3.5-MoE 78.9%
  85. 85 Nemotron 3.5 Lightning 30B-A3B Base 78.6%
  86. 86 Gemma 3 27B 78.6%
  87. 87 Nemotron 3 Nano 30B-A3B Base 78.6%
  88. 88 Claude 2 78.5%
  89. 89 Granite 4.0 H Small 78.4%
  90. 90 DeepSeek-V2 (MoE-236B, May 2024) 78.4%
  91. 91 phi-3-medium 14B 78.2%
  92. 92 Code World Model (CWM) 77.7%
  93. 93 Qwen1.5-72B 77.5%
  94. 94 OLMo 2 32B Instruct 77.3%
  95. 95 Claude 1.3 77%
  96. 96 Claude 3 Haiku 76.7%
  97. 97 InternLM3-8B-Instruct 76.6%
  98. 98 Yi-34B 76.3%
  99. 99 Gemma 2 27B 76.2%
  100. 100 phi-3-small 7.4B 75.7%
  101. 101 Qwen2.5-Coder-14B 75.2%
  102. 102 Claude Instant 73.4%
  103. 103 Grok-1 73%
  104. 104 Llama 3.1-8B 73%
  105. 105 Inflection-1 72.7%
  106. 106 Mixtral 8x7B 72.5%
  107. 107 Gemma 2 9B 72.3%
  108. 108 Gemma 3 12B 71.9%
  109. 109 GPT-3.5 Turbo Nov 2023 71.4%
  110. 110 DeepSeek LLM 67B 71.1%
  111. 111 Falcon-180B 70.6%
  112. 112 davinci-002 70%
  113. 113 Moonlight-16B-A3B 70%
  114. 114 Llama 2-70B 69.9%
  115. 115 Apertus 70B 69.6%
  116. 116 PaLM (540B) 69.3%
  117. 117 Phi-3.5-mini 69%
  118. 118 phi-3-mini 3.8B 68.8%
  119. 119 Stable Beluga 2 68.6%
  120. 120 Yi-9B 68.4%
  121. 121 Llama 3-8B 68.4%
  122. 122 Qwen2.5-Coder (7B) 68%
  123. 123 Chinchilla 67.5%
  124. 124 Phi-4 Mini 67.3%
  125. 125 Granite 3.2 8B Instruct 66.8%
  126. 126 Qwen-14B 66.5%
  127. 127 AFM-4.5B 65.9%
  128. 128 Granite 3.3 8B Instruct 65.5%
  129. 129 Gemma 3n E4B 64.9%
  130. 130 LFM2-8B-A1B 64.8%
  131. 131 Gemma 7B 64.3%
  132. 132 StarCoder 2 15B 64.1%
  133. 133 LLaMA-65B 63.4%
  134. 134 Falcon-Arabic-7B 63.3%
  135. 135 Yi 6B 63.2%
  136. 136 Llama 2-34B 62.6%
  137. 137 Mistral 7B v0.1 62.5%
  138. 138 Mistral 7B v0.2 62.5%
  139. 139 ChatGLM3-6B 61.4%
  140. 140 DeepSeek-Coder-V2-Lite-Base 60.5%
  141. 141 Gopher (280B) few-shot 60%
  142. 142 granite-4.0-1b 59.4%
  143. 143 Baichuan2-13B 59.2%
  144. 144 LLaMA-33B 58.7%
  145. 145 Nemotron-4 15B 58.7%
  146. 146 Falcon 2 11B 58.4%
  147. 147 Gemma 3 4B 58.1%
  148. 148 internlm-chat-20b 57.4%
  149. 149 Qwen-7B 57%
  150. 150 Falcon-40B 56.9%
  151. 151 Phi-2 56.7%
  152. 152 Llama 2-13B 55.6%
  153. 153 LFM2-1.2B 55.2%
  154. 154 Baichuan2-13B-Chat 55.1%
  155. 155 Baichuan 2-7B 54.2%
  156. 156 PaLM 62B 53.7%
  157. 157 Qwen2.5-Coder (1.5B) 53.6%
  158. 158 BitNet b1.58 2B4T 53.2%
  159. 159 Baichuan 1-13B 51.6%
  160. 160 internlm-7b 51%
  161. 161 LFM2-VL-1.6B 51%
  162. 162 North Micro Vision Instruct 50.4%
  163. 163 INTELLECT-1 49.9%
  164. 164 MPT-30B 47.9%
  165. 165 chatglm2-6b 47.9%
  166. 166 LLaMA-13B 47.7%
  167. 167 Llama 2-7B 45.7%
  168. 168 InstructGPT 175B 43.9%
  169. 169 Qwen-1_8B 43.9%
  170. 170 GPT-3 6.7B fine-tuned 43.2%
  171. 171 Baichuan1-7B 42.3%
  172. 172 Gemma 2B 42.3%
  173. 173 Qwen2.5-Coder-0.5B 42%
  174. 174 CodeQwen1.5-7B 40.5%
  175. 175 OLMo 2 1B Instruct 40%
  176. 176 DeepSeek Coder 33B 39.4%
  177. 177 StarCoder 2 7B 38.8%
  178. 178 Gemma 3 1B 38.8%
  179. 179 Phi-1.5 37.6%
  180. 180 StarCoder 2 3B 36.6%
  181. 181 DeepSeek Coder 6.7B 36.4%
  182. 182 XGen-7B 36.3%
  183. 183 LLaMA-7B 35.6%
  184. 184 Falcon-7B 35%
  185. 185 MPT-7B 30.8%
  186. 186 open_llama_7b 29.9%
  187. 187 RedPajama-INCITE-7B-Base 26.3%
  188. 188 Cerebras-GPT-13B 26.2%
  189. 189 Dolly 2.0-12b 26.2%
  190. 190 DeepSeek Coder 1.3B 25.8%
  191. 191 GPT-J-6B 25.7%
  192. 192 OPT-1.3B 25.1%

Sources: Labs’ model cards, another board, via Epoch AI (CC BY 4.0) and official MMLU leaderboard

#ModelScoreSettingDateSource
1 Seed1.8 92.3% Dec 2025 Lab model card
2 o1 91.8% Dec 2024 Lab model card
3 Gemini 1.5 Pro May 2024 91.7% May 2024 Lab model card
4 Cogito v2 Preview 671B MoE 91.3% Jul 2025 Lab model card
5 Cogito v1 Preview 70B 91% Apr 2025 Lab model card
6 o1-preview 90.8% Sep 2024 Lab model card
7 DeepSeek-R1 90.8% Jan 2025 Lab model card
8 Qwen3-VL-235B-A22B 90.6% Sep 2025 Lab model card
9 Sarvam 105B 90.6% Mar 2026 Lab model card
10 Claude 3.5 Sonnet (October 2024) 90.5% Oct 2024 Lab model card
11 Claude 3.5 Sonnet 90.4% Jun 2024 Lab model card
12 LongCat-Flash-Omni 90.3% Oct 2025 Lab model card
13 DeepSeek-V4-Pro 90.1% Apr 2026 Lab model card
14 Gemini 1.0 Ultra 90% Dec 2023 Lab model card
15 gpt-oss-120b 90% Aug 2025 Lab model card
16 GLM-4.5 90% Aug 2025 Lab model card
17 LongCat-Flash-Chat 89.7% Aug 2025 Lab model card
18 Hunyuan-TurboS 89.5% Feb 2025 Lab model card
19 Kimi K2 Jul 2025 89.5% Jul 2025 Lab model card
20 InternVL3.5-241B-A28B 89.1% Aug 2025 Lab model card
21 Nemotron 3 Ultra 550B-A55B Base 89.1% Jun 2026 Lab model card
22 GPT-4o May 2024 88.7% May 2024 Lab model card
23 Llama 3.1-405B 88.6% Jul 2024 Lab model card
24 DeepSeek-V3 88.5% Dec 2024 Lab model card
25 MiniMax-Text-01 88.5% Jan 2025 Lab model card
26 GLM-5 88.3% Feb 2026 Lab model card
27 Claude 3 Opus 88.2% Mar 2024 Lab model card
28 Solar Open 100B 88.2% Jan 2026 Lab model card
29 Solar Pro 2 88.2% Jul 2025 Lab model card
30 Qwen3-Coder-Next 87.7% Feb 2026 Lab model card
31 Hermes 4.3 36B 87.7% Dec 2025 Lab model card
32 Grok-2 Aug 2024 87.5% Aug 2024 Lab model card
33 GPT-4.1 mini 87.5% Apr 2025 Lab model card
34 Kimi k1.5 87.4% Jan 2025 Lab model card
35 Amazon Nova Premier 87.4% Apr 2025 Lab model card
36 GLM-4.5-Air 87.4% Aug 2025 Lab model card
37 seed-oss-36b-instruct 87.4% Aug 2025 Lab model card
38 Trinity Large Preview 87.2% Feb 2026 Lab model card
39 Hermes 4 405B 87.2% Aug 2025 Lab model card
40 sonar 87.1% Feb 2025 Lab model card
41 Tulu 3 405B 87% Jan 2025 Lab model card
42 Sarvam-M 87% May 2025 Lab model card
43 o3-mini 86.9% Jan 2025 Lab model card
44 GPT-4 Turbo Apr 2024 86.7% Apr 2024 Lab model card
45 ERNIE-4.5-300B-A47B 86.5% Jun 2025 Lab model card
46 GPT-4 Mar 2023 86.4% Mar 2023 Lab model card
47 Olmo 3.1 32B Think 86.4% Dec 2025 Lab model card
48 Qwen2.5-72B 86.1% Sep 2024 Lab model card
49 Llama 3.1-70B 86% Jul 2024 Lab model card
50 Llama 3.3 70B Instruct 86% Dec 2024 Lab model card
51 Cohere Command A 85.5% Apr 2025 Lab model card
52 Llama 4 Maverick 85.5% Apr 2025 Lab model card
53 Olmo 3 32B Think 85.4% Nov 2025 Lab model card
54 gpt-oss-20b 85.3% Aug 2025 Lab model card
55 Sarvam 30B 85.1% Mar 2026 Lab model card
56 Trinity Mini 85% Dec 2025 Lab model card
57 Phi-4 84.8% Dec 2024 Lab model card
58 Nemotron 3.5 Super 120B-A12B Base 84.1% Oct 2026 Lab model card
59 Falcon-H1-34B 84% May 2025 Lab model card
60 Mistral Large 2 Jul 2024 84% Jul 2024 Lab model card
61 NVIDIA Nemotron 3.5 Transcribe 30B-A4B 84% Oct 2026 Lab model card
62 EXAONE Deep 32B 83% Mar 2025 Lab model card
63 Qwen2-72B 82.3% Jun 2024 Lab model card
64 Llama 3-70B 82% Apr 2024 Lab model card
65 GPT-4o mini 82% Jul 2024 Lab model card
66 Kimi-VL-A3B-Thinking-2506 82% Jun 2025 Lab model card
67 Gemini 1.5 Pro Feb 2024 81.9% Feb 2024 Lab model card
68 GLM-4 81.5% Jan 2024 Lab model card
69 Claude 3 Sonnet 81.5% Mar 2024 Lab model card
70 Grok-1.5 81.3% Mar 2024 Lab model card
71 Mistral Large 81.2% Feb 2024 Lab model card
72 Claude 3.5 Haiku 80.9% Oct 2024 Lab model card
73 Mistral Small 3 80.7% Jan 2025 Lab model card
74 Mistral Small 3.1 80.6% Mar 2025 Lab model card
75 Mistral Small 3.2 80.5% Jun 2025 Lab model card
76 Apple Foundation Model Server (2025) 80.2% Jul 2025 Lab model card
77 Granite 4.1 30B 80.2% Apr 2026 Lab model card
78 GPT-4.1 nano 80.1% Apr 2025 Lab model card
79 Llama 4 Scout 79.6% Apr 2025 Lab model card
80 Phi-3.5-MoE 78.9% Aug 2024 Lab model card
81 Nemotron 3.5 Lightning 30B-A3B Base 78.6% Oct 2026 Lab model card
82 Gemma 3 27B 78.6% Mar 2025 Lab model card
83 Nemotron 3 Nano 30B-A3B Base 78.6% Dec 2025 Lab model card
84 Claude 2 78.5% Jul 2023 Lab model card
85 Granite 4.0 H Small 78.4% Oct 2025 Lab model card
86 phi-3-medium 14B 78.2% Apr 2024 Lab model card
87 DeepSeek-V2 (MoE-236B, May 2024) 77.8% May 2024 Lab model card
88 Code World Model (CWM) 77.7% Sep 2025 Lab model card
89 Qwen1.5-72B 77.5% Feb 2024 Lab model card
90 OLMo 2 32B Instruct 77.3% Mar 2025 Lab model card
91 Claude 3 Haiku 76.7% Mar 2024 Lab model card
92 InternLM3-8B-Instruct 76.6% Jan 2025 Lab model card
93 Gemma 2 27B 76.2% Jul 2024 Lab model card
94 phi-3-small 7.4B 75.7% Aug 2024 Lab model card
95 Grok-1 73% Nov 2023 Lab model card
96 Llama 3.1-8B 73% Jul 2024 Lab model card
97 Mixtral 8x7B 72.5% Dec 2023 Lab model card
98 Gemma 2 9B 72.3% Jul 2024 Lab model card
99 Gemma 3 12B 71.9% Mar 2025 Lab model card
100 DeepSeek LLM 67B 71.1% Nov 2023 Lab model card
101 Moonlight-16B-A3B 70% Feb 2025 Lab model card
102 Apertus 70B 69.6% Sep 2025 Lab model card
103 Phi-3.5-mini 69% Aug 2024 Lab model card
104 Llama 2-70B 68.9% Jul 2023 Lab model card
105 phi-3-mini 3.8B 68.8% Aug 2024 Lab model card
106 Llama 3-8B 68.4% Apr 2024 Lab model card
107 Phi-4 Mini 67.3% Feb 2025 Lab model card
108 Granite 3.2 8B Instruct 66.8% Feb 2025 Lab model card
109 Qwen-14B 66.5% Sep 2023 Lab model card
110 AFM-4.5B 65.9% Jul 2025 Lab model card
111 Granite 3.3 8B Instruct 65.5% Apr 2025 Lab model card
112 Gemma 3n E4B 64.9% Jun 2025 Lab model card
113 LFM2-8B-A1B 64.8% Oct 2025 Lab model card
114 Gemma 7B 64.3% Feb 2024 Lab model card
115 Falcon-Arabic-7B 63.3% Nov 2025 Lab model card
116 Llama 2-34B 62.6% Jul 2023 Lab model card
117 ChatGLM3-6B 61.4% Oct 2023 Lab model card
118 Mistral 7B v0.1 60.1% Sep 2023 Lab model card
119 granite-4.0-1b 59.4% Oct 2025 Lab model card
120 Gemma 3 4B 58.1% Mar 2025 Lab model card
121 Qwen-7B 57% Sep 2023 Lab model card
122 Phi-2 56.7% Dec 2023 Lab model card
123 LFM2-1.2B 55.2% Jul 2025 Lab model card
124 Llama 2-13B 54.8% Jul 2023 Lab model card
125 BitNet b1.58 2B4T 53.2% Apr 2025 Lab model card
126 LFM2-VL-1.6B 51% Aug 2025 Lab model card
127 North Micro Vision Instruct 50.4% Aug 2026 Lab model card
128 Llama 2-7B 45.3% Jul 2023 Lab model card
129 Qwen-1_8B 43.9% Sep 2023 Lab model card
130 Gemma 2B 42.3% Mar 2024 Lab model card
131 OLMo 2 1B Instruct 40% Apr 2025 Lab model card
132 Gemma 3 1B 38.8% Mar 2025 Lab model card
#ModelScoreSettingDateSource
1 Gopher (280B) 60% few-shot Dec 2021 Official leaderboard
2 GPT-3 6.7B 43.2% fine-tuned Jun 2020 Official leaderboard
#ModelScoreSettingDateSource
1 Seed1.8 92.3% Dec 2025 Lab model card
2 o1 91.8% Dec 2024 Lab model card
3 Gemini 1.5 Pro May 2024 91.7% May 2024 Lab model card
4 Cogito v2 Preview 671B MoE 91.3% Jul 2025 Lab model card
5 Cogito v1 Preview 70B 91% Apr 2025 Lab model card
6 o1-preview 90.8% Sep 2024 Lab model card
7 DeepSeek-R1 90.8% Jan 2025 Lab model card
8 Qwen3-VL-235B-A22B 90.6% Sep 2025 Lab model card
9 Sarvam 105B 90.6% Mar 2026 Lab model card
10 Claude 3.5 Sonnet (October 2024) 90.5% Oct 2024 Lab model card
11 Claude 3.5 Sonnet 90.4% Jun 2024 Lab model card
12 LongCat-Flash-Omni 90.3% Oct 2025 Lab model card
13 DeepSeek-V4-Pro 90.1% Apr 2026 Lab model card
14 Gemini 1.0 Ultra 90% Dec 2023 Lab model card
15 gpt-oss-120b 90% Aug 2025 Lab model card
16 GLM-4.5 90% Aug 2025 Lab model card
17 LongCat-Flash-Chat 89.7% Aug 2025 Lab model card
18 Hunyuan-TurboS 89.5% Feb 2025 Lab model card
19 Kimi K2 Jul 2025 89.5% Jul 2025 Lab model card
20 InternVL3.5-241B-A28B 89.1% Aug 2025 Lab model card
21 Nemotron 3 Ultra 550B-A55B Base 89.1% Jun 2026 Lab model card
22 GPT-4o May 2024 88.7% May 2024 Lab model card
23 Llama 3.1-405B 88.6% Jul 2024 Lab model card
24 DeepSeek-V3 88.5% Dec 2024 Lab model card
25 MiniMax-Text-01 88.5% Jan 2025 Lab model card
26 GLM-5 88.3% Feb 2026 Lab model card
27 Claude 3 Opus 88.2% Mar 2024 Lab model card
28 Solar Open 100B 88.2% Jan 2026 Lab model card
29 Solar Pro 2 88.2% Jul 2025 Lab model card
30 GPT-4o Nov 2024 88.1% Nov 2024 Claim in Phi-4 Technical Report, via Epoch AI
31 Qwen3-Coder-Next 87.7% Feb 2026 Lab model card
32 Hermes 4.3 36B 87.7% Dec 2025 Lab model card
33 Grok-2 Aug 2024 87.5% Aug 2024 Lab model card
34 GPT-4.1 mini 87.5% Apr 2025 Lab model card
35 Kimi k1.5 87.4% Jan 2025 Lab model card
36 Amazon Nova Premier 87.4% Apr 2025 Lab model card
37 GLM-4.5-Air 87.4% Aug 2025 Lab model card
38 seed-oss-36b-instruct 87.4% Aug 2025 Lab model card
39 Trinity Large Preview 87.2% Feb 2026 Lab model card
40 Hermes 4 405B 87.2% Aug 2025 Lab model card
41 sonar 87.1% Feb 2025 Lab model card
42 Tulu 3 405B 87% Jan 2025 Lab model card
43 Sarvam-M 87% May 2025 Lab model card
44 o3-mini 86.9% Jan 2025 Lab model card
45 GPT-4 Turbo Apr 2024 86.7% Apr 2024 Lab model card
46 ERNIE-4.5-300B-A47B 86.5% Jun 2025 Lab model card
47 GPT-4 Mar 2023 86.4% Mar 2023 Lab model card
48 Olmo 3.1 32B Think 86.4% Dec 2025 Lab model card
49 Llama 3.3 70B 86.3% Dec 2024 Claim in Phi-4 Technical Report, via Epoch AI
50 Qwen2.5-72B 86.1% Sep 2024 Lab model card
51 Llama 3.1-70B 86% Jul 2024 Lab model card
52 Llama 3.3 70B Instruct 86% Dec 2024 Lab model card
53 Cohere Command A 85.5% Apr 2025 Lab model card
54 Llama 4 Maverick 85.5% Apr 2025 Lab model card
55 Olmo 3 32B Think 85.4% Nov 2025 Lab model card
56 gpt-oss-20b 85.3% Aug 2025 Lab model card
57 Sarvam 30B 85.1% Mar 2026 Lab model card
58 Trinity Mini 85% Dec 2025 Lab model card
59 Phi-4 84.8% Dec 2024 Lab model card
60 Nemotron 3.5 Super 120B-A12B Base 84.1% Oct 2026 Lab model card
61 Falcon-H1-34B 84% May 2025 Lab model card
62 Mistral Large 2 Jul 2024 84% Jul 2024 Lab model card
63 NVIDIA Nemotron 3.5 Transcribe 30B-A4B 84% Oct 2026 Lab model card
64 EXAONE Deep 32B 83% Mar 2025 Lab model card
65 Qwen2-72B 82.3% Jun 2024 Lab model card
66 Llama 3-70B 82% Apr 2024 Lab model card
67 GPT-4o mini 82% Jul 2024 Lab model card
68 Kimi-VL-A3B-Thinking-2506 82% Jun 2025 Lab model card
69 Gemini 1.5 Pro Feb 2024 81.9% Feb 2024 Lab model card
70 GLM-4 81.5% Jan 2024 Lab model card
71 Claude 3 Sonnet 81.5% Mar 2024 Lab model card
72 Grok-1.5 81.3% Mar 2024 Lab model card
73 Mistral Large 81.2% Feb 2024 Lab model card
74 Claude 3.5 Haiku 80.9% Oct 2024 Lab model card
75 Mistral Small 3 80.7% Jan 2025 Lab model card
76 Mistral Small 3.1 80.6% Mar 2025 Lab model card
77 Mistral Small 3.2 80.5% Jun 2025 Lab model card
78 Apple Foundation Model Server (2025) 80.2% Jul 2025 Lab model card
79 Granite 4.1 30B 80.2% Apr 2026 Lab model card
80 GPT-4.1 nano 80.1% Apr 2025 Lab model card
81 Qwen2.5-14B 79.9% Sep 2024 Claim in Phi-4 Technical Report, via Epoch AI
82 Llama 4 Scout 79.6% Apr 2025 Lab model card
83 Qwen2.5-Coder-32B 79.1% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
84 Phi-3.5-MoE 78.9% Aug 2024 Lab model card
85 Nemotron 3.5 Lightning 30B-A3B Base 78.6% Oct 2026 Lab model card
86 Gemma 3 27B 78.6% Mar 2025 Lab model card
87 Nemotron 3 Nano 30B-A3B Base 78.6% Dec 2025 Lab model card
88 Claude 2 78.5% Jul 2023 Lab model card
89 Granite 4.0 H Small 78.4% Oct 2025 Lab model card
90 DeepSeek-V2 (MoE-236B, May 2024) 78.4% May 2024 Claim in DeepSeek-V3 Technical Report, via Epoch AI
91 phi-3-medium 14B 78.2% Apr 2024 Lab model card
92 Code World Model (CWM) 77.7% Sep 2025 Lab model card
93 Qwen1.5-72B 77.5% Feb 2024 Lab model card
94 OLMo 2 32B Instruct 77.3% Mar 2025 Lab model card
95 Claude 1.3 77% Apr 2023 Claim in Model Card and Evaluations for Claude Models, via Epoch AI
96 Claude 3 Haiku 76.7% Mar 2024 Lab model card
97 InternLM3-8B-Instruct 76.6% Jan 2025 Lab model card
98 Yi-34B 76.3% Nov 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
99 Gemma 2 27B 76.2% Jul 2024 Lab model card
100 phi-3-small 7.4B 75.7% Aug 2024 Lab model card
101 Qwen2.5-Coder-14B 75.2% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
102 Claude Instant 73.4% Aug 2023 Claim in Model Card and Evaluations for Claude Models, via Epoch AI
103 Grok-1 73% Nov 2023 Lab model card
104 Llama 3.1-8B 73% Jul 2024 Lab model card
105 Inflection-1 72.7% Jun 2023 Claim in Inflection-1, via Epoch AI
106 Mixtral 8x7B 72.5% Dec 2023 Lab model card
107 Gemma 2 9B 72.3% Jul 2024 Lab model card
108 Gemma 3 12B 71.9% Mar 2025 Lab model card
109 GPT-3.5 Turbo Nov 2023 71.4% Nov 2023 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
110 DeepSeek LLM 67B 71.1% Nov 2023 Lab model card
111 Falcon-180B 70.6% Sep 2023 Claim in The Falcon Series of Open Language Models, via Epoch AI
112 davinci-002 70% Mar 2022 Claim in GPT-4 Technical Report, via Epoch AI
113 Moonlight-16B-A3B 70% Feb 2025 Lab model card
114 Llama 2-70B 69.9% Jul 2023 Claim in Mixtral of Experts, via Epoch AI
115 Apertus 70B 69.6% Sep 2025 Lab model card
116 PaLM (540B) 69.3% Apr 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
117 Phi-3.5-mini 69% Aug 2024 Lab model card
118 phi-3-mini 3.8B 68.8% Aug 2024 Lab model card
119 Stable Beluga 2 68.6% Jul 2023 Claim in Qwen Technical Report, via Epoch AI
120 Yi-9B 68.4% Mar 2024 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
121 Llama 3-8B 68.4% Apr 2024 Lab model card
122 Qwen2.5-Coder (7B) 68% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
123 Chinchilla 67.5% Mar 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
124 Phi-4 Mini 67.3% Feb 2025 Lab model card
125 Granite 3.2 8B Instruct 66.8% Feb 2025 Lab model card
126 Qwen-14B 66.5% Sep 2023 Lab model card
127 AFM-4.5B 65.9% Jul 2025 Lab model card
128 Granite 3.3 8B Instruct 65.5% Apr 2025 Lab model card
129 Gemma 3n E4B 64.9% Jun 2025 Lab model card
130 LFM2-8B-A1B 64.8% Oct 2025 Lab model card
131 Gemma 7B 64.3% Feb 2024 Lab model card
132 StarCoder 2 15B 64.1% Feb 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
133 LLaMA-65B 63.4% Feb 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
134 Falcon-Arabic-7B 63.3% Nov 2025 Lab model card
135 Yi 6B 63.2% Nov 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
136 Llama 2-34B 62.6% Jul 2023 Lab model card
137 Mistral 7B v0.1 62.5% Sep 2023 Claim in Mixtral of Experts, via Epoch AI
138 Mistral 7B v0.2 62.5% Dec 2023 Claim in Gemma: Open Models Based on Gemini Research and Technology, via Epoch AI
139 ChatGLM3-6B 61.4% Oct 2023 Lab model card
140 DeepSeek-Coder-V2-Lite-Base 60.5% Jun 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
141 Gopher (280B) 60% few-shot Dec 2021 Official leaderboard
142 granite-4.0-1b 59.4% Oct 2025 Lab model card
143 Baichuan2-13B 59.2% Sep 2023 Claim in Nemotron-4 15B Technical Report, via Epoch AI
144 LLaMA-33B 58.7% Feb 2023 Claim in Qwen Technical Report, via Epoch AI
145 Nemotron-4 15B 58.7% Feb 2024 Claim in Nemotron-4 15B Technical Report, via Epoch AI
146 Falcon 2 11B 58.4% May 2024 Claim in Falcon2-11B Technical Report, via Epoch AI
147 Gemma 3 4B 58.1% Mar 2025 Lab model card
148 internlm-chat-20b 57.4% Sep 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
149 Qwen-7B 57% Sep 2023 Lab model card
150 Falcon-40B 56.9% Mar 2023 Claim in Falcon2-11B Technical ReportFalcon2-11B Technical Report, via Epoch AI
151 Phi-2 56.7% Dec 2023 Lab model card
152 Llama 2-13B 55.6% Jul 2023 Claim in Mixtral of Experts, via Epoch AI
153 LFM2-1.2B 55.2% Jul 2025 Lab model card
154 Baichuan2-13B-Chat 55.1% Sep 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
155 Baichuan 2-7B 54.2% Sep 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
156 PaLM 62B 53.7% Apr 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
157 Qwen2.5-Coder (1.5B) 53.6% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
158 BitNet b1.58 2B4T 53.2% Apr 2025 Lab model card
159 Baichuan 1-13B 51.6% Jul 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
160 internlm-7b 51% Jul 2023 Claim in Qwen Technical Report, via Epoch AI
161 LFM2-VL-1.6B 51% Aug 2025 Lab model card
162 North Micro Vision Instruct 50.4% Aug 2026 Lab model card
163 INTELLECT-1 49.9% Nov 2024 Claim in INTELLECT-1 Technical Report, via Epoch AI
164 MPT-30B 47.9% Jun 2023 Claim in Qwen Technical Report, via Epoch AI
165 chatglm2-6b 47.9% Jun 2023 Claim in Qwen Technical Report, via Epoch AI
166 LLaMA-13B 47.7% Feb 2023 Claim in Qwen Technical Report, via Epoch AI
167 Llama 2-7B 45.7% Jul 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
168 InstructGPT 175B 43.9% Jan 2022 Claim in Scaling Language Models: Methods, Analysis & Insights fro…, via Epoch AI
169 Qwen-1_8B 43.9% Sep 2023 Lab model card
170 GPT-3 6.7B 43.2% fine-tuned Jun 2020 Official leaderboard
171 Baichuan1-7B 42.3% Jun 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
172 Gemma 2B 42.3% Mar 2024 Lab model card
173 Qwen2.5-Coder-0.5B 42% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
174 CodeQwen1.5-7B 40.5% Apr 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
175 OLMo 2 1B Instruct 40% Apr 2025 Lab model card
176 DeepSeek Coder 33B 39.4% Nov 2023 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
177 StarCoder 2 7B 38.8% Feb 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
178 Gemma 3 1B 38.8% Mar 2025 Lab model card
179 Phi-1.5 37.6% Sep 2023 Claim in Textbooks Are All You Need II: phi-1.5 technical report, via Epoch AI
180 StarCoder 2 3B 36.6% Feb 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
181 DeepSeek Coder 6.7B 36.4% Nov 2023 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
182 XGen-7B 36.3% Jun 2023 Claim in XGen-7B Technical Report, via Epoch AI
183 LLaMA-7B 35.6% Feb 2023 Claim in Qwen Technical Report, via Epoch AI
184 Falcon-7B 35% Apr 2023 Claim in Falcon2-11B Technical Report, via Epoch AI
185 MPT-7B 30.8% May 2023 Claim in Qwen Technical Report, via Epoch AI
186 open_llama_7b 29.9% Jun 2023 Claim in XGen-7B Technical Report, via Epoch AI
187 RedPajama-INCITE-7B-Base 26.3% May 2023 Claim in XGen-7B Technical Report, via Epoch AI
188 Cerebras-GPT-13B 26.2% Mar 2023 Claim in XGen-7B Technical Report, via Epoch AI
189 Dolly 2.0-12b 26.2% Apr 2023 Claim in XGen-7B Technical Report, via Epoch AI
190 DeepSeek Coder 1.3B 25.8% Nov 2023 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
191 GPT-J-6B 25.7% Aug 2021 Claim in XGen-7B Technical Report, via Epoch AI
192 OPT-1.3B 25.1% May 2022 Claim in XGen-7B Technical Report, via Epoch AI

The benchmark's own leaderboard has 10 entries, 2 of them models we track. Epoch AI collected claims for 85 models from labs’ reports and papers. Switch the scores above to rank them together.

5 models have held number one in labs’ own numbers since 2023 Number one over time 6 models have held number one with claims included since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Google DeepMind ByteDance Seed
Lab claims Passed 85% in Mar 2023, by GPT-4
Seed1.8 92.3%

Source: Labs’ model cards

OpenAI’s own number for GPT-4 passed 85% in March 2023

That was 30 months after its release. All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Reported by the lab in its model card A claim Epoch AI collected from a report or paper Best so far
0% 20% 40% 60% 80% 100% 2020 2021 2022 2023 2024 2025 2026 2027 Near the ceiling (85%) Random guessing Seed1.8 Gemini 1.5 Pro GPT-4 davinci-002 Gopher (280B) GPT-3 6.7B 0% 20% 40% 60% 80% 100% 2020 2022 2024 2026 Near the ceiling (85%) Random guessing Seed1.8 davinci-002

Sources: Official MMLU leaderboard, labs’ model cards and claims collected by Epoch AI (CC BY 4.0)

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

OpenAI
Anthropic
Meta
Qwen
Mistral AI
Z.ai
xAI
DeepSeek
Google DeepMind
Microsoft
Shanghai AI Lab
MiniMax
Moonshot AI
Ai2
Perplexity
IBM
Tencent
Cohere
Deep Cogito
Amazon
Technology Innovation Institute
Sarvam AI
Baidu
Upstage
Liquid AI
Arcee AI
ByteDance Seed
Nous Research
Meituan
NVIDIA
DateLaunch postScoreAs the post writes it
Oct 28, 2026 NVIDIA Nemotron 3.5 Transcribe 30B-A4B 84 MMLU — Accuracy
Aug 12, 2026 North Micro Vision Instruct 50.4 MMLU test
Apr 29, 2026 Granite 4.1 30B 80.2 MMLU (5-shot)
Apr 24, 2026 DeepSeek-V4-Pro 90.1 MMLU (EM)
Mar 6, 2026 Sarvam 30B 85.1 MMLU
Mar 6, 2026 Sarvam 105B 90.6 MMLU
Jan 27, 2026 Trinity Large Preview 87.2 MMLU
Jan 5, 2026 Solar Open 100B 88.2 MMLU
Dec 18, 2025 Seed1.8 92.3 MMLU
Dec 12, 2025 Olmo 3.1 32B Think 86.4 MMLU
Dec 3, 2025 Hermes 4.3 36B 87.7 MMLU
Dec 1, 2025 Trinity Mini 85 MMLU (Zero Shot)
Nov 20, 2025 Olmo 3 32B Think 85.4 MMLU
Oct 31, 2025 LongCat-Flash-Omni 90.3 MMLU(acc)
Oct 28, 2025 granite-4.0-1b 59.4 MMLU (5-shot)
Oct 7, 2025 LFM2-8B-A1B 64.8 MMLU
Oct 2, 2025 Granite 4.0 H Small 78.4 MMLU (5-shot)
Sep 23, 2025 Qwen3-VL-235B-A22B 90.6 MMLU
Aug 29, 2025 LongCat-Flash-Chat 89.7 MMLU(acc)
Aug 26, 2025 Hermes 4 405B 87.2 MMLU (reasoning)
Aug 20, 2025 seed-oss-36b-instruct 87.4 MMLU
Aug 12, 2025 LFM2-VL-1.6B 51 Text MMLU
Aug 5, 2025 gpt-oss-20b 85.3 MMLU
Aug 5, 2025 gpt-oss-120b 90 MMLU Questions across academic disciplines
Jul 31, 2025 Cogito v2 Preview 671B MoE 91.3 MMLU (thinking)
Jul 29, 2025 AFM-4.5B 65.9 MMLU
Jul 11, 2025 Kimi K2 89.5 MMLU, EM
Jul 10, 2025 LFM2-1.2B 55.2 MMLU
Jul 10, 2025 Solar Pro 2 88.2 MMLU (0-shot, CoT)
Jun 30, 2025 ERNIE-4.5-300B-A47B 86.5 MMLU
Jun 26, 2025 Gemma 3n E4B 64.9 MMLU 0-shot (IT)
Jun 21, 2025 Kimi-VL-A3B-Thinking-2506 82 MMLU
Jun 20, 2025 Mistral Small 3.2 80.5 MMLU
May 23, 2025 Sarvam-M 87 MMLU (0.87)
May 20, 2025 Falcon-H1-34B 84 MMLU
Apr 30, 2025 Amazon Nova Premier 87.4 MMLU
Apr 29, 2025 OLMo 2 1B Instruct 40 MMLU
Apr 16, 2025 Granite 3.3 8B Instruct 65.5 MMLU
Apr 15, 2025 BitNet b1.58 2B4T 53.2 MMLU
Apr 14, 2025 GPT-4.1 nano 80.1 MMLU
Apr 14, 2025 GPT-4.1 mini 87.5 MMLU
Apr 8, 2025 Cogito v1 Preview 70B 91 MMLU (thinking)
Apr 5, 2025 Llama 4 Scout 79.6 MMLU (5-shot) [pretrained]
Apr 5, 2025 Llama 4 Maverick 85.5 MMLU (5-shot) [pretrained]
Mar 17, 2025 Mistral Small 3.1 80.6 MMLU
Mar 13, 2025 Cohere Command A MMLU (chart only)
Mar 13, 2025 OLMo 2 32B Instruct 77.3 MMLU
Mar 12, 2025 Gemma 3 27B 78.6 MMLU 5-shot (PT)
Feb 27, 2025 Hunyuan-TurboS 89.5 MMLU
Feb 26, 2025 Phi-4 Mini 67.3 MMLU (5-shot)
Feb 26, 2025 Granite 3.2 8B Instruct 66.8 MMLU
Feb 22, 2025 Moonlight-16B-A3B 70 MMLU
Feb 11, 2025 sonar 87.1 MMLU (0-shot)
Jan 31, 2025 o3-mini 86.9 MMLU (pass@1)
Jan 30, 2025 Tulu 3 405B 87 MMLU (5 shot, CoT)
Jan 30, 2025 Mistral Small 3 80.7 Best of MMLU (5-shot) [base] / French MMLU [base] / German MMLU [base] / Spanish MMLU [base]
Jan 28, 2025 Qwen2.5-Max MMLU
Jan 20, 2025 Kimi k1.5 87.4 MMLU (EM), short-CoT
Jan 20, 2025 DeepSeek-R1 90.8 MMLU (Pass@1)
Jan 15, 2025 MiniMax-Text-01 88.5 MMLU* (0-shot CoT)
Jan 13, 2025 InternLM3-8B-Instruct 76.6 MMLU(0-shot)
Dec 26, 2024 DeepSeek-V3 88.5 MMLU (EM)
Dec 17, 2024 o1 91.8 MMLU (pass @1)
Dec 12, 2024 Phi-4 84.8 MMLU (simple-evals)
Dec 6, 2024 Llama 3.3 70B Instruct 86 MMLU (CoT), 0-shot, macro_avg/acc
Oct 22, 2024 Claude 3.5 Sonnet (October 2024) 90.5 MMLU (5-shot CoT)
Sep 19, 2024 Qwen2.5-72B 86.1 MMLU
Sep 12, 2024 o1-preview 90.8 MMLU pass@1
Aug 20, 2024 Phi-3.5-MoE 78.9 MMLU (5-shot)
Aug 13, 2024 Grok-2 87.5 MMLU (0-shot CoT)
Jul 24, 2024 Mistral Large 2 84 Best of MMLU [base pretrained] / Multilingual MMLU French [base] / Multilingual MMLU German [base] / Multilingual MMLU Spanish [base] / Multilingual MMLU Italian [base]
Jul 23, 2024 Llama 3.1-405B 88.6 MMLU (0-shot, CoT)
Jul 18, 2024 GPT-4o mini 82 MMLU
Jun 27, 2024 Gemma 2 27B 75.2 MMLU (5-shot, top-1)
Jun 20, 2024 Claude 3.5 Sonnet 90.4 MMLU (5-shot CoT)
Jun 7, 2024 Qwen2-72B 82.3 MMLU
May 14, 2024 Gemini 1.5 Pro 91.7 MMLU, maj@32 (pretrained) (5-shot = 85.9%)
May 13, 2024 GPT-4o 88.7 MMLU (0-shot CoT)
May 7, 2024 DeepSeek-V2 (MoE-236B, May 2024) 77.8 MMLU
Apr 22, 2024 phi-3-medium 14B 78.2 MMLU (5-Shot)
Apr 18, 2024 Llama 3-70B 82 MMLU (5-shot) [instruct]
Apr 11, 2024 GPT-4 Turbo 86.7 MMLU%
Mar 28, 2024 Grok-1.5 81.3 MMLU (5-shot)
Mar 4, 2024 Claude 3 Opus 88.2 MMLU (5-shot CoT)
Feb 26, 2024 Mistral Large 81.2 Best of MMLU / French MMLU / German MMLU / Spanish MMLU / Italian MMLU
Feb 21, 2024 Gemma 7B 64.3 MMLU (5-shot, top-1)
Feb 15, 2024 Gemini 1.5 Pro 81.9 MMLU, 5-shot
Feb 5, 2024 Qwen1.5-72B 77.5 MMLU
Jan 16, 2024 GLM-4 81.5 MMLU (5-shot)
Dec 12, 2023 Phi-2 56.7 MMLU, Table 2
Dec 11, 2023 Mixtral 8x7B 72.5 Best of MMLU (MCQ in 57 subjects) / French MMLU / German MMLU / Spanish MMLU / Italian MMLU
Dec 6, 2023 Gemini 1.0 Ultra 90 MMLU, CoT@32 (5-shot = 83.7%)
Nov 29, 2023 DeepSeek LLM 67B 71.1 MMLU
Nov 3, 2023 Grok-1 73 MMLU (5-shot)
Oct 27, 2023 ChatGLM3-6B 61.4 MMLU
Sep 27, 2023 Mistral 7B v0.1 60.1 MMLU
Sep 25, 2023 Qwen-14B 66.5 MMLU (5-shot)
Jul 18, 2023 Llama 2-70B 68.9 MMLU
Jul 11, 2023 Claude 2 78.5 MMLU (5-shot CoT)
Mar 14, 2023 GPT-4 86.4 MMLU (5-shot)

91 system cards report it, from 27 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 15, 2026 Nemotron 3.5 Super 120B-A12B Base technical report 84.1 MMLU, exact match, Table 6 Capabilities
Oct 15, 2026 Nemotron 3.5 Lightning 30B-A3B Base technical report 78.6 MMLU, exact match, Table 6 Capabilities
Jun 4, 2026 Nemotron 3 Ultra 550B-A55B Base model card 89.1 MMLU, 5-shot acc Capabilities
Mar 21, 2026 Seed1.8 model card 92.3 MMLU (Table 1) Capabilities
Feb 28, 2026 Qwen3-Coder-Next technical report 87.7 MMLU (Table 8) Capabilities
Feb 19, 2026 Trinity Large Preview technical report 87.2 MMLU, Trinity Large Preview (Table 4) Capabilities
Feb 17, 2026 GLM-5 technical report 88.3 GLM-5-Base, MMLU (EM) (Table 11) Capabilities
Jan 11, 2026 Solar Open 100B technical report 88.2 MMLU (Table 6) Capabilities
Dec 23, 2025 Nemotron 3 Nano 30B-A3B Base technical report 78.6 MMLU, 5-shot accuracy, base model Table 2 Capabilities
Dec 15, 2025 Olmo 3 32B Think technical report 85.4 MMLU, Final Think 3.0 column (Table 14) Capabilities
Dec 15, 2025 Olmo 3.1 32B Think technical report 86.4 MMLU (Tables 1 and 14) Capabilities
Nov 28, 2025 LFM2-8B-A1B technical report 64.8 MMLU, LFM2-8B-A1B (Table 7) Capabilities
Nov 28, 2025 LFM2-1.2B technical report 55.2 MMLU, LFM2-1.2B (Table 6) Capabilities
Nov 8, 2025 Falcon-Arabic-7B technical report 63.3 MMLU 5-shot (Table 8) Capabilities
Oct 31, 2025 LongCat-Flash-Omni technical report 90.3 MMLU (acc) (Table 13) Capabilities
Oct 8, 2025 OLMo 2 1B Instruct technical report 40 MMLU (Table 7, Table 23) Capabilities
Sep 30, 2025 Code World Model (CWM) technical report 77.7 MMLU, without reasoning (Table 13) Capabilities
Sep 17, 2025 Apertus 70B technical report 69.6 MMLU, 0-shot CoT exact match (Table 17) Capabilities
Sep 1, 2025 LongCat-Flash-Chat technical report 89.7 MMLU (acc), model-judged answer matching (Table 3) Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 89.1 MMLU, OpenCompass (Table 14) Capabilities
Aug 25, 2025 Hermes 4 405B technical report 87.2 MMLU, reasoning mode (Table 3). Non-reasoning 73.6 Capabilities
Aug 8, 2025 GLM-4.5 technical report 90 MMLU (EM) (Table 6) Capabilities
Aug 8, 2025 GLM-4.5-Air technical report 87.4 MMLU (EM) (Table 6) Capabilities
Aug 5, 2025 gpt-oss-20b model card 85.3 MMLU, high reasoning (Table 3) Capabilities
Aug 5, 2025 gpt-oss-120b model card 90 MMLU, high reasoning (Table 3) Capabilities
Jul 30, 2025 Falcon-H1-34B technical report 84 MMLU, logprobs 5-shot (Table 24) Capabilities
Jul 21, 2025 Kimi K2 technical report 89.5 MMLU (EM) (Table 3) Capabilities
Jul 17, 2025 Voxtral Small technical report MMLU 5-shot, text-only, chart without printed numbers (Figure 6). Speech-synthesized MMLU* 74.3 (Table 8) Capabilities
Jul 17, 2025 Apple Foundation Model Server (2025) technical report 80.2 MMLU, Simple-Evals, AFM Server (Table 2). Table 3 prints 80.0 at 16 bits and 79.2 for AFM Server Opt at 3.6 bits per weight Capabilities
Jun 29, 2025 ERNIE-4.5-300B-A47B technical report 86.5 MMLU (Table 5) Capabilities
Apr 30, 2025 Amazon Nova Premier technical report 87.4 MMLU, 0-shot CoT, macro average (Table 1) Capabilities
Apr 16, 2025 BitNet b1.58 2B4T technical report 53.2 MMLU (5-shot; Acc) (Table 1, Table 2, Table 3) Capabilities
Apr 5, 2025 Llama 4 Scout model card 79.6 MMLU, 5-shot, macro_avg/acc_char, pretrained (Pre-trained models table) Capabilities
Apr 5, 2025 Llama 4 Maverick model card 85.5 MMLU, 5-shot, macro_avg/acc_char, pretrained (Pre-trained models table) Capabilities
Apr 1, 2025 Cohere Command A technical report 85.5 MMLU, 5-shot CoT, simple-evals (Table 3, Table 1) Capabilities
Mar 25, 2025 Gemma 3 4B technical report 58.1 MMLU, IT, 0-shot (Table 18) Capabilities
Mar 25, 2025 Gemma 3 27B technical report 76.9 MMLU, IT, 0-shot (Table 18) Capabilities
Mar 25, 2025 Gemma 3 1B technical report 38.8 MMLU, IT, 0-shot (Table 18) Capabilities
Mar 25, 2025 Gemma 3 12B technical report 71.9 MMLU, IT, 0-shot (Table 18) Capabilities
Mar 16, 2025 EXAONE Deep 32B technical report 83 MMLU, accuracy (Table 3) Capabilities
Mar 3, 2025 Phi-4 Mini technical report 67.3 MMLU (5-Shot) (Table 7) Capabilities
Jan 20, 2025 Kimi k1.5 technical report 87.4 MMLU (EM), short-CoT (Table 3) Capabilities
Jan 20, 2025 DeepSeek-R1 technical report 90.8 MMLU (EM), DeepSeek-R1 (Table 8, same as final column of Table 3) Capabilities
Jan 14, 2025 MiniMax-Text-01 technical report 88.5 MMLU, 0-shot CoT (Table 8) Capabilities
Dec 31, 2024 OLMo 2 32B Instruct technical report 77.3 MMLU, 0-shot CoT (Tables 7 and 16) Capabilities
Dec 26, 2024 DeepSeek-V3 technical report 88.5 MMLU (EM), DeepSeek-V3 (Table 6) Capabilities
Dec 12, 2024 Phi-4 technical report 84.8 MMLU, simple-evals (Table 1) Capabilities
Dec 6, 2024 Llama 3.3 70B Instruct model card 86 MMLU (CoT), 0-shot, macro_avg/acc (Instruction tuned models table) Capabilities
Nov 23, 2024 Llama 3.1-8B technical report 73 MMLU (0-shot, CoT), post-trained (Table 2). 5-shot = 69.4 Capabilities
Nov 23, 2024 Llama 3.1-70B technical report 86 MMLU (0-shot, CoT), post-trained (Table 2). 5-shot = 83.6 Capabilities
Nov 23, 2024 Llama 3.1-405B technical report 88.6 MMLU (0-shot, CoT), post-trained (Table 2). 5-shot = 87.3 Capabilities
Nov 22, 2024 Tulu 3 405B technical report 87 MMLU (5 shot, CoT), RLVR (Table 4) Capabilities
Oct 22, 2024 Claude 3.5 Sonnet (October 2024) model card 90.5 MMLU, 5-shot CoT. 5-shot 88.7%, 0-shot CoT 89.3% (Table 7) Capabilities
Oct 22, 2024 Claude 3.5 Haiku model card 80.9 MMLU, 5-shot CoT. 5-shot 77.6%, 0-shot CoT 80.3% (Table 8) Capabilities
Aug 30, 2024 phi-3-small 7.4B technical report 75.7 MMLU (5-Shot) (Section 3 table) Capabilities
Aug 30, 2024 phi-3-mini 3.8B technical report 68.8 MMLU (5-Shot) (Section 3 table) Capabilities
Aug 30, 2024 phi-3-medium 14B technical report 78 MMLU (5-Shot) (Section 3 table) Capabilities
Aug 30, 2024 Phi-3.5-MoE technical report 78.9 MMLU (5-shot) (Table 3) Capabilities
Aug 30, 2024 Phi-3.5-mini technical report 69 MMLU (5-shot) (Table 3) Capabilities
Jul 31, 2024 Gemma 2 9B technical report 72.3 MMLU, IT, 5-shot (Table 17). PT 5-shot (Table 13): 71.3 Capabilities
Jul 31, 2024 Gemma 2 27B technical report 76.2 MMLU, IT, 5-shot (Table 17). PT 5-shot (Table 13): 75.2 Capabilities
Jul 23, 2024 Llama 3.1-8B model card 73 MMLU (CoT), 0-shot, macro_avg/acc, instruction tuned (Instruction tuned models table). 5-shot = 69.4 Capabilities
Jul 23, 2024 Llama 3.1-70B model card 86 MMLU (CoT), 0-shot, macro_avg/acc, instruction tuned (Instruction tuned models table). 5-shot = 83.6 Capabilities
Jul 23, 2024 Llama 3.1-405B model card 88.6 MMLU (CoT), 0-shot, macro_avg/acc, instruction tuned (Instruction tuned models table). 5-shot = 87.3 Capabilities
Jul 15, 2024 Qwen2-72B technical report 82.3 MMLU, Qwen2-72B-Instruct (Table 6). 83.2 on the non-contaminated subset (Table 15) Capabilities
Jun 20, 2024 Claude 3.5 Sonnet model card 90.4 MMLU, 5-shot CoT. 5-shot 88.7, 0-shot CoT 88.3 (Table 1) Capabilities
Jun 18, 2024 GLM-4 technical report 81.5 GLM-4 (0116), MMLU (Table 2) Capabilities
Jun 18, 2024 ChatGLM3-6B technical report 61.4 ChatGLM3-6B-Base, MMLU (Table 1) Capabilities
May 7, 2024 DeepSeek-V2 (MoE-236B, May 2024) technical report 77.8 MMLU 5-shot, DeepSeek-V2 Chat (RL) (Table 3) Capabilities
Apr 18, 2024 Llama 3-8B model card 68.4 MMLU (5-shot), instruction tuned (Instruction tuned models table). Pretrained = 66.6 Capabilities
Apr 18, 2024 Llama 3-70B model card 82 MMLU (5-shot), instruction tuned (Instruction tuned models table). Pretrained = 79.5 Capabilities
Mar 13, 2024 Gemma 7B technical report 64.3 MMLU, PT, 5-shot top-1 (Table 6). HuggingFace H6 run (Table 7): 64.6 Capabilities
Mar 13, 2024 Gemma 2B technical report 42.3 MMLU, PT, 5-shot top-1 (Table 6) Capabilities
Mar 4, 2024 Claude 3 Sonnet model card 81.5 MMLU, 5-shot CoT. 5-shot 79.0 (Table 1) Capabilities
Mar 4, 2024 Claude 3 Opus model card 88.2 MMLU, 5-shot CoT. 5-shot 86.8 (Table 1) Capabilities
Mar 4, 2024 Claude 3 Haiku model card 76.7 MMLU, 5-shot CoT. 5-shot 75.2 (Table 1) Capabilities
Jan 8, 2024 Mixtral 8x7B technical report 70.6 MMLU (MCQ in 57 subjects), 5-shot (Table 3; same in Table 2) Capabilities
Jan 5, 2024 DeepSeek LLM 67B technical report 71.1 MMLU 0-shot, DeepSeek 67B Chat (Table 6) Capabilities
Oct 10, 2023 Mistral 7B v0.1 technical report 60.1 MMLU, 5-shot (Table 2) Capabilities
Sep 28, 2023 Qwen-7B technical report 57 MMLU 5-shot, higher of the two settings, 0-shot 55.8 (Table 5) Capabilities
Sep 28, 2023 Qwen-14B technical report 66.5 MMLU 5-shot, higher of the two settings, 0-shot 64.6 (Table 5) Capabilities
Sep 28, 2023 Qwen-1_8B technical report 43.9 MMLU 5-shot, higher of the two settings, 0-shot 42.4 (Table 5) Capabilities
Jul 19, 2023 Llama 2-7B technical report 45.3 MMLU (5-shot), pretrained (Table 3, Table 19) Capabilities
Jul 19, 2023 Llama 2-70B technical report 68.9 MMLU (5-shot), pretrained (Table 3, Table 19) Capabilities
Jul 19, 2023 Llama 2-34B technical report 62.6 MMLU (5-shot), pretrained (Table 3, Table 19) Capabilities
Jul 19, 2023 Llama 2-13B technical report 54.8 MMLU (5-shot), pretrained (Table 3, Table 19) Capabilities
Jul 18, 2023 Llama 2-7B model card 45.3 MMLU, pretrained (Overall performance on grouped academic benchmarks) Capabilities
Jul 18, 2023 Llama 2-70B model card 68.9 MMLU, pretrained (Overall performance on grouped academic benchmarks) Capabilities
Jul 18, 2023 Llama 2-13B model card 54.8 MMLU, pretrained (Overall performance on grouped academic benchmarks) Capabilities
Jul 11, 2023 Claude 2 model card 78.5 MMLU (5-shot CoT) (4.3) Capabilities
Mar 14, 2023 GPT-4 technical report 86.4 MMLU, 5-shot (Table 2, base model) Capabilities

Versions

More in Knowledge

All 332 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • HLE-Diamond In use Models answer 1,000 cleaned expert questions from Humanity's Last Exam, 500 on reasoning and 500 on knowledge, closed book. Graded on exact answers. Claude Fable 5.2 77.9%
  • HLE-Verified In use Verified and revised version of Humanity's Last Exam (1,811 items) with uncertain items removed. Gemini 3 Pro 48.2%

Data audit

8 of 9 checks pass, 1 warning. Look first at the Known issues check. 1 finding disputed or fixed only in a copy: Wrong labels and unclear tasks. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (10 entries), launch posts (98 scores), system cards (90 scores) and claims Epoch AI collected (85 models).
  • Pass Models scored 192 models we track, plus 8 other entries on its leaderboard.
  • Pass Newest score The newest is from October 2026, for NVIDIA Nemotron 3.5 Transcribe 30B-A4B.
  • Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V4-Pro (#70).
  • Note Headroom Saturated. The best is 92.3%, past 85% since March 2023.
Leaderboard import 3 pass
  • Pass Importer We read the board on github.com: 2 rows for models we track and 8 other entries at the last read.
  • Pass Last read November 18th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 warning 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Warning Known issues 1 finding disputed or fixed only in a copy: Wrong labels and unclear tasks.

How we check