Benchmark Almanac

MMLU

Seed1.8 92.3% leads since December 2025
Gopher (280B) 60% leads
Seed1.8 92.3% leads since December 2025
Fading 1 known issue

57-subject multiple-choice exam questions from elementary to professional level. Scored by accuracy.

Saturated and contaminated. Reported only as a base-model metric.

Released
September 2020
Kind
Exam questions
Level
University
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Strongest model tested
DeepSeek-V4-Pro #69 on the capability index
Last in a launch post
August 2026, dropped after it saturated
Last model added
October 2026
Scores refreshed
November 12th, 2026

In the authors’ words

We propose a new test to measure a text model's multitask accuracy. The test covers 57 tasks including elementary mathematics, US history, computer science, law, and more. To attain high accuracy on this test, models must possess extensive world knowledge and problem solving ability. We find that while most recent models have near random-chance accuracy, the very largest GPT-3 model improves over random chance by almost 20 percentage points on average. However, on every one of the 57 tasks, the best models still need substantial improvements before they can reach expert-level accuracy. Models also have lopsided performance and frequently do not know when they are wrong. Worse, they still have near-random accuracy on some socially important subjects such as morality and law. By comprehensively evaluating the breadth and depth of a model's academic and professional understanding, our test can be used to analyze models across many tasks and to identify important shortcomings.

Abstract of Measuring Massive Multitask Language Understanding (arXiv:2009.03300, CC0)

Known issues

A published audit found a problem in MMLU. We summarize each finding in our words and link its source. The scores on this page don't include any fixes. See all audits

  1. Wrong labels Unclear tasks Fixed in a copy

    Gema et al. report numerous ground-truth errors in MMLU, with Virology the worst subject they analyzed. They re-annotated 5,700 questions across all 57 subjects as MMLU-Redux and estimate the error rate for all of MMLU.

    57% of the analyzed Virology questions contain errors, and the authors estimate that 6.49% of all MMLU questions contain errors.

    For example, we find that 57% of the analysed questions in the Virology subset contain errors.

    Are We Done with MMLU? by Gema et al., June 6th, 2024

131
Models scored in lab cards
99
Launch posts list it, from 29 labs
91
System cards report it, from 27 labs

Seed1.8 leads with 92.3%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Gopher (280B) leads with 60% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Seed1.8 leads with 92.3%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

131 models. Scroll sideways for the rest.

  1. 92.3 Seed1.8
  2. 91.8 o1
  3. 91.7 Gemini 1.5 Pro
  4. 91.3 Cogito v2 Preview 671B MoE
  5. 91.0 Cogito v1 Preview 70B
  6. 90.8 o1-preview
  7. 90.8 DeepSeek-R1
  8. 90.6 Qwen3-VL-235B-A22B
  9. 90.6 Sarvam 105B
  10. 90.5 Claude 3.5 Sonnet (October 2024)
  11. 90.4 Claude 3.5 Sonnet
  12. 90.3 LongCat-Flash-Omni
  13. 90.1 DeepSeek-V4-Pro
  14. 90.0 Gemini 1.0 Ultra
  15. 90.0 gpt-oss-120b
  16. 90.0 GLM-4.5
  17. 89.7 LongCat-Flash-Chat
  18. 89.5 Hunyuan-TurboS
  19. 89.5 Kimi K2
  20. 89.1 InternVL3.5-241B-A28B
  21. 89.1 Nemotron 3 Ultra 550B-A55B Base
  22. 88.7 GPT-4o
  23. 88.6 Llama 3.1-405B
  24. 88.5 DeepSeek-V3
  25. 88.5 MiniMax-Text-01
  26. 88.3 GLM-5
  27. 88.2 Claude 3 Opus
  28. 88.2 Solar Open 100B
  29. 88.2 Solar Pro 2
  30. 87.7 Qwen3-Coder-Next
  31. 87.7 Hermes 4.3 36B
  32. 87.5 Grok-2
  33. 87.5 GPT-4.1 mini
  34. 87.4 Kimi k1.5
  35. 87.4 Amazon Nova Premier
  36. 87.4 GLM-4.5-Air
  37. 87.4 seed-oss-36b-instruct
  38. 87.2 Trinity Large Preview
  39. 87.2 Hermes 4 405B
  40. 87.1 sonar
  41. 87.0 Tulu 3 405B
  42. 87.0 Sarvam-M
  43. 86.9 o3-mini
  44. 86.7 GPT-4 Turbo
  45. 86.5 ERNIE-4.5-300B-A47B
  46. 86.4 GPT-4
  47. 86.4 Olmo 3.1 32B Think
  48. 86.1 Qwen2.5-72B
  49. 86.0 Llama 3.1-70B
  50. 86.0 Llama 3.3 70B Instruct
  51. 85.5 Cohere Command A
  52. 85.5 Llama 4 Maverick
  53. 85.4 Olmo 3 32B Think
  54. 85.3 gpt-oss-20b
  55. 85.1 Sarvam 30B
  56. 85.0 Trinity Mini
  57. 84.8 Phi-4
  58. 84.1 Nemotron 3.5 Super 120B-A12B Base
  59. 84.0 Falcon-H1-34B
  60. 84.0 Mistral Large 2
  61. 83.0 EXAONE Deep 32B
  62. 82.3 Qwen2-72B
  63. 82.0 Llama 3-70B
  64. 82.0 GPT-4o mini
  65. 82.0 Kimi-VL-A3B-Thinking-2506
  66. 81.9 Gemini 1.5 Pro
  67. 81.5 GLM-4
  68. 81.5 Claude 3 Sonnet
  69. 81.3 Grok-1.5
  70. 81.2 Mistral Large
  71. 80.9 Claude 3.5 Haiku
  72. 80.7 Mistral Small 3
  73. 80.6 Mistral Small 3.1
  74. 80.5 Mistral Small 3.2
  75. 80.2 Apple Foundation Model Server (2025)
  76. 80.2 Granite 4.1 30B
  77. 80.1 GPT-4.1 nano
  78. 79.6 Llama 4 Scout
  79. 78.9 Phi-3.5-MoE
  80. 78.6 Nemotron 3.5 Lightning 30B-A3B Base
  81. 78.6 Gemma 3 27B
  82. 78.6 Nemotron 3 Nano 30B-A3B Base
  83. 78.5 Claude 2
  84. 78.4 Granite 4.0 H Small
  85. 78.2 phi-3-medium 14B
  86. 77.8 DeepSeek-V2 (MoE-236B, May 2024)
  87. 77.7 Code World Model (CWM)
  88. 77.5 Qwen1.5-72B
  89. 77.3 OLMo 2 32B Instruct
  90. 76.7 Claude 3 Haiku
  91. 76.6 InternLM3-8B-Instruct
  92. 76.2 Gemma 2 27B
  93. 75.7 phi-3-small 7.4B
  94. 73.0 Grok-1
  95. 73.0 Llama 3.1-8B
  96. 72.5 Mixtral 8x7B
  97. 72.3 Gemma 2 9B
  98. 71.9 Gemma 3 12B
  99. 71.1 DeepSeek LLM 67B
  100. 70.0 Moonlight-16B-A3B
  101. 69.6 Apertus 70B
  102. 69.0 Phi-3.5-mini
  103. 68.9 Llama 2-70B
  104. 68.8 phi-3-mini 3.8B
  105. 68.4 Llama 3-8B
  106. 67.3 Phi-4 Mini
  107. 66.8 Granite 3.2 8B Instruct
  108. 66.5 Qwen-14B
  109. 65.9 AFM-4.5B
  110. 65.5 Granite 3.3 8B Instruct
  111. 64.9 Gemma 3n E4B
  112. 64.8 LFM2-8B-A1B
  113. 64.3 Gemma 7B
  114. 63.3 Falcon-Arabic-7B
  115. 62.6 Llama 2-34B
  116. 61.4 ChatGLM3-6B
  117. 60.1 Mistral 7B v0.1
  118. 59.4 granite-4.0-1b
  119. 58.1 Gemma 3 4B
  120. 57.0 Qwen-7B
  121. 56.7 Phi-2
  122. 55.2 LFM2-1.2B
  123. 54.8 Llama 2-13B
  124. 53.2 BitNet b1.58 2B4T
  125. 51.0 LFM2-VL-1.6B
  126. 50.4 North Micro Vision Instruct
  127. 45.3 Llama 2-7B
  128. 43.9 Qwen-1_8B
  129. 42.3 Gemma 2B
  130. 40.0 OLMo 2 1B Instruct
  131. 38.8 Gemma 3 1B
  1. 1 Seed1.8 92.3%
  2. 2 o1 91.8%
  3. 3 Gemini 1.5 Pro May 2024 91.7%
  4. 4 Cogito v2 Preview 671B MoE 91.3%
  5. 5 Cogito v1 Preview 70B 91%
  6. 6 o1-preview 90.8%
  7. 7 DeepSeek-R1 90.8%
  8. 8 Qwen3-VL-235B-A22B 90.6%
  9. 9 Sarvam 105B 90.6%
  10. 10 Claude 3.5 Sonnet (October 2024) 90.5%
  11. 11 Claude 3.5 Sonnet 90.4%
  12. 12 LongCat-Flash-Omni 90.3%
  13. 13 DeepSeek-V4-Pro 90.1%
  14. 14 Gemini 1.0 Ultra 90%
  15. 15 gpt-oss-120b 90%
  16. 16 GLM-4.5 90%
  17. 17 LongCat-Flash-Chat 89.7%
  18. 18 Hunyuan-TurboS 89.5%
  19. 19 Kimi K2 Jul 2025 89.5%
  20. 20 InternVL3.5-241B-A28B 89.1%
  21. 21 Nemotron 3 Ultra 550B-A55B Base 89.1%
  22. 22 GPT-4o May 2024 88.7%
  23. 23 Llama 3.1-405B 88.6%
  24. 24 DeepSeek-V3 88.5%
  25. 25 MiniMax-Text-01 88.5%
  26. 26 GLM-5 88.3%
  27. 27 Claude 3 Opus 88.2%
  28. 28 Solar Open 100B 88.2%
  29. 29 Solar Pro 2 88.2%
  30. 30 Qwen3-Coder-Next 87.7%
  31. 31 Hermes 4.3 36B 87.7%
  32. 32 Grok-2 Aug 2024 87.5%
  33. 33 GPT-4.1 mini 87.5%
  34. 34 Kimi k1.5 87.4%
  35. 35 Amazon Nova Premier 87.4%
  36. 36 GLM-4.5-Air 87.4%
  37. 37 seed-oss-36b-instruct 87.4%
  38. 38 Trinity Large Preview 87.2%
  39. 39 Hermes 4 405B 87.2%
  40. 40 sonar 87.1%
  41. 41 Tulu 3 405B 87%
  42. 42 Sarvam-M 87%
  43. 43 o3-mini 86.9%
  44. 44 GPT-4 Turbo Apr 2024 86.7%
  45. 45 ERNIE-4.5-300B-A47B 86.5%
  46. 46 GPT-4 Mar 2023 86.4%
  47. 47 Olmo 3.1 32B Think 86.4%
  48. 48 Qwen2.5-72B 86.1%
  49. 49 Llama 3.1-70B 86%
  50. 50 Llama 3.3 70B Instruct 86%
  51. 51 Cohere Command A 85.5%
  52. 52 Llama 4 Maverick 85.5%
  53. 53 Olmo 3 32B Think 85.4%
  54. 54 gpt-oss-20b 85.3%
  55. 55 Sarvam 30B 85.1%
  56. 56 Trinity Mini 85%
  57. 57 Phi-4 84.8%
  58. 58 Nemotron 3.5 Super 120B-A12B Base 84.1%
  59. 59 Falcon-H1-34B 84%
  60. 60 Mistral Large 2 Jul 2024 84%
  61. 61 EXAONE Deep 32B 83%
  62. 62 Qwen2-72B 82.3%
  63. 63 Llama 3-70B 82%
  64. 64 GPT-4o mini 82%
  65. 65 Kimi-VL-A3B-Thinking-2506 82%
  66. 66 Gemini 1.5 Pro Feb 2024 81.9%
  67. 67 GLM-4 81.5%
  68. 68 Claude 3 Sonnet 81.5%
  69. 69 Grok-1.5 81.3%
  70. 70 Mistral Large 81.2%
  71. 71 Claude 3.5 Haiku 80.9%
  72. 72 Mistral Small 3 80.7%
  73. 73 Mistral Small 3.1 80.6%
  74. 74 Mistral Small 3.2 80.5%
  75. 75 Apple Foundation Model Server (2025) 80.2%
  76. 76 Granite 4.1 30B 80.2%
  77. 77 GPT-4.1 nano 80.1%
  78. 78 Llama 4 Scout 79.6%
  79. 79 Phi-3.5-MoE 78.9%
  80. 80 Nemotron 3.5 Lightning 30B-A3B Base 78.6%
  81. 81 Gemma 3 27B 78.6%
  82. 82 Nemotron 3 Nano 30B-A3B Base 78.6%
  83. 83 Claude 2 78.5%
  84. 84 Granite 4.0 H Small 78.4%
  85. 85 phi-3-medium 14B 78.2%
  86. 86 DeepSeek-V2 (MoE-236B, May 2024) 77.8%
  87. 87 Code World Model (CWM) 77.7%
  88. 88 Qwen1.5-72B 77.5%
  89. 89 OLMo 2 32B Instruct 77.3%
  90. 90 Claude 3 Haiku 76.7%
  91. 91 InternLM3-8B-Instruct 76.6%
  92. 92 Gemma 2 27B 76.2%
  93. 93 phi-3-small 7.4B 75.7%
  94. 94 Grok-1 73%
  95. 95 Llama 3.1-8B 73%
  96. 96 Mixtral 8x7B 72.5%
  97. 97 Gemma 2 9B 72.3%
  98. 98 Gemma 3 12B 71.9%
  99. 99 DeepSeek LLM 67B 71.1%
  100. 100 Moonlight-16B-A3B 70%
  101. 101 Apertus 70B 69.6%
  102. 102 Phi-3.5-mini 69%
  103. 103 Llama 2-70B 68.9%
  104. 104 phi-3-mini 3.8B 68.8%
  105. 105 Llama 3-8B 68.4%
  106. 106 Phi-4 Mini 67.3%
  107. 107 Granite 3.2 8B Instruct 66.8%
  108. 108 Qwen-14B 66.5%
  109. 109 AFM-4.5B 65.9%
  110. 110 Granite 3.3 8B Instruct 65.5%
  111. 111 Gemma 3n E4B 64.9%
  112. 112 LFM2-8B-A1B 64.8%
  113. 113 Gemma 7B 64.3%
  114. 114 Falcon-Arabic-7B 63.3%
  115. 115 Llama 2-34B 62.6%
  116. 116 ChatGLM3-6B 61.4%
  117. 117 Mistral 7B v0.1 60.1%
  118. 118 granite-4.0-1b 59.4%
  119. 119 Gemma 3 4B 58.1%
  120. 120 Qwen-7B 57%
  121. 121 Phi-2 56.7%
  122. 122 LFM2-1.2B 55.2%
  123. 123 Llama 2-13B 54.8%
  124. 124 BitNet b1.58 2B4T 53.2%
  125. 125 LFM2-VL-1.6B 51%
  126. 126 North Micro Vision Instruct 50.4%
  127. 127 Llama 2-7B 45.3%
  128. 128 Qwen-1_8B 43.9%
  129. 129 Gemma 2B 42.3%
  130. 130 OLMo 2 1B Instruct 40%
  131. 131 Gemma 3 1B 38.8%

Source: Labs’ model cards

Source: Official MMLU leaderboard

191 models. Scroll sideways for the rest.

  1. 92.3 Seed1.8
  2. 91.8 o1
  3. 91.7 Gemini 1.5 Pro
  4. 91.3 Cogito v2 Preview 671B MoE
  5. 91.0 Cogito v1 Preview 70B
  6. 90.8 o1-preview
  7. 90.8 DeepSeek-R1
  8. 90.6 Qwen3-VL-235B-A22B
  9. 90.6 Sarvam 105B
  10. 90.5 Claude 3.5 Sonnet (October 2024)
  11. 90.4 Claude 3.5 Sonnet
  12. 90.3 LongCat-Flash-Omni
  13. 90.1 DeepSeek-V4-Pro
  14. 90.0 Gemini 1.0 Ultra
  15. 90.0 gpt-oss-120b
  16. 90.0 GLM-4.5
  17. 89.7 LongCat-Flash-Chat
  18. 89.5 Hunyuan-TurboS
  19. 89.5 Kimi K2
  20. 89.1 InternVL3.5-241B-A28B
  21. 89.1 Nemotron 3 Ultra 550B-A55B Base
  22. 88.7 GPT-4o
  23. 88.6 Llama 3.1-405B
  24. 88.5 DeepSeek-V3
  25. 88.5 MiniMax-Text-01
  26. 88.3 GLM-5
  27. 88.2 Claude 3 Opus
  28. 88.2 Solar Open 100B
  29. 88.2 Solar Pro 2
  30. 88.1 GPT-4o
  31. 87.7 Qwen3-Coder-Next
  32. 87.7 Hermes 4.3 36B
  33. 87.5 Grok-2
  34. 87.5 GPT-4.1 mini
  35. 87.4 Kimi k1.5
  36. 87.4 Amazon Nova Premier
  37. 87.4 GLM-4.5-Air
  38. 87.4 seed-oss-36b-instruct
  39. 87.2 Trinity Large Preview
  40. 87.2 Hermes 4 405B
  41. 87.1 sonar
  42. 87.0 Tulu 3 405B
  43. 87.0 Sarvam-M
  44. 86.9 o3-mini
  45. 86.7 GPT-4 Turbo
  46. 86.5 ERNIE-4.5-300B-A47B
  47. 86.4 GPT-4
  48. 86.4 Olmo 3.1 32B Think
  49. 86.3 Llama 3.3 70B
  50. 86.1 Qwen2.5-72B
  51. 86.0 Llama 3.1-70B
  52. 86.0 Llama 3.3 70B Instruct
  53. 85.5 Cohere Command A
  54. 85.5 Llama 4 Maverick
  55. 85.4 Olmo 3 32B Think
  56. 85.3 gpt-oss-20b
  57. 85.1 Sarvam 30B
  58. 85.0 Trinity Mini
  59. 84.8 Phi-4
  60. 84.1 Nemotron 3.5 Super 120B-A12B Base
  61. 84.0 Falcon-H1-34B
  62. 84.0 Mistral Large 2
  63. 83.0 EXAONE Deep 32B
  64. 82.3 Qwen2-72B
  65. 82.0 Llama 3-70B
  66. 82.0 GPT-4o mini
  67. 82.0 Kimi-VL-A3B-Thinking-2506
  68. 81.9 Gemini 1.5 Pro
  69. 81.5 GLM-4
  70. 81.5 Claude 3 Sonnet
  71. 81.3 Grok-1.5
  72. 81.2 Mistral Large
  73. 80.9 Claude 3.5 Haiku
  74. 80.7 Mistral Small 3
  75. 80.6 Mistral Small 3.1
  76. 80.5 Mistral Small 3.2
  77. 80.2 Apple Foundation Model Server (2025)
  78. 80.2 Granite 4.1 30B
  79. 80.1 GPT-4.1 nano
  80. 79.9 Qwen2.5-14B
  81. 79.6 Llama 4 Scout
  82. 79.1 Qwen2.5-Coder-32B
  83. 78.9 Phi-3.5-MoE
  84. 78.6 Nemotron 3.5 Lightning 30B-A3B Base
  85. 78.6 Gemma 3 27B
  86. 78.6 Nemotron 3 Nano 30B-A3B Base
  87. 78.5 Claude 2
  88. 78.4 Granite 4.0 H Small
  89. 78.4 DeepSeek-V2 (MoE-236B, May 2024)
  90. 78.2 phi-3-medium 14B
  91. 77.7 Code World Model (CWM)
  92. 77.5 Qwen1.5-72B
  93. 77.3 OLMo 2 32B Instruct
  94. 77.0 Claude 1.3
  95. 76.7 Claude 3 Haiku
  96. 76.6 InternLM3-8B-Instruct
  97. 76.3 Yi-34B
  98. 76.2 Gemma 2 27B
  99. 75.7 phi-3-small 7.4B
  100. 75.2 Qwen2.5-Coder-14B
  101. 73.4 Claude Instant
  102. 73.0 Grok-1
  103. 73.0 Llama 3.1-8B
  104. 72.7 Inflection-1
  105. 72.5 Mixtral 8x7B
  106. 72.3 Gemma 2 9B
  107. 71.9 Gemma 3 12B
  108. 71.4 GPT-3.5 Turbo
  109. 71.1 DeepSeek LLM 67B
  110. 70.6 Falcon-180B
  111. 70.0 davinci-002
  112. 70.0 Moonlight-16B-A3B
  113. 69.9 Llama 2-70B
  114. 69.6 Apertus 70B
  115. 69.3 PaLM (540B)
  116. 69.0 Phi-3.5-mini
  117. 68.8 phi-3-mini 3.8B
  118. 68.6 Stable Beluga 2
  119. 68.4 Yi-9B
  120. 68.4 Llama 3-8B
  121. 68.0 Qwen2.5-Coder (7B)
  122. 67.5 Chinchilla
  123. 67.3 Phi-4 Mini
  124. 66.8 Granite 3.2 8B Instruct
  125. 66.5 Qwen-14B
  126. 65.9 AFM-4.5B
  127. 65.5 Granite 3.3 8B Instruct
  128. 64.9 Gemma 3n E4B
  129. 64.8 LFM2-8B-A1B
  130. 64.3 Gemma 7B
  131. 64.1 StarCoder 2 15B
  132. 63.4 LLaMA-65B
  133. 63.3 Falcon-Arabic-7B
  134. 63.2 Yi 6B
  135. 62.6 Llama 2-34B
  136. 62.5 Mistral 7B v0.1
  137. 62.5 Mistral 7B v0.2
  138. 61.4 ChatGLM3-6B
  139. 60.5 DeepSeek-Coder-V2-Lite-Base
  140. 60.0 Gopher (280B)
  141. 59.4 granite-4.0-1b
  142. 59.2 Baichuan2-13B
  143. 58.7 LLaMA-33B
  144. 58.7 Nemotron-4 15B
  145. 58.4 Falcon 2 11B
  146. 58.1 Gemma 3 4B
  147. 57.4 internlm-chat-20b
  148. 57.0 Qwen-7B
  149. 56.9 Falcon-40B
  150. 56.7 Phi-2
  151. 55.6 Llama 2-13B
  152. 55.2 LFM2-1.2B
  153. 55.1 Baichuan2-13B-Chat
  154. 54.2 Baichuan 2-7B
  155. 53.7 PaLM 62B
  156. 53.6 Qwen2.5-Coder (1.5B)
  157. 53.2 BitNet b1.58 2B4T
  158. 51.6 Baichuan 1-13B
  159. 51.0 internlm-7b
  160. 51.0 LFM2-VL-1.6B
  161. 50.4 North Micro Vision Instruct
  162. 49.9 INTELLECT-1
  163. 47.9 MPT-30B
  164. 47.9 chatglm2-6b
  165. 47.7 LLaMA-13B
  166. 45.7 Llama 2-7B
  167. 43.9 InstructGPT 175B
  168. 43.9 Qwen-1_8B
  169. 43.2 GPT-3 6.7B
  170. 42.3 Baichuan1-7B
  171. 42.3 Gemma 2B
  172. 42.0 Qwen2.5-Coder-0.5B
  173. 40.5 CodeQwen1.5-7B
  174. 40.0 OLMo 2 1B Instruct
  175. 39.4 DeepSeek Coder 33B
  176. 38.8 StarCoder 2 7B
  177. 38.8 Gemma 3 1B
  178. 37.6 Phi-1.5
  179. 36.6 StarCoder 2 3B
  180. 36.4 DeepSeek Coder 6.7B
  181. 36.3 XGen-7B
  182. 35.6 LLaMA-7B
  183. 35.0 Falcon-7B
  184. 30.8 MPT-7B
  185. 29.9 open_llama_7b
  186. 26.3 RedPajama-INCITE-7B-Base
  187. 26.2 Cerebras-GPT-13B
  188. 26.2 Dolly 2.0-12b
  189. 25.8 DeepSeek Coder 1.3B
  190. 25.7 GPT-J-6B
  191. 25.1 OPT-1.3B
  1. 1 Seed1.8 92.3%
  2. 2 o1 91.8%
  3. 3 Gemini 1.5 Pro May 2024 91.7%
  4. 4 Cogito v2 Preview 671B MoE 91.3%
  5. 5 Cogito v1 Preview 70B 91%
  6. 6 o1-preview 90.8%
  7. 7 DeepSeek-R1 90.8%
  8. 8 Qwen3-VL-235B-A22B 90.6%
  9. 9 Sarvam 105B 90.6%
  10. 10 Claude 3.5 Sonnet (October 2024) 90.5%
  11. 11 Claude 3.5 Sonnet 90.4%
  12. 12 LongCat-Flash-Omni 90.3%
  13. 13 DeepSeek-V4-Pro 90.1%
  14. 14 Gemini 1.0 Ultra 90%
  15. 15 gpt-oss-120b 90%
  16. 16 GLM-4.5 90%
  17. 17 LongCat-Flash-Chat 89.7%
  18. 18 Hunyuan-TurboS 89.5%
  19. 19 Kimi K2 Jul 2025 89.5%
  20. 20 InternVL3.5-241B-A28B 89.1%
  21. 21 Nemotron 3 Ultra 550B-A55B Base 89.1%
  22. 22 GPT-4o May 2024 88.7%
  23. 23 Llama 3.1-405B 88.6%
  24. 24 DeepSeek-V3 88.5%
  25. 25 MiniMax-Text-01 88.5%
  26. 26 GLM-5 88.3%
  27. 27 Claude 3 Opus 88.2%
  28. 28 Solar Open 100B 88.2%
  29. 29 Solar Pro 2 88.2%
  30. 30 GPT-4o Nov 2024 88.1%
  31. 31 Qwen3-Coder-Next 87.7%
  32. 32 Hermes 4.3 36B 87.7%
  33. 33 Grok-2 Aug 2024 87.5%
  34. 34 GPT-4.1 mini 87.5%
  35. 35 Kimi k1.5 87.4%
  36. 36 Amazon Nova Premier 87.4%
  37. 37 GLM-4.5-Air 87.4%
  38. 38 seed-oss-36b-instruct 87.4%
  39. 39 Trinity Large Preview 87.2%
  40. 40 Hermes 4 405B 87.2%
  41. 41 sonar 87.1%
  42. 42 Tulu 3 405B 87%
  43. 43 Sarvam-M 87%
  44. 44 o3-mini 86.9%
  45. 45 GPT-4 Turbo Apr 2024 86.7%
  46. 46 ERNIE-4.5-300B-A47B 86.5%
  47. 47 GPT-4 Mar 2023 86.4%
  48. 48 Olmo 3.1 32B Think 86.4%
  49. 49 Llama 3.3 70B 86.3%
  50. 50 Qwen2.5-72B 86.1%
  51. 51 Llama 3.1-70B 86%
  52. 52 Llama 3.3 70B Instruct 86%
  53. 53 Cohere Command A 85.5%
  54. 54 Llama 4 Maverick 85.5%
  55. 55 Olmo 3 32B Think 85.4%
  56. 56 gpt-oss-20b 85.3%
  57. 57 Sarvam 30B 85.1%
  58. 58 Trinity Mini 85%
  59. 59 Phi-4 84.8%
  60. 60 Nemotron 3.5 Super 120B-A12B Base 84.1%
  61. 61 Falcon-H1-34B 84%
  62. 62 Mistral Large 2 Jul 2024 84%
  63. 63 EXAONE Deep 32B 83%
  64. 64 Qwen2-72B 82.3%
  65. 65 Llama 3-70B 82%
  66. 66 GPT-4o mini 82%
  67. 67 Kimi-VL-A3B-Thinking-2506 82%
  68. 68 Gemini 1.5 Pro Feb 2024 81.9%
  69. 69 GLM-4 81.5%
  70. 70 Claude 3 Sonnet 81.5%
  71. 71 Grok-1.5 81.3%
  72. 72 Mistral Large 81.2%
  73. 73 Claude 3.5 Haiku 80.9%
  74. 74 Mistral Small 3 80.7%
  75. 75 Mistral Small 3.1 80.6%
  76. 76 Mistral Small 3.2 80.5%
  77. 77 Apple Foundation Model Server (2025) 80.2%
  78. 78 Granite 4.1 30B 80.2%
  79. 79 GPT-4.1 nano 80.1%
  80. 80 Qwen2.5-14B 79.9%
  81. 81 Llama 4 Scout 79.6%
  82. 82 Qwen2.5-Coder-32B 79.1%
  83. 83 Phi-3.5-MoE 78.9%
  84. 84 Nemotron 3.5 Lightning 30B-A3B Base 78.6%
  85. 85 Gemma 3 27B 78.6%
  86. 86 Nemotron 3 Nano 30B-A3B Base 78.6%
  87. 87 Claude 2 78.5%
  88. 88 Granite 4.0 H Small 78.4%
  89. 89 DeepSeek-V2 (MoE-236B, May 2024) 78.4%
  90. 90 phi-3-medium 14B 78.2%
  91. 91 Code World Model (CWM) 77.7%
  92. 92 Qwen1.5-72B 77.5%
  93. 93 OLMo 2 32B Instruct 77.3%
  94. 94 Claude 1.3 77%
  95. 95 Claude 3 Haiku 76.7%
  96. 96 InternLM3-8B-Instruct 76.6%
  97. 97 Yi-34B 76.3%
  98. 98 Gemma 2 27B 76.2%
  99. 99 phi-3-small 7.4B 75.7%
  100. 100 Qwen2.5-Coder-14B 75.2%
  101. 101 Claude Instant 73.4%
  102. 102 Grok-1 73%
  103. 103 Llama 3.1-8B 73%
  104. 104 Inflection-1 72.7%
  105. 105 Mixtral 8x7B 72.5%
  106. 106 Gemma 2 9B 72.3%
  107. 107 Gemma 3 12B 71.9%
  108. 108 GPT-3.5 Turbo Nov 2023 71.4%
  109. 109 DeepSeek LLM 67B 71.1%
  110. 110 Falcon-180B 70.6%
  111. 111 davinci-002 70%
  112. 112 Moonlight-16B-A3B 70%
  113. 113 Llama 2-70B 69.9%
  114. 114 Apertus 70B 69.6%
  115. 115 PaLM (540B) 69.3%
  116. 116 Phi-3.5-mini 69%
  117. 117 phi-3-mini 3.8B 68.8%
  118. 118 Stable Beluga 2 68.6%
  119. 119 Yi-9B 68.4%
  120. 120 Llama 3-8B 68.4%
  121. 121 Qwen2.5-Coder (7B) 68%
  122. 122 Chinchilla 67.5%
  123. 123 Phi-4 Mini 67.3%
  124. 124 Granite 3.2 8B Instruct 66.8%
  125. 125 Qwen-14B 66.5%
  126. 126 AFM-4.5B 65.9%
  127. 127 Granite 3.3 8B Instruct 65.5%
  128. 128 Gemma 3n E4B 64.9%
  129. 129 LFM2-8B-A1B 64.8%
  130. 130 Gemma 7B 64.3%
  131. 131 StarCoder 2 15B 64.1%
  132. 132 LLaMA-65B 63.4%
  133. 133 Falcon-Arabic-7B 63.3%
  134. 134 Yi 6B 63.2%
  135. 135 Llama 2-34B 62.6%
  136. 136 Mistral 7B v0.1 62.5%
  137. 137 Mistral 7B v0.2 62.5%
  138. 138 ChatGLM3-6B 61.4%
  139. 139 DeepSeek-Coder-V2-Lite-Base 60.5%
  140. 140 Gopher (280B) few-shot 60%
  141. 141 granite-4.0-1b 59.4%
  142. 142 Baichuan2-13B 59.2%
  143. 143 LLaMA-33B 58.7%
  144. 144 Nemotron-4 15B 58.7%
  145. 145 Falcon 2 11B 58.4%
  146. 146 Gemma 3 4B 58.1%
  147. 147 internlm-chat-20b 57.4%
  148. 148 Qwen-7B 57%
  149. 149 Falcon-40B 56.9%
  150. 150 Phi-2 56.7%
  151. 151 Llama 2-13B 55.6%
  152. 152 LFM2-1.2B 55.2%
  153. 153 Baichuan2-13B-Chat 55.1%
  154. 154 Baichuan 2-7B 54.2%
  155. 155 PaLM 62B 53.7%
  156. 156 Qwen2.5-Coder (1.5B) 53.6%
  157. 157 BitNet b1.58 2B4T 53.2%
  158. 158 Baichuan 1-13B 51.6%
  159. 159 internlm-7b 51%
  160. 160 LFM2-VL-1.6B 51%
  161. 161 North Micro Vision Instruct 50.4%
  162. 162 INTELLECT-1 49.9%
  163. 163 MPT-30B 47.9%
  164. 164 chatglm2-6b 47.9%
  165. 165 LLaMA-13B 47.7%
  166. 166 Llama 2-7B 45.7%
  167. 167 InstructGPT 175B 43.9%
  168. 168 Qwen-1_8B 43.9%
  169. 169 GPT-3 6.7B fine-tuned 43.2%
  170. 170 Baichuan1-7B 42.3%
  171. 171 Gemma 2B 42.3%
  172. 172 Qwen2.5-Coder-0.5B 42%
  173. 173 CodeQwen1.5-7B 40.5%
  174. 174 OLMo 2 1B Instruct 40%
  175. 175 DeepSeek Coder 33B 39.4%
  176. 176 StarCoder 2 7B 38.8%
  177. 177 Gemma 3 1B 38.8%
  178. 178 Phi-1.5 37.6%
  179. 179 StarCoder 2 3B 36.6%
  180. 180 DeepSeek Coder 6.7B 36.4%
  181. 181 XGen-7B 36.3%
  182. 182 LLaMA-7B 35.6%
  183. 183 Falcon-7B 35%
  184. 184 MPT-7B 30.8%
  185. 185 open_llama_7b 29.9%
  186. 186 RedPajama-INCITE-7B-Base 26.3%
  187. 187 Cerebras-GPT-13B 26.2%
  188. 188 Dolly 2.0-12b 26.2%
  189. 189 DeepSeek Coder 1.3B 25.8%
  190. 190 GPT-J-6B 25.7%
  191. 191 OPT-1.3B 25.1%

Sources: Labs’ model cards, Another board, via Epoch AI (CC BY 4.0) and official MMLU leaderboard

#ModelScoreSettingDateSource
1 Seed1.8 92.3% Dec 2025 Lab model card
2 o1 91.8% Dec 2024 Lab model card
3 Gemini 1.5 Pro May 2024 91.7% May 2024 Lab model card
4 Cogito v2 Preview 671B MoE 91.3% Jul 2025 Lab model card
5 Cogito v1 Preview 70B 91% Apr 2025 Lab model card
6 o1-preview 90.8% Sep 2024 Lab model card
7 DeepSeek-R1 90.8% Jan 2025 Lab model card
8 Qwen3-VL-235B-A22B 90.6% Sep 2025 Lab model card
9 Sarvam 105B 90.6% Mar 2026 Lab model card
10 Claude 3.5 Sonnet (October 2024) 90.5% Oct 2024 Lab model card
11 Claude 3.5 Sonnet 90.4% Jun 2024 Lab model card
12 LongCat-Flash-Omni 90.3% Oct 2025 Lab model card
13 DeepSeek-V4-Pro 90.1% Apr 2026 Lab model card
14 Gemini 1.0 Ultra 90% Dec 2023 Lab model card
15 gpt-oss-120b 90% Aug 2025 Lab model card
16 GLM-4.5 90% Aug 2025 Lab model card
17 LongCat-Flash-Chat 89.7% Aug 2025 Lab model card
18 Hunyuan-TurboS 89.5% Feb 2025 Lab model card
19 Kimi K2 Jul 2025 89.5% Jul 2025 Lab model card
20 InternVL3.5-241B-A28B 89.1% Aug 2025 Lab model card
21 Nemotron 3 Ultra 550B-A55B Base 89.1% Jun 2026 Lab model card
22 GPT-4o May 2024 88.7% May 2024 Lab model card
23 Llama 3.1-405B 88.6% Jul 2024 Lab model card
24 DeepSeek-V3 88.5% Dec 2024 Lab model card
25 MiniMax-Text-01 88.5% Jan 2025 Lab model card
26 GLM-5 88.3% Feb 2026 Lab model card
27 Claude 3 Opus 88.2% Mar 2024 Lab model card
28 Solar Open 100B 88.2% Dec 2025 Lab model card
29 Solar Pro 2 88.2% Jul 2025 Lab model card
30 Qwen3-Coder-Next 87.7% Feb 2026 Lab model card
31 Hermes 4.3 36B 87.7% Dec 2025 Lab model card
32 Grok-2 Aug 2024 87.5% Aug 2024 Lab model card
33 GPT-4.1 mini 87.5% Apr 2025 Lab model card
34 Kimi k1.5 87.4% Jan 2025 Lab model card
35 Amazon Nova Premier 87.4% Apr 2025 Lab model card
36 GLM-4.5-Air 87.4% Aug 2025 Lab model card
37 seed-oss-36b-instruct 87.4% Aug 2025 Lab model card
38 Trinity Large Preview 87.2% Feb 2026 Lab model card
39 Hermes 4 405B 87.2% Aug 2025 Lab model card
40 sonar 87.1% Feb 2025 Lab model card
41 Tulu 3 405B 87% Jan 2025 Lab model card
42 Sarvam-M 87% May 2025 Lab model card
43 o3-mini 86.9% Jan 2025 Lab model card
44 GPT-4 Turbo Apr 2024 86.7% Apr 2024 Lab model card
45 ERNIE-4.5-300B-A47B 86.5% Jun 2025 Lab model card
46 GPT-4 Mar 2023 86.4% Mar 2023 Lab model card
47 Olmo 3.1 32B Think 86.4% Dec 2025 Lab model card
48 Qwen2.5-72B 86.1% Sep 2024 Lab model card
49 Llama 3.1-70B 86% Jul 2024 Lab model card
50 Llama 3.3 70B Instruct 86% Dec 2024 Lab model card
51 Cohere Command A 85.5% Apr 2025 Lab model card
52 Llama 4 Maverick 85.5% Apr 2025 Lab model card
53 Olmo 3 32B Think 85.4% Nov 2025 Lab model card
54 gpt-oss-20b 85.3% Aug 2025 Lab model card
55 Sarvam 30B 85.1% Mar 2026 Lab model card
56 Trinity Mini 85% Dec 2025 Lab model card
57 Phi-4 84.8% Dec 2024 Lab model card
58 Nemotron 3.5 Super 120B-A12B Base 84.1% Oct 2026 Lab model card
59 Falcon-H1-34B 84% May 2025 Lab model card
60 Mistral Large 2 Jul 2024 84% Jul 2024 Lab model card
61 EXAONE Deep 32B 83% Mar 2025 Lab model card
62 Qwen2-72B 82.3% Jun 2024 Lab model card
63 Llama 3-70B 82% Apr 2024 Lab model card
64 GPT-4o mini 82% Jul 2024 Lab model card
65 Kimi-VL-A3B-Thinking-2506 82% Jun 2025 Lab model card
66 Gemini 1.5 Pro Feb 2024 81.9% Feb 2024 Lab model card
67 GLM-4 81.5% Jan 2024 Lab model card
68 Claude 3 Sonnet 81.5% Mar 2024 Lab model card
69 Grok-1.5 81.3% Mar 2024 Lab model card
70 Mistral Large 81.2% Feb 2024 Lab model card
71 Claude 3.5 Haiku 80.9% Oct 2024 Lab model card
72 Mistral Small 3 80.7% Jan 2025 Lab model card
73 Mistral Small 3.1 80.6% Mar 2025 Lab model card
74 Mistral Small 3.2 80.5% Jun 2025 Lab model card
75 Apple Foundation Model Server (2025) 80.2% Jul 2025 Lab model card
76 Granite 4.1 30B 80.2% Apr 2026 Lab model card
77 GPT-4.1 nano 80.1% Apr 2025 Lab model card
78 Llama 4 Scout 79.6% Apr 2025 Lab model card
79 Phi-3.5-MoE 78.9% Aug 2024 Lab model card
80 Nemotron 3.5 Lightning 30B-A3B Base 78.6% Oct 2026 Lab model card
81 Gemma 3 27B 78.6% Mar 2025 Lab model card
82 Nemotron 3 Nano 30B-A3B Base 78.6% Dec 2025 Lab model card
83 Claude 2 78.5% Jul 2023 Lab model card
84 Granite 4.0 H Small 78.4% Oct 2025 Lab model card
85 phi-3-medium 14B 78.2% Apr 2024 Lab model card
86 DeepSeek-V2 (MoE-236B, May 2024) 77.8% May 2024 Lab model card
87 Code World Model (CWM) 77.7% Sep 2025 Lab model card
88 Qwen1.5-72B 77.5% Feb 2024 Lab model card
89 OLMo 2 32B Instruct 77.3% Mar 2025 Lab model card
90 Claude 3 Haiku 76.7% Mar 2024 Lab model card
91 InternLM3-8B-Instruct 76.6% Jan 2025 Lab model card
92 Gemma 2 27B 76.2% Jul 2024 Lab model card
93 phi-3-small 7.4B 75.7% Aug 2024 Lab model card
94 Grok-1 73% Nov 2023 Lab model card
95 Llama 3.1-8B 73% Jul 2024 Lab model card
96 Mixtral 8x7B 72.5% Dec 2023 Lab model card
97 Gemma 2 9B 72.3% Jul 2024 Lab model card
98 Gemma 3 12B 71.9% Mar 2025 Lab model card
99 DeepSeek LLM 67B 71.1% Nov 2023 Lab model card
100 Moonlight-16B-A3B 70% Feb 2025 Lab model card
101 Apertus 70B 69.6% Sep 2025 Lab model card
102 Phi-3.5-mini 69% Aug 2024 Lab model card
103 Llama 2-70B 68.9% Jul 2023 Lab model card
104 phi-3-mini 3.8B 68.8% Aug 2024 Lab model card
105 Llama 3-8B 68.4% Apr 2024 Lab model card
106 Phi-4 Mini 67.3% Feb 2025 Lab model card
107 Granite 3.2 8B Instruct 66.8% Feb 2025 Lab model card
108 Qwen-14B 66.5% Sep 2023 Lab model card
109 AFM-4.5B 65.9% Jul 2025 Lab model card
110 Granite 3.3 8B Instruct 65.5% Apr 2025 Lab model card
111 Gemma 3n E4B 64.9% Jun 2025 Lab model card
112 LFM2-8B-A1B 64.8% Oct 2025 Lab model card
113 Gemma 7B 64.3% Feb 2024 Lab model card
114 Falcon-Arabic-7B 63.3% Nov 2025 Lab model card
115 Llama 2-34B 62.6% Jul 2023 Lab model card
116 ChatGLM3-6B 61.4% Oct 2023 Lab model card
117 Mistral 7B v0.1 60.1% Sep 2023 Lab model card
118 granite-4.0-1b 59.4% Oct 2025 Lab model card
119 Gemma 3 4B 58.1% Mar 2025 Lab model card
120 Qwen-7B 57% Sep 2023 Lab model card
121 Phi-2 56.7% Dec 2023 Lab model card
122 LFM2-1.2B 55.2% Jul 2025 Lab model card
123 Llama 2-13B 54.8% Jul 2023 Lab model card
124 BitNet b1.58 2B4T 53.2% Apr 2025 Lab model card
125 LFM2-VL-1.6B 51% Aug 2025 Lab model card
126 North Micro Vision Instruct 50.4% Aug 2026 Lab model card
127 Llama 2-7B 45.3% Jul 2023 Lab model card
128 Qwen-1_8B 43.9% Sep 2023 Lab model card
129 Gemma 2B 42.3% Mar 2024 Lab model card
130 OLMo 2 1B Instruct 40% Apr 2025 Lab model card
131 Gemma 3 1B 38.8% Mar 2025 Lab model card
#ModelScoreSettingDateSource
1 Gopher (280B) 60% few-shot Dec 2021 Official leaderboard
2 GPT-3 6.7B 43.2% fine-tuned Jun 2020 Official leaderboard
#ModelScoreSettingDateSource
1 Seed1.8 92.3% Dec 2025 Lab model card
2 o1 91.8% Dec 2024 Lab model card
3 Gemini 1.5 Pro May 2024 91.7% May 2024 Lab model card
4 Cogito v2 Preview 671B MoE 91.3% Jul 2025 Lab model card
5 Cogito v1 Preview 70B 91% Apr 2025 Lab model card
6 o1-preview 90.8% Sep 2024 Lab model card
7 DeepSeek-R1 90.8% Jan 2025 Lab model card
8 Qwen3-VL-235B-A22B 90.6% Sep 2025 Lab model card
9 Sarvam 105B 90.6% Mar 2026 Lab model card
10 Claude 3.5 Sonnet (October 2024) 90.5% Oct 2024 Lab model card
11 Claude 3.5 Sonnet 90.4% Jun 2024 Lab model card
12 LongCat-Flash-Omni 90.3% Oct 2025 Lab model card
13 DeepSeek-V4-Pro 90.1% Apr 2026 Lab model card
14 Gemini 1.0 Ultra 90% Dec 2023 Lab model card
15 gpt-oss-120b 90% Aug 2025 Lab model card
16 GLM-4.5 90% Aug 2025 Lab model card
17 LongCat-Flash-Chat 89.7% Aug 2025 Lab model card
18 Hunyuan-TurboS 89.5% Feb 2025 Lab model card
19 Kimi K2 Jul 2025 89.5% Jul 2025 Lab model card
20 InternVL3.5-241B-A28B 89.1% Aug 2025 Lab model card
21 Nemotron 3 Ultra 550B-A55B Base 89.1% Jun 2026 Lab model card
22 GPT-4o May 2024 88.7% May 2024 Lab model card
23 Llama 3.1-405B 88.6% Jul 2024 Lab model card
24 DeepSeek-V3 88.5% Dec 2024 Lab model card
25 MiniMax-Text-01 88.5% Jan 2025 Lab model card
26 GLM-5 88.3% Feb 2026 Lab model card
27 Claude 3 Opus 88.2% Mar 2024 Lab model card
28 Solar Open 100B 88.2% Dec 2025 Lab model card
29 Solar Pro 2 88.2% Jul 2025 Lab model card
30 GPT-4o Nov 2024 88.1% Nov 2024 Claim in Phi-4 Technical Report, via Epoch AI
31 Qwen3-Coder-Next 87.7% Feb 2026 Lab model card
32 Hermes 4.3 36B 87.7% Dec 2025 Lab model card
33 Grok-2 Aug 2024 87.5% Aug 2024 Lab model card
34 GPT-4.1 mini 87.5% Apr 2025 Lab model card
35 Kimi k1.5 87.4% Jan 2025 Lab model card
36 Amazon Nova Premier 87.4% Apr 2025 Lab model card
37 GLM-4.5-Air 87.4% Aug 2025 Lab model card
38 seed-oss-36b-instruct 87.4% Aug 2025 Lab model card
39 Trinity Large Preview 87.2% Feb 2026 Lab model card
40 Hermes 4 405B 87.2% Aug 2025 Lab model card
41 sonar 87.1% Feb 2025 Lab model card
42 Tulu 3 405B 87% Jan 2025 Lab model card
43 Sarvam-M 87% May 2025 Lab model card
44 o3-mini 86.9% Jan 2025 Lab model card
45 GPT-4 Turbo Apr 2024 86.7% Apr 2024 Lab model card
46 ERNIE-4.5-300B-A47B 86.5% Jun 2025 Lab model card
47 GPT-4 Mar 2023 86.4% Mar 2023 Lab model card
48 Olmo 3.1 32B Think 86.4% Dec 2025 Lab model card
49 Llama 3.3 70B 86.3% Dec 2024 Claim in Phi-4 Technical Report, via Epoch AI
50 Qwen2.5-72B 86.1% Sep 2024 Lab model card
51 Llama 3.1-70B 86% Jul 2024 Lab model card
52 Llama 3.3 70B Instruct 86% Dec 2024 Lab model card
53 Cohere Command A 85.5% Apr 2025 Lab model card
54 Llama 4 Maverick 85.5% Apr 2025 Lab model card
55 Olmo 3 32B Think 85.4% Nov 2025 Lab model card
56 gpt-oss-20b 85.3% Aug 2025 Lab model card
57 Sarvam 30B 85.1% Mar 2026 Lab model card
58 Trinity Mini 85% Dec 2025 Lab model card
59 Phi-4 84.8% Dec 2024 Lab model card
60 Nemotron 3.5 Super 120B-A12B Base 84.1% Oct 2026 Lab model card
61 Falcon-H1-34B 84% May 2025 Lab model card
62 Mistral Large 2 Jul 2024 84% Jul 2024 Lab model card
63 EXAONE Deep 32B 83% Mar 2025 Lab model card
64 Qwen2-72B 82.3% Jun 2024 Lab model card
65 Llama 3-70B 82% Apr 2024 Lab model card
66 GPT-4o mini 82% Jul 2024 Lab model card
67 Kimi-VL-A3B-Thinking-2506 82% Jun 2025 Lab model card
68 Gemini 1.5 Pro Feb 2024 81.9% Feb 2024 Lab model card
69 GLM-4 81.5% Jan 2024 Lab model card
70 Claude 3 Sonnet 81.5% Mar 2024 Lab model card
71 Grok-1.5 81.3% Mar 2024 Lab model card
72 Mistral Large 81.2% Feb 2024 Lab model card
73 Claude 3.5 Haiku 80.9% Oct 2024 Lab model card
74 Mistral Small 3 80.7% Jan 2025 Lab model card
75 Mistral Small 3.1 80.6% Mar 2025 Lab model card
76 Mistral Small 3.2 80.5% Jun 2025 Lab model card
77 Apple Foundation Model Server (2025) 80.2% Jul 2025 Lab model card
78 Granite 4.1 30B 80.2% Apr 2026 Lab model card
79 GPT-4.1 nano 80.1% Apr 2025 Lab model card
80 Qwen2.5-14B 79.9% Sep 2024 Claim in Phi-4 Technical Report, via Epoch AI
81 Llama 4 Scout 79.6% Apr 2025 Lab model card
82 Qwen2.5-Coder-32B 79.1% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
83 Phi-3.5-MoE 78.9% Aug 2024 Lab model card
84 Nemotron 3.5 Lightning 30B-A3B Base 78.6% Oct 2026 Lab model card
85 Gemma 3 27B 78.6% Mar 2025 Lab model card
86 Nemotron 3 Nano 30B-A3B Base 78.6% Dec 2025 Lab model card
87 Claude 2 78.5% Jul 2023 Lab model card
88 Granite 4.0 H Small 78.4% Oct 2025 Lab model card
89 DeepSeek-V2 (MoE-236B, May 2024) 78.4% May 2024 Claim in DeepSeek-V3 Technical Report, via Epoch AI
90 phi-3-medium 14B 78.2% Apr 2024 Lab model card
91 Code World Model (CWM) 77.7% Sep 2025 Lab model card
92 Qwen1.5-72B 77.5% Feb 2024 Lab model card
93 OLMo 2 32B Instruct 77.3% Mar 2025 Lab model card
94 Claude 1.3 77% Apr 2023 Claim in Model Card and Evaluations for Claude Models, via Epoch AI
95 Claude 3 Haiku 76.7% Mar 2024 Lab model card
96 InternLM3-8B-Instruct 76.6% Jan 2025 Lab model card
97 Yi-34B 76.3% Nov 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
98 Gemma 2 27B 76.2% Jul 2024 Lab model card
99 phi-3-small 7.4B 75.7% Aug 2024 Lab model card
100 Qwen2.5-Coder-14B 75.2% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
101 Claude Instant 73.4% Aug 2023 Claim in Model Card and Evaluations for Claude Models, via Epoch AI
102 Grok-1 73% Nov 2023 Lab model card
103 Llama 3.1-8B 73% Jul 2024 Lab model card
104 Inflection-1 72.7% Jun 2023 Claim in Inflection-1, via Epoch AI
105 Mixtral 8x7B 72.5% Dec 2023 Lab model card
106 Gemma 2 9B 72.3% Jul 2024 Lab model card
107 Gemma 3 12B 71.9% Mar 2025 Lab model card
108 GPT-3.5 Turbo Nov 2023 71.4% Nov 2023 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
109 DeepSeek LLM 67B 71.1% Nov 2023 Lab model card
110 Falcon-180B 70.6% Sep 2023 Claim in The Falcon Series of Open Language Models, via Epoch AI
111 davinci-002 70% Mar 2022 Claim in GPT-4 Technical Report, via Epoch AI
112 Moonlight-16B-A3B 70% Feb 2025 Lab model card
113 Llama 2-70B 69.9% Jul 2023 Claim in Mixtral of Experts, via Epoch AI
114 Apertus 70B 69.6% Sep 2025 Lab model card
115 PaLM (540B) 69.3% Apr 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
116 Phi-3.5-mini 69% Aug 2024 Lab model card
117 phi-3-mini 3.8B 68.8% Aug 2024 Lab model card
118 Stable Beluga 2 68.6% Jul 2023 Claim in Qwen Technical Report, via Epoch AI
119 Yi-9B 68.4% Mar 2024 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
120 Llama 3-8B 68.4% Apr 2024 Lab model card
121 Qwen2.5-Coder (7B) 68% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
122 Chinchilla 67.5% Mar 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
123 Phi-4 Mini 67.3% Feb 2025 Lab model card
124 Granite 3.2 8B Instruct 66.8% Feb 2025 Lab model card
125 Qwen-14B 66.5% Sep 2023 Lab model card
126 AFM-4.5B 65.9% Jul 2025 Lab model card
127 Granite 3.3 8B Instruct 65.5% Apr 2025 Lab model card
128 Gemma 3n E4B 64.9% Jun 2025 Lab model card
129 LFM2-8B-A1B 64.8% Oct 2025 Lab model card
130 Gemma 7B 64.3% Feb 2024 Lab model card
131 StarCoder 2 15B 64.1% Feb 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
132 LLaMA-65B 63.4% Feb 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
133 Falcon-Arabic-7B 63.3% Nov 2025 Lab model card
134 Yi 6B 63.2% Nov 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
135 Llama 2-34B 62.6% Jul 2023 Lab model card
136 Mistral 7B v0.1 62.5% Sep 2023 Claim in Mixtral of Experts, via Epoch AI
137 Mistral 7B v0.2 62.5% Dec 2023 Claim in Gemma: Open Models Based on Gemini Research and Technology, via Epoch AI
138 ChatGLM3-6B 61.4% Oct 2023 Lab model card
139 DeepSeek-Coder-V2-Lite-Base 60.5% Jun 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
140 Gopher (280B) 60% few-shot Dec 2021 Official leaderboard
141 granite-4.0-1b 59.4% Oct 2025 Lab model card
142 Baichuan2-13B 59.2% Sep 2023 Claim in Nemotron-4 15B Technical Report, via Epoch AI
143 LLaMA-33B 58.7% Feb 2023 Claim in Qwen Technical Report, via Epoch AI
144 Nemotron-4 15B 58.7% Feb 2024 Claim in Nemotron-4 15B Technical Report, via Epoch AI
145 Falcon 2 11B 58.4% May 2024 Claim in Falcon2-11B Technical Report, via Epoch AI
146 Gemma 3 4B 58.1% Mar 2025 Lab model card
147 internlm-chat-20b 57.4% Sep 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
148 Qwen-7B 57% Sep 2023 Lab model card
149 Falcon-40B 56.9% Mar 2023 Claim in Falcon2-11B Technical ReportFalcon2-11B Technical Report, via Epoch AI
150 Phi-2 56.7% Dec 2023 Lab model card
151 Llama 2-13B 55.6% Jul 2023 Claim in Mixtral of Experts, via Epoch AI
152 LFM2-1.2B 55.2% Jul 2025 Lab model card
153 Baichuan2-13B-Chat 55.1% Sep 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
154 Baichuan 2-7B 54.2% Sep 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
155 PaLM 62B 53.7% Apr 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
156 Qwen2.5-Coder (1.5B) 53.6% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
157 BitNet b1.58 2B4T 53.2% Apr 2025 Lab model card
158 Baichuan 1-13B 51.6% Jul 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
159 internlm-7b 51% Jul 2023 Claim in Qwen Technical Report, via Epoch AI
160 LFM2-VL-1.6B 51% Aug 2025 Lab model card
161 North Micro Vision Instruct 50.4% Aug 2026 Lab model card
162 INTELLECT-1 49.9% Nov 2024 Claim in INTELLECT-1 Technical Report, via Epoch AI
163 MPT-30B 47.9% Jun 2023 Claim in Qwen Technical Report, via Epoch AI
164 chatglm2-6b 47.9% Jun 2023 Claim in Qwen Technical Report, via Epoch AI
165 LLaMA-13B 47.7% Feb 2023 Claim in Qwen Technical Report, via Epoch AI
166 Llama 2-7B 45.7% Jul 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
167 InstructGPT 175B 43.9% Jan 2022 Claim in Scaling Language Models: Methods, Analysis & Insights fro…, via Epoch AI
168 Qwen-1_8B 43.9% Sep 2023 Lab model card
169 GPT-3 6.7B 43.2% fine-tuned Jun 2020 Official leaderboard
170 Baichuan1-7B 42.3% Jun 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
171 Gemma 2B 42.3% Mar 2024 Lab model card
172 Qwen2.5-Coder-0.5B 42% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
173 CodeQwen1.5-7B 40.5% Apr 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
174 OLMo 2 1B Instruct 40% Apr 2025 Lab model card
175 DeepSeek Coder 33B 39.4% Nov 2023 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
176 StarCoder 2 7B 38.8% Feb 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
177 Gemma 3 1B 38.8% Mar 2025 Lab model card
178 Phi-1.5 37.6% Sep 2023 Claim in Textbooks Are All You Need II: phi-1.5 technical report, via Epoch AI
179 StarCoder 2 3B 36.6% Feb 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
180 DeepSeek Coder 6.7B 36.4% Nov 2023 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
181 XGen-7B 36.3% Jun 2023 Claim in XGen-7B Technical Report, via Epoch AI
182 LLaMA-7B 35.6% Feb 2023 Claim in Qwen Technical Report, via Epoch AI
183 Falcon-7B 35% Apr 2023 Claim in Falcon2-11B Technical Report, via Epoch AI
184 MPT-7B 30.8% May 2023 Claim in Qwen Technical Report, via Epoch AI
185 open_llama_7b 29.9% Jun 2023 Claim in XGen-7B Technical Report, via Epoch AI
186 RedPajama-INCITE-7B-Base 26.3% May 2023 Claim in XGen-7B Technical Report, via Epoch AI
187 Cerebras-GPT-13B 26.2% Mar 2023 Claim in XGen-7B Technical Report, via Epoch AI
188 Dolly 2.0-12b 26.2% Apr 2023 Claim in XGen-7B Technical Report, via Epoch AI
189 DeepSeek Coder 1.3B 25.8% Nov 2023 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
190 GPT-J-6B 25.7% Aug 2021 Claim in XGen-7B Technical Report, via Epoch AI
191 OPT-1.3B 25.1% May 2022 Claim in XGen-7B Technical Report, via Epoch AI

The benchmark's own leaderboard has 10 entries, 2 of them models we track. Epoch AI collected claims for 85 models from labs’ reports and papers. Switch the scores above to rank them together.

5 models have held number one in labs’ own numbers since 2023 Number one over time 6 models have held number one with claims included since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Google DeepMind ByteDance Seed
Lab claims Passed 85% in Mar 2023, by GPT-4
Seed1.8 92.3%

Source: Labs’ model cards

OpenAI’s own number for GPT-4 passed 85% in March 2023

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card A claim Epoch AI collected from a report or paper Best so far
0% 20% 40% 60% 80% 100% 2020 2021 2022 2023 2024 2025 2026 Near the ceiling (85%) Random guessing Seed1.8 Gemini 1.5 Pro GPT-4 davinci-002 Gopher (280B) GPT-3 6.7B 0% 20% 40% 60% 80% 100% 2020 2022 2024 2026 Near the ceiling (85%) Random guessing Seed1.8 davinci-002

Sources: Official MMLU leaderboard, labs’ model cards and claims collected by Epoch AI (CC BY 4.0)

Labs headline it less often now, last in August 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

OpenAI
Anthropic
Meta
Qwen
Mistral AI
Z.ai
xAI
DeepSeek
Google DeepMind
Microsoft
Shanghai AI Lab
MiniMax
Moonshot AI
Ai2
Perplexity
IBM
Tencent
Cohere
Deep Cogito
Amazon
Technology Innovation Institute
Sarvam AI
Baidu
Upstage
Liquid AI
Arcee AI
ByteDance Seed
Nous Research
Meituan
DateLaunch postScoreAs the post writes it
Aug 12, 2026 North Micro Vision Instruct 50.4 MMLU test
Apr 29, 2026 Granite 4.1 30B 80.2 MMLU (5-shot)
Apr 24, 2026 DeepSeek-V4-Pro 90.1 MMLU (EM)
Mar 6, 2026 Sarvam 30B 85.1 MMLU
Mar 6, 2026 Sarvam 105B 90.6 MMLU
Jan 27, 2026 Trinity Large Preview 87.2 MMLU
Dec 31, 2025 Solar Open 100B 88.2 MMLU
Dec 18, 2025 Seed1.8 92.3 MMLU
Dec 12, 2025 Olmo 3.1 32B Think 86.4 MMLU
Dec 3, 2025 Hermes 4.3 36B 87.7 MMLU
Dec 1, 2025 Trinity Mini 85 MMLU (Zero Shot)
Nov 20, 2025 Olmo 3 32B Think 85.4 MMLU
Oct 31, 2025 LongCat-Flash-Omni 90.3 MMLU(acc)
Oct 28, 2025 granite-4.0-1b 59.4 MMLU (5-shot)
Oct 7, 2025 LFM2-8B-A1B 64.8 MMLU
Oct 2, 2025 Granite 4.0 H Small 78.4 MMLU (5-shot)
Sep 23, 2025 Qwen3-VL-235B-A22B 90.6 MMLU
Aug 29, 2025 LongCat-Flash-Chat 89.7 MMLU(acc)
Aug 26, 2025 Hermes 4 405B 87.2 MMLU (reasoning)
Aug 20, 2025 seed-oss-36b-instruct 87.4 MMLU
Aug 12, 2025 LFM2-VL-1.6B 51 Text MMLU
Aug 5, 2025 gpt-oss-20b 85.3 MMLU
Aug 5, 2025 gpt-oss-120b 90 MMLU Questions across academic disciplines
Jul 31, 2025 Cogito v2 Preview 671B MoE 91.3 MMLU (thinking)
Jul 29, 2025 AFM-4.5B 65.9 MMLU
Jul 11, 2025 Kimi K2 89.5 MMLU, EM
Jul 10, 2025 LFM2-1.2B 55.2 MMLU
Jul 10, 2025 Solar Pro 2 88.2 MMLU (0-shot, CoT)
Jun 30, 2025 ERNIE-4.5-300B-A47B 86.5 MMLU
Jun 26, 2025 Gemma 3n E4B 64.9 MMLU 0-shot (IT)
Jun 21, 2025 Kimi-VL-A3B-Thinking-2506 82 MMLU
Jun 20, 2025 Mistral Small 3.2 80.5 MMLU
May 23, 2025 Sarvam-M 87 MMLU (0.87)
May 20, 2025 Falcon-H1-34B 84 MMLU
Apr 30, 2025 Amazon Nova Premier 87.4 MMLU
Apr 29, 2025 OLMo 2 1B Instruct 40 MMLU
Apr 16, 2025 Granite 3.3 8B Instruct 65.5 MMLU
Apr 15, 2025 BitNet b1.58 2B4T 53.2 MMLU
Apr 14, 2025 GPT-4.1 nano 80.1 MMLU
Apr 14, 2025 GPT-4.1 mini 87.5 MMLU
Apr 8, 2025 Cogito v1 Preview 70B 91 MMLU (thinking)
Apr 5, 2025 Llama 4 Scout 79.6 MMLU (5-shot) [pretrained]
Apr 5, 2025 Llama 4 Maverick 85.5 MMLU (5-shot) [pretrained]
Mar 17, 2025 Mistral Small 3.1 80.6 MMLU
Mar 13, 2025 Cohere Command A MMLU (chart only)
Mar 13, 2025 OLMo 2 32B Instruct 77.3 MMLU
Mar 12, 2025 Gemma 3 27B 78.6 MMLU 5-shot (PT)
Feb 27, 2025 Hunyuan-TurboS 89.5 MMLU
Feb 26, 2025 Phi-4 Mini 67.3 MMLU (5-shot)
Feb 26, 2025 Granite 3.2 8B Instruct 66.8 MMLU
Feb 22, 2025 Moonlight-16B-A3B 70 MMLU
Feb 11, 2025 sonar 87.1 MMLU (0-shot)
Jan 31, 2025 o3-mini 86.9 MMLU (pass@1)
Jan 30, 2025 Tulu 3 405B 87 MMLU (5 shot, CoT)
Jan 30, 2025 Mistral Small 3 80.7 Best of MMLU (5-shot) [base] / French MMLU [base] / German MMLU [base] / Spanish MMLU [base]
Jan 28, 2025 Qwen2.5-Max MMLU
Jan 20, 2025 Kimi k1.5 87.4 MMLU (EM), short-CoT
Jan 20, 2025 DeepSeek-R1 90.8 MMLU (Pass@1)
Jan 15, 2025 MiniMax-Text-01 88.5 MMLU* (0-shot CoT)
Jan 13, 2025 InternLM3-8B-Instruct 76.6 MMLU(0-shot)
Dec 26, 2024 DeepSeek-V3 88.5 MMLU (EM)
Dec 17, 2024 o1 91.8 MMLU (pass @1)
Dec 12, 2024 Phi-4 84.8 MMLU (simple-evals)
Dec 6, 2024 Llama 3.3 70B Instruct 86 MMLU (CoT), 0-shot, macro_avg/acc
Oct 22, 2024 Claude 3.5 Sonnet (October 2024) 90.5 MMLU (5-shot CoT)
Sep 19, 2024 Qwen2.5-72B 86.1 MMLU
Sep 12, 2024 o1-preview 90.8 MMLU pass@1
Aug 20, 2024 Phi-3.5-MoE 78.9 MMLU (5-shot)
Aug 13, 2024 Grok-2 87.5 MMLU (0-shot CoT)
Jul 24, 2024 Mistral Large 2 84 Best of MMLU [base pretrained] / Multilingual MMLU French [base] / Multilingual MMLU German [base] / Multilingual MMLU Spanish [base] / Multilingual MMLU Italian [base]
Jul 23, 2024 Llama 3.1-405B 88.6 MMLU (0-shot, CoT)
Jul 18, 2024 GPT-4o mini 82 MMLU
Jun 27, 2024 Gemma 2 27B 75.2 MMLU (5-shot, top-1)
Jun 20, 2024 Claude 3.5 Sonnet 90.4 MMLU (5-shot CoT)
Jun 7, 2024 Qwen2-72B 82.3 MMLU
May 14, 2024 Gemini 1.5 Pro 91.7 MMLU, maj@32 (pretrained) (5-shot = 85.9%)
May 13, 2024 GPT-4o 88.7 MMLU (0-shot CoT)
May 7, 2024 DeepSeek-V2 (MoE-236B, May 2024) 77.8 MMLU
Apr 22, 2024 phi-3-medium 14B 78.2 MMLU (5-Shot)
Apr 18, 2024 Llama 3-70B 82 MMLU (5-shot) [instruct]
Apr 11, 2024 GPT-4 Turbo 86.7 MMLU%
Mar 28, 2024 Grok-1.5 81.3 MMLU (5-shot)
Mar 4, 2024 Claude 3 Opus 88.2 MMLU (5-shot CoT)
Feb 26, 2024 Mistral Large 81.2 Best of MMLU / French MMLU / German MMLU / Spanish MMLU / Italian MMLU
Feb 21, 2024 Gemma 7B 64.3 MMLU (5-shot, top-1)
Feb 15, 2024 Gemini 1.5 Pro 81.9 MMLU, 5-shot
Feb 5, 2024 Qwen1.5-72B 77.5 MMLU
Jan 16, 2024 GLM-4 81.5 MMLU (5-shot)
Dec 12, 2023 Phi-2 56.7 MMLU, Table 2
Dec 11, 2023 Mixtral 8x7B 72.5 Best of MMLU (MCQ in 57 subjects) / French MMLU / German MMLU / Spanish MMLU / Italian MMLU
Dec 6, 2023 Gemini 1.0 Ultra 90 MMLU, CoT@32 (5-shot = 83.7%)
Nov 29, 2023 DeepSeek LLM 67B 71.1 MMLU
Nov 3, 2023 Grok-1 73 MMLU (5-shot)
Oct 27, 2023 ChatGLM3-6B 61.4 MMLU
Sep 27, 2023 Mistral 7B v0.1 60.1 MMLU
Sep 25, 2023 Qwen-14B 66.5 MMLU (5-shot)
Jul 18, 2023 Llama 2-70B 68.9 MMLU
Jul 11, 2023 Claude 2 78.5 MMLU (5-shot CoT)
Mar 14, 2023 GPT-4 86.4 MMLU (5-shot)

91 system cards report it, from 27 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 15, 2026 Nemotron 3.5 Super 120B-A12B Base technical report 84.1 MMLU, exact match, Table 6 Capabilities
Oct 15, 2026 Nemotron 3.5 Lightning 30B-A3B Base technical report 78.6 MMLU, exact match, Table 6 Capabilities
Jun 4, 2026 Nemotron 3 Ultra 550B-A55B Base model card 89.1 MMLU, 5-shot acc Capabilities
Mar 21, 2026 Seed1.8 model card 92.3 MMLU (Table 1) Capabilities
Feb 28, 2026 Qwen3-Coder-Next technical report 87.7 MMLU (Table 8) Capabilities
Feb 19, 2026 Trinity Large Preview technical report 87.2 MMLU, Trinity Large Preview (Table 4) Capabilities
Feb 17, 2026 GLM-5 technical report 88.3 GLM-5-Base, MMLU (EM) (Table 11) Capabilities
Jan 11, 2026 Solar Open 100B technical report 88.2 MMLU (Table 6) Capabilities
Dec 23, 2025 Nemotron 3 Nano 30B-A3B Base technical report 78.6 MMLU, 5-shot accuracy, base model Table 2 Capabilities
Dec 15, 2025 Olmo 3 32B Think technical report 85.4 MMLU, Final Think 3.0 column (Table 14) Capabilities
Dec 15, 2025 Olmo 3.1 32B Think technical report 86.4 MMLU (Tables 1 and 14) Capabilities
Nov 28, 2025 LFM2-8B-A1B technical report 64.8 MMLU, LFM2-8B-A1B (Table 7) Capabilities
Nov 28, 2025 LFM2-1.2B technical report 55.2 MMLU, LFM2-1.2B (Table 6) Capabilities
Nov 8, 2025 Falcon-Arabic-7B technical report 63.3 MMLU 5-shot (Table 8) Capabilities
Oct 31, 2025 LongCat-Flash-Omni technical report 90.3 MMLU (acc) (Table 13) Capabilities
Oct 8, 2025 OLMo 2 1B Instruct technical report 40 MMLU (Table 7, Table 23) Capabilities
Sep 30, 2025 Code World Model (CWM) technical report 77.7 MMLU, without reasoning (Table 13) Capabilities
Sep 17, 2025 Apertus 70B technical report 69.6 MMLU, 0-shot CoT exact match (Table 17) Capabilities
Sep 1, 2025 LongCat-Flash-Chat technical report 89.7 MMLU (acc), model-judged answer matching (Table 3) Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 89.1 MMLU, OpenCompass (Table 14) Capabilities
Aug 25, 2025 Hermes 4 405B technical report 87.2 MMLU, reasoning mode (Table 3). Non-reasoning 73.6 Capabilities
Aug 8, 2025 GLM-4.5 technical report 90 MMLU (EM) (Table 6) Capabilities
Aug 8, 2025 GLM-4.5-Air technical report 87.4 MMLU (EM) (Table 6) Capabilities
Aug 5, 2025 gpt-oss-20b model card 85.3 MMLU, high reasoning (Table 3) Capabilities
Aug 5, 2025 gpt-oss-120b model card 90 MMLU, high reasoning (Table 3) Capabilities
Jul 30, 2025 Falcon-H1-34B technical report 84 MMLU, logprobs 5-shot (Table 24) Capabilities
Jul 21, 2025 Kimi K2 technical report 89.5 MMLU (EM) (Table 3) Capabilities
Jul 17, 2025 Voxtral Small technical report MMLU 5-shot, text-only, chart without printed numbers (Figure 6). Speech-synthesized MMLU* 74.3 (Table 8) Capabilities
Jul 17, 2025 Apple Foundation Model Server (2025) technical report 80.2 MMLU, Simple-Evals, AFM Server (Table 2). Table 3 prints 80.0 at 16 bits and 79.2 for AFM Server Opt at 3.6 bits per weight Capabilities
Jun 29, 2025 ERNIE-4.5-300B-A47B technical report 86.5 MMLU (Table 5) Capabilities
Apr 30, 2025 Amazon Nova Premier technical report 87.4 MMLU, 0-shot CoT, macro average (Table 1) Capabilities
Apr 16, 2025 BitNet b1.58 2B4T technical report 53.2 MMLU (5-shot; Acc) (Table 1, Table 2, Table 3) Capabilities
Apr 5, 2025 Llama 4 Scout model card 79.6 MMLU, 5-shot, macro_avg/acc_char, pretrained (Pre-trained models table) Capabilities
Apr 5, 2025 Llama 4 Maverick model card 85.5 MMLU, 5-shot, macro_avg/acc_char, pretrained (Pre-trained models table) Capabilities
Apr 1, 2025 Cohere Command A technical report 85.5 MMLU, 5-shot CoT, simple-evals (Table 3, Table 1) Capabilities
Mar 25, 2025 Gemma 3 4B technical report 58.1 MMLU, IT, 0-shot (Table 18) Capabilities
Mar 25, 2025 Gemma 3 27B technical report 76.9 MMLU, IT, 0-shot (Table 18) Capabilities
Mar 25, 2025 Gemma 3 1B technical report 38.8 MMLU, IT, 0-shot (Table 18) Capabilities
Mar 25, 2025 Gemma 3 12B technical report 71.9 MMLU, IT, 0-shot (Table 18) Capabilities
Mar 16, 2025 EXAONE Deep 32B technical report 83 MMLU, accuracy (Table 3) Capabilities
Mar 3, 2025 Phi-4 Mini technical report 67.3 MMLU (5-Shot) (Table 7) Capabilities
Jan 20, 2025 Kimi k1.5 technical report 87.4 MMLU (EM), short-CoT (Table 3) Capabilities
Jan 20, 2025 DeepSeek-R1 technical report 90.8 MMLU (EM), DeepSeek-R1 (Table 8, same as final column of Table 3) Capabilities
Jan 14, 2025 MiniMax-Text-01 technical report 88.5 MMLU, 0-shot CoT (Table 8) Capabilities
Dec 31, 2024 OLMo 2 32B Instruct technical report 77.3 MMLU, 0-shot CoT (Tables 7 and 16) Capabilities
Dec 26, 2024 DeepSeek-V3 technical report 88.5 MMLU (EM), DeepSeek-V3 (Table 6) Capabilities
Dec 12, 2024 Phi-4 technical report 84.8 MMLU, simple-evals (Table 1) Capabilities
Dec 6, 2024 Llama 3.3 70B Instruct model card 86 MMLU (CoT), 0-shot, macro_avg/acc (Instruction tuned models table) Capabilities
Nov 23, 2024 Llama 3.1-8B technical report 73 MMLU (0-shot, CoT), post-trained (Table 2). 5-shot = 69.4 Capabilities
Nov 23, 2024 Llama 3.1-70B technical report 86 MMLU (0-shot, CoT), post-trained (Table 2). 5-shot = 83.6 Capabilities
Nov 23, 2024 Llama 3.1-405B technical report 88.6 MMLU (0-shot, CoT), post-trained (Table 2). 5-shot = 87.3 Capabilities
Nov 22, 2024 Tulu 3 405B technical report 87 MMLU (5 shot, CoT), RLVR (Table 4) Capabilities
Oct 22, 2024 Claude 3.5 Sonnet (October 2024) model card 90.5 MMLU, 5-shot CoT. 5-shot 88.7%, 0-shot CoT 89.3% (Table 7) Capabilities
Oct 22, 2024 Claude 3.5 Haiku model card 80.9 MMLU, 5-shot CoT. 5-shot 77.6%, 0-shot CoT 80.3% (Table 8) Capabilities
Aug 30, 2024 phi-3-small 7.4B technical report 75.7 MMLU (5-Shot) (Section 3 table) Capabilities
Aug 30, 2024 phi-3-mini 3.8B technical report 68.8 MMLU (5-Shot) (Section 3 table) Capabilities
Aug 30, 2024 phi-3-medium 14B technical report 78 MMLU (5-Shot) (Section 3 table) Capabilities
Aug 30, 2024 Phi-3.5-MoE technical report 78.9 MMLU (5-shot) (Table 3) Capabilities
Aug 30, 2024 Phi-3.5-mini technical report 69 MMLU (5-shot) (Table 3) Capabilities
Jul 31, 2024 Gemma 2 9B technical report 72.3 MMLU, IT, 5-shot (Table 17). PT 5-shot (Table 13): 71.3 Capabilities
Jul 31, 2024 Gemma 2 27B technical report 76.2 MMLU, IT, 5-shot (Table 17). PT 5-shot (Table 13): 75.2 Capabilities
Jul 23, 2024 Llama 3.1-8B model card 73 MMLU (CoT), 0-shot, macro_avg/acc, instruction tuned (Instruction tuned models table). 5-shot = 69.4 Capabilities
Jul 23, 2024 Llama 3.1-70B model card 86 MMLU (CoT), 0-shot, macro_avg/acc, instruction tuned (Instruction tuned models table). 5-shot = 83.6 Capabilities
Jul 23, 2024 Llama 3.1-405B model card 88.6 MMLU (CoT), 0-shot, macro_avg/acc, instruction tuned (Instruction tuned models table). 5-shot = 87.3 Capabilities
Jul 15, 2024 Qwen2-72B technical report 82.3 MMLU, Qwen2-72B-Instruct (Table 6). 83.2 on the non-contaminated subset (Table 15) Capabilities
Jun 20, 2024 Claude 3.5 Sonnet model card 90.4 MMLU, 5-shot CoT. 5-shot 88.7, 0-shot CoT 88.3 (Table 1) Capabilities
Jun 18, 2024 GLM-4 technical report 81.5 GLM-4 (0116), MMLU (Table 2) Capabilities
Jun 18, 2024 ChatGLM3-6B technical report 61.4 ChatGLM3-6B-Base, MMLU (Table 1) Capabilities
May 7, 2024 DeepSeek-V2 (MoE-236B, May 2024) technical report 77.8 MMLU 5-shot, DeepSeek-V2 Chat (RL) (Table 3) Capabilities
Apr 18, 2024 Llama 3-8B model card 68.4 MMLU (5-shot), instruction tuned (Instruction tuned models table). Pretrained = 66.6 Capabilities
Apr 18, 2024 Llama 3-70B model card 82 MMLU (5-shot), instruction tuned (Instruction tuned models table). Pretrained = 79.5 Capabilities
Mar 13, 2024 Gemma 7B technical report 64.3 MMLU, PT, 5-shot top-1 (Table 6). HuggingFace H6 run (Table 7): 64.6 Capabilities
Mar 13, 2024 Gemma 2B technical report 42.3 MMLU, PT, 5-shot top-1 (Table 6) Capabilities
Mar 4, 2024 Claude 3 Sonnet model card 81.5 MMLU, 5-shot CoT. 5-shot 79.0 (Table 1) Capabilities
Mar 4, 2024 Claude 3 Opus model card 88.2 MMLU, 5-shot CoT. 5-shot 86.8 (Table 1) Capabilities
Mar 4, 2024 Claude 3 Haiku model card 76.7 MMLU, 5-shot CoT. 5-shot 75.2 (Table 1) Capabilities
Jan 8, 2024 Mixtral 8x7B technical report 70.6 MMLU (MCQ in 57 subjects), 5-shot (Table 3; same in Table 2) Capabilities
Jan 5, 2024 DeepSeek LLM 67B technical report 71.1 MMLU 0-shot, DeepSeek 67B Chat (Table 6) Capabilities
Oct 10, 2023 Mistral 7B v0.1 technical report 60.1 MMLU, 5-shot (Table 2) Capabilities
Sep 28, 2023 Qwen-7B technical report 57 MMLU 5-shot, higher of the two settings, 0-shot 55.8 (Table 5) Capabilities
Sep 28, 2023 Qwen-14B technical report 66.5 MMLU 5-shot, higher of the two settings, 0-shot 64.6 (Table 5) Capabilities
Sep 28, 2023 Qwen-1_8B technical report 43.9 MMLU 5-shot, higher of the two settings, 0-shot 42.4 (Table 5) Capabilities
Jul 19, 2023 Llama 2-7B technical report 45.3 MMLU (5-shot), pretrained (Table 3, Table 19) Capabilities
Jul 19, 2023 Llama 2-70B technical report 68.9 MMLU (5-shot), pretrained (Table 3, Table 19) Capabilities
Jul 19, 2023 Llama 2-34B technical report 62.6 MMLU (5-shot), pretrained (Table 3, Table 19) Capabilities
Jul 19, 2023 Llama 2-13B technical report 54.8 MMLU (5-shot), pretrained (Table 3, Table 19) Capabilities
Jul 18, 2023 Llama 2-7B model card 45.3 MMLU, pretrained (Overall performance on grouped academic benchmarks) Capabilities
Jul 18, 2023 Llama 2-70B model card 68.9 MMLU, pretrained (Overall performance on grouped academic benchmarks) Capabilities
Jul 18, 2023 Llama 2-13B model card 54.8 MMLU, pretrained (Overall performance on grouped academic benchmarks) Capabilities
Jul 11, 2023 Claude 2 model card 78.5 MMLU (5-shot CoT) (4.3) Capabilities
Mar 14, 2023 GPT-4 technical report 86.4 MMLU, 5-shot (Table 2, base model) Capabilities

Versions

Replaced by MMLU-Pro

More in Knowledge

All 121 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • KMMLU-Pro In use Models answer multiple-choice questions taken from Korean National Professional Licensure exams. It tests professional knowledge used in Korean industry. Solar Pro 4 79.2%
  • HLE-Verified In use Verified and revised version of Humanity's Last Exam (1,811 items) with uncertain items removed. Seed2.0 Pro 73.6%