Benchmark Almanac

GSM8K

Llama 3.1-405B 96.8% leads since July 2024
davinci-002 56.8% leads since March 2022
Llama 3.1-405B 96.8% leads since July 2024
Fading 1 known issue

8.5K grade-school math word problems. Final numeric answer scored by exact match.

Saturated.

Made by
OpenAI
Released
October 2021
Kind
Word problems
Level
School
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Strongest model tested
DeepSeek V4.1 Flash #34 on the capability index
Last in a launch post
September 2026, dropped after it saturated
Last model added
October 2026
Scores refreshed
November 12th, 2026

In the authors’ words

State-of-the-art language models can match human performance on many tasks, but they still struggle to robustly perform multi-step mathematical reasoning. To diagnose the failures of current models and support research, we introduce GSM8K, a dataset of 8.5K high quality linguistically diverse grade school math word problems. We find that even the largest transformer models fail to achieve high test performance, despite the conceptual simplicity of this problem distribution. To increase performance, we propose training verifiers to judge the correctness of model completions. At test time, we generate many candidate solutions and select the one ranked highest by the verifier. We demonstrate that verification significantly improves performance on GSM8K, and we provide strong empirical evidence that verification scales more effectively with increased data than a finetuning baseline.

Abstract of Training Verifiers to Solve Math Word Problems (arXiv:2110.14168, CC0)

Known issues

A published audit found a problem in GSM8K. We summarize each finding in our words and link its source. The scores on this page don't include any fixes. See all audits

  1. Contamination Still open

    Scale AI reports that many models score lower on GSM1k, a new set written to mirror GSM8K, than on GSM8K itself. It reads the gap as evidence that some models partially memorized GSM8K, while frontier models show minimal signs of overfitting.

    Accuracy drops by up to 8% on GSM1k. A model's chance of generating GSM8K examples tracks its GSM8K-to-GSM1k gap (Spearman's r² = 0.36).

    Our analysis confirms the widespread suspicion in the field that many models are contaminated by benchmark data, with the worst models performing 8% worse on GSM1k compared to GSM8k.

    A Careful Examination of Large Language Model Performance on Grade School Arithmetic by Scale AI, May 1st, 2024

92
Models scored in lab cards
57
Launch posts list it, from 19 labs
72
System cards report it, from 17 labs

Llama 3.1-405B leads with 96.8%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

davinci-002 leads with 56.8% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Llama 3.1-405B leads with 96.8%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

92 models. Scroll sideways for the rest.

  1. 96.8 Llama 3.1-405B
  2. 96.6 ERNIE-4.5-300B-A47B
  3. 96.4 Cogito v1 Preview 70B
  4. 96.4 Claude 3.5 Sonnet
  5. 96.3 DiffusionGemma 26B A4B
  6. 96.2 Cogito v2 Preview 671B MoE
  7. 95.9 Qwen2.5-32B
  8. 95.9 Gemma 3 27B
  9. 95.8 Qwen2.5-72B
  10. 95.5 Tulu 3 405B
  11. 95.3 Qwen2.5-VL-72B
  12. 95.1 Llama 3.1-70B
  13. 95.0 Claude 3 Opus
  14. 94.8 Qwen2.5-14B
  15. 94.8 MiniMax-Text-01
  16. 94.8 Qwen2.5-14B-Instruct-1M
  17. 94.4 Gemini 1.0 Ultra
  18. 94.4 Gemma 3 12B
  19. 94.2 Granite 4.1 30B
  20. 94.0 Sarvam-M
  21. 93.8 Qwen-Turbo
  22. 93.3 Code World Model (CWM)
  23. 93.2 Qwen2-72B
  24. 93.0 Llama 3-70B
  25. 93.0 Mistral Large 2
  26. 93.0 DeepSeek V4.1 Flash
  27. 93.0 DeepSeek-V4.1-Flash Base
  28. 92.6 DeepSeek-V4-Pro
  29. 92.3 Nemotron 3 Nano 30B-A3B Base
  30. 92.3 Claude 3 Sonnet
  31. 92.2 DeepSeek-V2 (MoE-236B, May 2024)
  32. 92.0 GPT-4
  33. 91.7 Nemotron 3.5 Lightning 30B-A3B Base
  34. 91.7 Nemotron 3.5 Super 120B-A12B Base
  35. 91.7 Gemini 1.5 Pro
  36. 91.6 Qwen2.5-7B
  37. 91.6 InternVL3.5-241B-A28B
  38. 91.2 Mistral Large
  39. 91.0 PaLM 2-L
  40. 91.0 phi-3-medium 14B
  41. 90.8 Gemini 1.5 Pro
  42. 90.0 Grok-1.5
  43. 89.6 phi-3-small 7.4B
  44. 89.3 DeepSeek-V3
  45. 89.2 Gemma 3 4B
  46. 88.9 Claude 3 Haiku
  47. 88.7 Phi-3.5-MoE
  48. 88.7 Qwen2.5-Omni-7B
  49. 88.6 Phi-4 Mini
  50. 88.1 Nemotron 3 Ultra 550B-A55B Base
  51. 88.0 Claude 2
  52. 87.6 GLM-4
  53. 87.6 OLMo 2 32B Instruct
  54. 87.3 Granite 4.0 H Small
  55. 86.2 Phi-3.5-mini
  56. 85.6 LFM2.5-1.2B-Thinking
  57. 84.5 Llama 3.1-8B
  58. 84.4 LFM2-8B-A1B
  59. 84.1 DeepSeek LLM 67B
  60. 83.6 Falcon-H1-34B
  61. 82.5 phi-3-mini 3.8B
  62. 81.7 Granite 3.2 8B Instruct
  63. 80.9 Granite 3.3 8B Instruct
  64. 80.7 Mistral Small 3
  65. 79.6 Llama 3-8B
  66. 79.5 Qwen1.5-72B
  67. 79.4 GLM-4.5
  68. 77.6 Apertus 70B
  69. 77.4 Moonlight-16B-A3B
  70. 76.3 granite-4.0-1b
  71. 74.0 Gemma 2 27B
  72. 72.3 ChatGLM3-6B
  73. 68.8 GLM-5
  74. 68.6 Gemma 2 9B
  75. 68.3 OLMo 2 1B Instruct
  76. 67.7 Falcon-Arabic-7B
  77. 62.9 Grok-1
  78. 62.8 Gemma 3 1B
  79. 61.1 Phi-2
  80. 60.1 Qwen-14B
  81. 58.4 Mixtral 8x7B
  82. 58.4 BitNet b1.58 2B4T
  83. 58.3 LFM2-1.2B
  84. 56.8 Llama 2-70B
  85. 54.1 Qwen-7B
  86. 52.2 Mistral 7B v0.1
  87. 46.4 Gemma 7B
  88. 42.2 Llama 2-34B
  89. 28.7 Llama 2-13B
  90. 27.8 Qwen-1_8B
  91. 17.7 Gemma 2B
  92. 14.6 Llama 2-7B
  1. 1 Llama 3.1-405B 96.8%
  2. 2 ERNIE-4.5-300B-A47B 96.6%
  3. 3 Cogito v1 Preview 70B 96.4%
  4. 4 Claude 3.5 Sonnet 96.4%
  5. 5 DiffusionGemma 26B A4B 96.3%
  6. 6 Cogito v2 Preview 671B MoE 96.2%
  7. 7 Qwen2.5-32B 95.9%
  8. 8 Gemma 3 27B 95.9%
  9. 9 Qwen2.5-72B 95.8%
  10. 10 Tulu 3 405B 95.5%
  11. 11 Qwen2.5-VL-72B 95.3%
  12. 12 Llama 3.1-70B 95.1%
  13. 13 Claude 3 Opus 95%
  14. 14 Qwen2.5-14B 94.8%
  15. 15 MiniMax-Text-01 94.8%
  16. 16 Qwen2.5-14B-Instruct-1M 94.8%
  17. 17 Gemini 1.0 Ultra 94.4%
  18. 18 Gemma 3 12B 94.4%
  19. 19 Granite 4.1 30B 94.2%
  20. 20 Sarvam-M 94%
  21. 21 Qwen-Turbo 93.8%
  22. 22 Code World Model (CWM) 93.3%
  23. 23 Qwen2-72B 93.2%
  24. 24 Llama 3-70B 93%
  25. 25 Mistral Large 2 Jul 2024 93%
  26. 26 DeepSeek V4.1 Flash 93%
  27. 27 DeepSeek-V4.1-Flash Base 93%
  28. 28 DeepSeek-V4-Pro 92.6%
  29. 29 Nemotron 3 Nano 30B-A3B Base 92.3%
  30. 30 Claude 3 Sonnet 92.3%
  31. 31 DeepSeek-V2 (MoE-236B, May 2024) 92.2%
  32. 32 GPT-4 Mar 2023 92%
  33. 33 Nemotron 3.5 Lightning 30B-A3B Base 91.7%
  34. 34 Nemotron 3.5 Super 120B-A12B Base 91.7%
  35. 35 Gemini 1.5 Pro Feb 2024 91.7%
  36. 36 Qwen2.5-7B 91.6%
  37. 37 InternVL3.5-241B-A28B 91.6%
  38. 38 Mistral Large 91.2%
  39. 39 PaLM 2-L 91%
  40. 40 phi-3-medium 14B 91%
  41. 41 Gemini 1.5 Pro May 2024 90.8%
  42. 42 Grok-1.5 90%
  43. 43 phi-3-small 7.4B 89.6%
  44. 44 DeepSeek-V3 89.3%
  45. 45 Gemma 3 4B 89.2%
  46. 46 Claude 3 Haiku 88.9%
  47. 47 Phi-3.5-MoE 88.7%
  48. 48 Qwen2.5-Omni-7B 88.7%
  49. 49 Phi-4 Mini 88.6%
  50. 50 Nemotron 3 Ultra 550B-A55B Base 88.1%
  51. 51 Claude 2 88%
  52. 52 GLM-4 87.6%
  53. 53 OLMo 2 32B Instruct 87.6%
  54. 54 Granite 4.0 H Small 87.3%
  55. 55 Phi-3.5-mini 86.2%
  56. 56 LFM2.5-1.2B-Thinking 85.6%
  57. 57 Llama 3.1-8B 84.5%
  58. 58 LFM2-8B-A1B 84.4%
  59. 59 DeepSeek LLM 67B 84.1%
  60. 60 Falcon-H1-34B 83.6%
  61. 61 phi-3-mini 3.8B 82.5%
  62. 62 Granite 3.2 8B Instruct 81.7%
  63. 63 Granite 3.3 8B Instruct 80.9%
  64. 64 Mistral Small 3 80.7%
  65. 65 Llama 3-8B 79.6%
  66. 66 Qwen1.5-72B 79.5%
  67. 67 GLM-4.5 79.4%
  68. 68 Apertus 70B 77.6%
  69. 69 Moonlight-16B-A3B 77.4%
  70. 70 granite-4.0-1b 76.3%
  71. 71 Gemma 2 27B 74%
  72. 72 ChatGLM3-6B 72.3%
  73. 73 GLM-5 68.8%
  74. 74 Gemma 2 9B 68.6%
  75. 75 OLMo 2 1B Instruct 68.3%
  76. 76 Falcon-Arabic-7B 67.7%
  77. 77 Grok-1 62.9%
  78. 78 Gemma 3 1B 62.8%
  79. 79 Phi-2 61.1%
  80. 80 Qwen-14B 60.1%
  81. 81 Mixtral 8x7B 58.4%
  82. 82 BitNet b1.58 2B4T 58.4%
  83. 83 LFM2-1.2B 58.3%
  84. 84 Llama 2-70B 56.8%
  85. 85 Qwen-7B 54.1%
  86. 86 Mistral 7B v0.1 52.2%
  87. 87 Gemma 7B 46.4%
  88. 88 Llama 2-34B 42.2%
  89. 89 Llama 2-13B 28.7%
  90. 90 Qwen-1_8B 27.8%
  91. 91 Gemma 2B 17.7%
  92. 92 Llama 2-7B 14.6%

Source: Labs’ model cards

24 models. Scroll sideways for the rest.

  1. 56.8 davinci-002
  2. 50.6 davinci-003
  3. 48.4 Llama 2-70B
  4. 46.9 GPT-3.5 Turbo
  5. 46.6 LLaMA-65B
  6. 38.1 Mistral 7B v0.1
  7. 34.4 MPT-30B
  8. 33.8 Falcon-40B
  9. 32.0 LLaMA-33B
  10. 24.5 Llama 2-13B
  11. 22.6 Vicuna-13B-v1.3
  12. 15.4 LLaMA-13B
  13. 9.50 BLOOM-176B
  14. 9.00 GPT-3 175B (davinci)
  15. 8.00 LLaMA-7B
  16. 4.00 OPT-175B
  17. 4.00 Falcon-7B
  18. 1.80 OPT-66B
  19. 1.60 GPT-3 6.7B
  20. 0.70 GPT-3 XL (babbage)
  21. 0.60 GPT-3 Medium
  22. 0.60 InstructGPT 6B
  23. 0.40 InstructGPT 350M
  24. 0.0 InstructGPT 1.3B
  1. 1 davinci-002 56.8%
  2. 2 davinci-003 50.6%
  3. 3 Llama 2-70B 48.4%
  4. 4 GPT-3.5 Turbo Jun 2023 46.9%
  5. 5 LLaMA-65B 46.6%
  6. 6 Mistral 7B v0.1 38.1%
  7. 7 MPT-30B 34.4%
  8. 8 Falcon-40B 33.8%
  9. 9 LLaMA-33B 32%
  10. 10 Llama 2-13B 24.5%
  11. 11 Vicuna-13B-v1.3 22.6%
  12. 12 LLaMA-13B 15.4%
  13. 13 BLOOM-176B 9.50%
  14. 14 GPT-3 175B (davinci) 9.00%
  15. 15 LLaMA-7B 8.00%
  16. 16 OPT-175B 4.00%
  17. 17 Falcon-7B 4.00%
  18. 18 OPT-66B 1.80%
  19. 19 GPT-3 6.7B 1.60%
  20. 20 GPT-3 XL (babbage) 0.70%
  21. 21 GPT-3 Medium 0.60%
  22. 22 InstructGPT 6B 0.60%
  23. 23 InstructGPT 350M 0.40%
  24. 24 InstructGPT 1.3B 0%

Source: Stanford HELM, via Epoch AI (CC BY 4.0)

158 models. Scroll sideways for the rest.

  1. 96.8 Llama 3.1-405B
  2. 96.6 ERNIE-4.5-300B-A47B
  3. 96.4 Cogito v1 Preview 70B
  4. 96.4 Claude 3.5 Sonnet
  5. 96.3 DiffusionGemma 26B A4B
  6. 96.2 Cogito v2 Preview 671B MoE
  7. 95.9 Qwen2.5-32B
  8. 95.9 Gemma 3 27B
  9. 95.8 Qwen2.5-72B
  10. 95.5 Tulu 3 405B
  11. 95.3 Qwen2.5-VL-72B
  12. 95.1 Llama 3.1-70B
  13. 95.0 Claude 3 Opus
  14. 94.8 Qwen2.5-14B
  15. 94.8 MiniMax-Text-01
  16. 94.8 Qwen2.5-14B-Instruct-1M
  17. 94.5 DeepSeek-Coder-V2 236B
  18. 94.4 Gemini 1.0 Ultra
  19. 94.4 Gemma 3 12B
  20. 94.2 Qwen2.5-Coder-14B-Instruct
  21. 94.2 Granite 4.1 30B
  22. 94.0 Sarvam-M
  23. 93.8 Qwen-Turbo
  24. 93.3 Code World Model (CWM)
  25. 93.2 Qwen2-72B
  26. 93.0 Llama 3-70B
  27. 93.0 Mistral Large 2
  28. 93.0 Qwen2.5-Coder-32B-Instruct
  29. 93.0 DeepSeek V4.1 Flash
  30. 93.0 DeepSeek-V4.1-Flash Base
  31. 92.6 DeepSeek-V4-Pro
  32. 92.3 Nemotron 3 Nano 30B-A3B Base
  33. 92.3 Claude 3 Sonnet
  34. 92.2 DeepSeek-V2 (MoE-236B, May 2024)
  35. 92.0 GPT-4
  36. 91.7 Nemotron 3.5 Lightning 30B-A3B Base
  37. 91.7 Nemotron 3.5 Super 120B-A12B Base
  38. 91.7 Gemini 1.5 Pro
  39. 91.6 Qwen2.5-7B
  40. 91.6 InternVL3.5-241B-A28B
  41. 91.3 GPT-4o mini
  42. 91.2 Mistral Large
  43. 91.1 Qwen2.5-Coder-32B
  44. 91.0 PaLM 2-L
  45. 91.0 phi-3-medium 14B
  46. 90.8 Gemini 1.5 Pro
  47. 90.0 Grok-1.5
  48. 90.0 GPT-4
  49. 89.6 phi-3-small 7.4B
  50. 89.3 DeepSeek-V3
  51. 89.2 Gemma 3 4B
  52. 88.9 Claude 3 Haiku
  53. 88.7 Phi-3.5-MoE
  54. 88.7 Qwen2.5-Coder-14B
  55. 88.7 Qwen2.5-Omni-7B
  56. 88.6 Phi-4 Mini
  57. 88.1 Nemotron 3 Ultra 550B-A55B Base
  58. 88.0 Claude 2
  59. 87.6 GLM-4
  60. 87.6 DeepSeek-Coder-V2-Lite-Instruct
  61. 87.6 OLMo 2 32B Instruct
  62. 87.3 Granite 4.0 H Small
  63. 86.7 Claude Instant
  64. 86.7 Qwen2.5-Coder (7B)
  65. 86.2 Phi-3.5-mini
  66. 85.6 LFM2.5-1.2B-Thinking
  67. 84.9 Gemma 2 9B
  68. 84.5 Llama 3.1-8B
  69. 84.4 LFM2-8B-A1B
  70. 84.2 Mistral NeMo
  71. 84.1 DeepSeek LLM 67B
  72. 83.6 Falcon-H1-34B
  73. 82.5 phi-3-mini 3.8B
  74. 82.4 Gemini 1.5 Flash
  75. 81.7 Granite 3.2 8B Instruct
  76. 80.9 Granite 3.3 8B Instruct
  77. 80.7 Mistral Small 3
  78. 80.7 Qwen2.5-Coder-3B-Instruct
  79. 79.6 Llama 3-8B
  80. 79.5 Qwen1.5-72B
  81. 79.4 GLM-4.5
  82. 78.2 davinci-003
  83. 77.6 Apertus 70B
  84. 77.4 Moonlight-16B-A3B
  85. 76.3 granite-4.0-1b
  86. 76.0 Yi-34B
  87. 75.7 Qwen2.5-Coder-3B
  88. 74.4 Mixtral 8x7B
  89. 74.0 Gemma 2 27B
  90. 72.3 ChatGLM3-6B
  91. 69.6 Llama 2-70B
  92. 69.6 Stable Beluga 2
  93. 68.8 GLM-5
  94. 68.3 OLMo 2 1B Instruct
  95. 67.7 Falcon-Arabic-7B
  96. 67.1 DeepSeek-Coder-V2-Lite-Base
  97. 65.8 Qwen2.5-Coder (1.5B)
  98. 62.9 internlm-20b
  99. 62.9 Grok-1
  100. 62.8 Gemma 3 1B
  101. 61.3 Qwen-14B
  102. 61.1 Phi-2
  103. 58.4 BitNet b1.58 2B4T
  104. 58.3 LFM2-1.2B
  105. 57.8 GPT-3.5 Turbo
  106. 57.7 StarCoder 2 15B
  107. 56.8 davinci-002
  108. 56.5 PaLM (540B)
  109. 54.4 LLaMA-65B
  110. 54.4 Falcon-180B
  111. 54.4 Mistral 7B v0.1
  112. 54.1 Qwen-7B
  113. 53.8 Falcon 2 11B
  114. 52.8 Baichuan2-13B
  115. 46.4 Gemma 7B
  116. 46.0 Nemotron-4 15B
  117. 45.7 Baichuan2-13B-Chat
  118. 44.9 Yi 6B
  119. 44.1 LLaMA-33B
  120. 42.2 Llama 2-34B
  121. 38.6 INTELLECT-1
  122. 37.7 CodeQwen1.5-7B
  123. 36.9 Llama 2-13B
  124. 35.4 DeepSeek Coder 33B
  125. 35.4 Mistral 7B v0.2
  126. 34.5 Qwen2.5-Coder-0.5B
  127. 34.4 MPT-30B
  128. 33.8 Falcon-40B
  129. 33.0 PaLM 62B
  130. 32.7 StarCoder 2 7B
  131. 32.4 chatglm2-6b
  132. 31.2 internlm-7b
  133. 28.1 vicuna-13b-v1.1
  134. 27.8 Qwen-1_8B
  135. 26.8 Baichuan 1-13B
  136. 24.6 Baichuan 2-7B
  137. 22.6 Vicuna-13B-v1.3
  138. 21.6 StarCoder 2 3B
  139. 21.3 DeepSeek Coder 6.7B
  140. 20.6 LLaMA-13B
  141. 17.7 Gemma 2B
  142. 16.7 Llama 2-7B
  143. 15.7 internlm-chat-20b
  144. 11.0 LLaMA-7B
  145. 9.50 BLOOM-176B
  146. 9.20 Baichuan1-7B
  147. 9.10 MPT-7B
  148. 9.00 GPT-3 175B (davinci)
  149. 6.80 Falcon-7B
  150. 4.40 DeepSeek Coder 1.3B
  151. 4.00 OPT-175B
  152. 1.80 OPT-66B
  153. 1.60 GPT-3 6.7B
  154. 0.70 GPT-3 XL (babbage)
  155. 0.60 GPT-3 Medium
  156. 0.60 InstructGPT 6B
  157. 0.40 InstructGPT 350M
  158. 0.0 InstructGPT 1.3B
  1. 1 Llama 3.1-405B 96.8%
  2. 2 ERNIE-4.5-300B-A47B 96.6%
  3. 3 Cogito v1 Preview 70B 96.4%
  4. 4 Claude 3.5 Sonnet 96.4%
  5. 5 DiffusionGemma 26B A4B 96.3%
  6. 6 Cogito v2 Preview 671B MoE 96.2%
  7. 7 Qwen2.5-32B 95.9%
  8. 8 Gemma 3 27B 95.9%
  9. 9 Qwen2.5-72B 95.8%
  10. 10 Tulu 3 405B 95.5%
  11. 11 Qwen2.5-VL-72B 95.3%
  12. 12 Llama 3.1-70B 95.1%
  13. 13 Claude 3 Opus 95%
  14. 14 Qwen2.5-14B 94.8%
  15. 15 MiniMax-Text-01 94.8%
  16. 16 Qwen2.5-14B-Instruct-1M 94.8%
  17. 17 DeepSeek-Coder-V2 236B 94.5%
  18. 18 Gemini 1.0 Ultra 94.4%
  19. 19 Gemma 3 12B 94.4%
  20. 20 Qwen2.5-Coder-14B-Instruct 94.2%
  21. 21 Granite 4.1 30B 94.2%
  22. 22 Sarvam-M 94%
  23. 23 Qwen-Turbo 93.8%
  24. 24 Code World Model (CWM) 93.3%
  25. 25 Qwen2-72B 93.2%
  26. 26 Llama 3-70B 93%
  27. 27 Mistral Large 2 Jul 2024 93%
  28. 28 Qwen2.5-Coder-32B-Instruct 93%
  29. 29 DeepSeek V4.1 Flash 93%
  30. 30 DeepSeek-V4.1-Flash Base 93%
  31. 31 DeepSeek-V4-Pro 92.6%
  32. 32 Nemotron 3 Nano 30B-A3B Base 92.3%
  33. 33 Claude 3 Sonnet 92.3%
  34. 34 DeepSeek-V2 (MoE-236B, May 2024) 92.2%
  35. 35 GPT-4 Mar 2023 92%
  36. 36 Nemotron 3.5 Lightning 30B-A3B Base 91.7%
  37. 37 Nemotron 3.5 Super 120B-A12B Base 91.7%
  38. 38 Gemini 1.5 Pro Feb 2024 91.7%
  39. 39 Qwen2.5-7B 91.6%
  40. 40 InternVL3.5-241B-A28B 91.6%
  41. 41 GPT-4o mini 91.3%
  42. 42 Mistral Large 91.2%
  43. 43 Qwen2.5-Coder-32B 91.1%
  44. 44 PaLM 2-L 91%
  45. 45 phi-3-medium 14B 91%
  46. 46 Gemini 1.5 Pro May 2024 90.8%
  47. 47 Grok-1.5 90%
  48. 48 GPT-4 Jun 2023 90%
  49. 49 phi-3-small 7.4B 89.6%
  50. 50 DeepSeek-V3 89.3%
  51. 51 Gemma 3 4B 89.2%
  52. 52 Claude 3 Haiku 88.9%
  53. 53 Phi-3.5-MoE 88.7%
  54. 54 Qwen2.5-Coder-14B 88.7%
  55. 55 Qwen2.5-Omni-7B 88.7%
  56. 56 Phi-4 Mini 88.6%
  57. 57 Nemotron 3 Ultra 550B-A55B Base 88.1%
  58. 58 Claude 2 88%
  59. 59 GLM-4 87.6%
  60. 60 DeepSeek-Coder-V2-Lite-Instruct 87.6%
  61. 61 OLMo 2 32B Instruct 87.6%
  62. 62 Granite 4.0 H Small 87.3%
  63. 63 Claude Instant 86.7%
  64. 64 Qwen2.5-Coder (7B) 86.7%
  65. 65 Phi-3.5-mini 86.2%
  66. 66 LFM2.5-1.2B-Thinking 85.6%
  67. 67 Gemma 2 9B 84.9%
  68. 68 Llama 3.1-8B 84.5%
  69. 69 LFM2-8B-A1B 84.4%
  70. 70 Mistral NeMo 84.2%
  71. 71 DeepSeek LLM 67B 84.1%
  72. 72 Falcon-H1-34B 83.6%
  73. 73 phi-3-mini 3.8B 82.5%
  74. 74 Gemini 1.5 Flash May 2024 82.4%
  75. 75 Granite 3.2 8B Instruct 81.7%
  76. 76 Granite 3.3 8B Instruct 80.9%
  77. 77 Mistral Small 3 80.7%
  78. 78 Qwen2.5-Coder-3B-Instruct 80.7%
  79. 79 Llama 3-8B 79.6%
  80. 80 Qwen1.5-72B 79.5%
  81. 81 GLM-4.5 79.4%
  82. 82 davinci-003 78.2%
  83. 83 Apertus 70B 77.6%
  84. 84 Moonlight-16B-A3B 77.4%
  85. 85 granite-4.0-1b 76.3%
  86. 86 Yi-34B 76%
  87. 87 Qwen2.5-Coder-3B 75.7%
  88. 88 Mixtral 8x7B 74.4%
  89. 89 Gemma 2 27B 74%
  90. 90 ChatGLM3-6B 72.3%
  91. 91 Llama 2-70B 69.6%
  92. 92 Stable Beluga 2 69.6%
  93. 93 GLM-5 68.8%
  94. 94 OLMo 2 1B Instruct 68.3%
  95. 95 Falcon-Arabic-7B 67.7%
  96. 96 DeepSeek-Coder-V2-Lite-Base 67.1%
  97. 97 Qwen2.5-Coder (1.5B) 65.8%
  98. 98 internlm-20b 62.9%
  99. 99 Grok-1 62.9%
  100. 100 Gemma 3 1B 62.8%
  101. 101 Qwen-14B 61.3%
  102. 102 Phi-2 61.1%
  103. 103 BitNet b1.58 2B4T 58.4%
  104. 104 LFM2-1.2B 58.3%
  105. 105 GPT-3.5 Turbo Jun 2023 57.8%
  106. 106 StarCoder 2 15B 57.7%
  107. 107 davinci-002 56.8%
  108. 108 PaLM (540B) 56.5%
  109. 109 LLaMA-65B 54.4%
  110. 110 Falcon-180B 54.4%
  111. 111 Mistral 7B v0.1 54.4%
  112. 112 Qwen-7B 54.1%
  113. 113 Falcon 2 11B 53.8%
  114. 114 Baichuan2-13B 52.8%
  115. 115 Gemma 7B 46.4%
  116. 116 Nemotron-4 15B 46%
  117. 117 Baichuan2-13B-Chat 45.7%
  118. 118 Yi 6B 44.9%
  119. 119 LLaMA-33B 44.1%
  120. 120 Llama 2-34B 42.2%
  121. 121 INTELLECT-1 38.6%
  122. 122 CodeQwen1.5-7B 37.7%
  123. 123 Llama 2-13B 36.9%
  124. 124 DeepSeek Coder 33B 35.4%
  125. 125 Mistral 7B v0.2 35.4%
  126. 126 Qwen2.5-Coder-0.5B 34.5%
  127. 127 MPT-30B 34.4%
  128. 128 Falcon-40B 33.8%
  129. 129 PaLM 62B 33%
  130. 130 StarCoder 2 7B 32.7%
  131. 131 chatglm2-6b 32.4%
  132. 132 internlm-7b 31.2%
  133. 133 vicuna-13b-v1.1 28.1%
  134. 134 Qwen-1_8B 27.8%
  135. 135 Baichuan 1-13B 26.8%
  136. 136 Baichuan 2-7B 24.6%
  137. 137 Vicuna-13B-v1.3 22.6%
  138. 138 StarCoder 2 3B 21.6%
  139. 139 DeepSeek Coder 6.7B 21.3%
  140. 140 LLaMA-13B 20.6%
  141. 141 Gemma 2B 17.7%
  142. 142 Llama 2-7B 16.7%
  143. 143 internlm-chat-20b 15.7%
  144. 144 LLaMA-7B 11%
  145. 145 BLOOM-176B 9.50%
  146. 146 Baichuan1-7B 9.20%
  147. 147 MPT-7B 9.10%
  148. 148 GPT-3 175B (davinci) 9.00%
  149. 149 Falcon-7B 6.80%
  150. 150 DeepSeek Coder 1.3B 4.40%
  151. 151 OPT-175B 4.00%
  152. 152 OPT-66B 1.80%
  153. 153 GPT-3 6.7B 1.60%
  154. 154 GPT-3 XL (babbage) 0.70%
  155. 155 GPT-3 Medium 0.60%
  156. 156 InstructGPT 6B 0.60%
  157. 157 InstructGPT 350M 0.40%
  158. 158 InstructGPT 1.3B 0%

Sources: Labs’ model cards and Stanford HELM, via Epoch AI (CC BY 4.0)

#ModelScoreSettingDateSource
1 Llama 3.1-405B 96.8% Jul 2024 Lab model card
2 ERNIE-4.5-300B-A47B 96.6% Jun 2025 Lab model card
3 Cogito v1 Preview 70B 96.4% Apr 2025 Lab model card
4 Claude 3.5 Sonnet 96.4% Jun 2024 Lab model card
5 DiffusionGemma 26B A4B 96.3% Jul 2026 Lab model card
6 Cogito v2 Preview 671B MoE 96.2% Jul 2025 Lab model card
7 Qwen2.5-32B 95.9% Dec 2024 Lab model card
8 Gemma 3 27B 95.9% Mar 2025 Lab model card
9 Qwen2.5-72B 95.8% Sep 2024 Lab model card
10 Tulu 3 405B 95.5% Jan 2025 Lab model card
11 Qwen2.5-VL-72B 95.3% Feb 2025 Lab model card
12 Llama 3.1-70B 95.1% Jul 2024 Lab model card
13 Claude 3 Opus 95% Mar 2024 Lab model card
14 Qwen2.5-14B 94.8% Dec 2024 Lab model card
15 MiniMax-Text-01 94.8% Jan 2025 Lab model card
16 Qwen2.5-14B-Instruct-1M 94.8% Jan 2025 Lab model card
17 Gemini 1.0 Ultra 94.4% Dec 2023 Lab model card
18 Gemma 3 12B 94.4% Mar 2025 Lab model card
19 Granite 4.1 30B 94.2% Apr 2026 Lab model card
20 Sarvam-M 94% May 2025 Lab model card
21 Qwen-Turbo 93.8% Dec 2024 Lab model card
22 Code World Model (CWM) 93.3% Sep 2025 Lab model card
23 Qwen2-72B 93.2% Jul 2024 Lab model card
24 Llama 3-70B 93% Apr 2024 Lab model card
25 Mistral Large 2 Jul 2024 93% Jul 2024 Lab model card
26 DeepSeek V4.1 Flash 93% Sep 2026 Lab model card
27 DeepSeek-V4.1-Flash Base 93% Sep 2026 Lab model card
28 DeepSeek-V4-Pro 92.6% Apr 2026 Lab model card
29 Nemotron 3 Nano 30B-A3B Base 92.3% Dec 2025 Lab model card
30 Claude 3 Sonnet 92.3% Mar 2024 Lab model card
31 DeepSeek-V2 (MoE-236B, May 2024) 92.2% May 2024 Lab model card
32 GPT-4 Mar 2023 92% Mar 2023 Lab model card
33 Nemotron 3.5 Lightning 30B-A3B Base 91.7% Oct 2026 Lab model card
34 Nemotron 3.5 Super 120B-A12B Base 91.7% Oct 2026 Lab model card
35 Gemini 1.5 Pro Feb 2024 91.7% Feb 2024 Lab model card
36 Qwen2.5-7B 91.6% Dec 2024 Lab model card
37 InternVL3.5-241B-A28B 91.6% Aug 2025 Lab model card
38 Mistral Large 91.2% Feb 2024 Lab model card
39 PaLM 2-L 91% May 2023 Lab model card
40 phi-3-medium 14B 91% Aug 2024 Lab model card
41 Gemini 1.5 Pro May 2024 90.8% May 2024 Lab model card
42 Grok-1.5 90% Mar 2024 Lab model card
43 phi-3-small 7.4B 89.6% Aug 2024 Lab model card
44 DeepSeek-V3 89.3% Dec 2024 Lab model card
45 Gemma 3 4B 89.2% Mar 2025 Lab model card
46 Claude 3 Haiku 88.9% Mar 2024 Lab model card
47 Phi-3.5-MoE 88.7% Aug 2024 Lab model card
48 Qwen2.5-Omni-7B 88.7% Mar 2025 Lab model card
49 Phi-4 Mini 88.6% Feb 2025 Lab model card
50 Nemotron 3 Ultra 550B-A55B Base 88.1% Jun 2026 Lab model card
51 Claude 2 88% Jul 2023 Lab model card
52 GLM-4 87.6% Jan 2024 Lab model card
53 OLMo 2 32B Instruct 87.6% Mar 2025 Lab model card
54 Granite 4.0 H Small 87.3% Oct 2025 Lab model card
55 Phi-3.5-mini 86.2% Aug 2024 Lab model card
56 LFM2.5-1.2B-Thinking 85.6% Jan 2026 Lab model card
57 Llama 3.1-8B 84.5% Jul 2024 Lab model card
58 LFM2-8B-A1B 84.4% Oct 2025 Lab model card
59 DeepSeek LLM 67B 84.1% Nov 2023 Lab model card
60 Falcon-H1-34B 83.6% May 2025 Lab model card
61 phi-3-mini 3.8B 82.5% Aug 2024 Lab model card
62 Granite 3.2 8B Instruct 81.7% Feb 2025 Lab model card
63 Granite 3.3 8B Instruct 80.9% Apr 2025 Lab model card
64 Mistral Small 3 80.7% Jan 2025 Lab model card
65 Llama 3-8B 79.6% Apr 2024 Lab model card
66 Qwen1.5-72B 79.5% Feb 2024 Lab model card
67 GLM-4.5 79.4% Aug 2025 Lab model card
68 Apertus 70B 77.6% Sep 2025 Lab model card
69 Moonlight-16B-A3B 77.4% Feb 2025 Lab model card
70 granite-4.0-1b 76.3% Oct 2025 Lab model card
71 Gemma 2 27B 74% Jun 2024 Lab model card
72 ChatGLM3-6B 72.3% Oct 2023 Lab model card
73 GLM-5 68.8% Feb 2026 Lab model card
74 Gemma 2 9B 68.6% Jul 2024 Lab model card
75 OLMo 2 1B Instruct 68.3% Apr 2025 Lab model card
76 Falcon-Arabic-7B 67.7% Nov 2025 Lab model card
77 Grok-1 62.9% Nov 2023 Lab model card
78 Gemma 3 1B 62.8% Mar 2025 Lab model card
79 Phi-2 61.1% Dec 2023 Lab model card
80 Qwen-14B 60.1% Sep 2023 Lab model card
81 Mixtral 8x7B 58.4% Dec 2023 Lab model card
82 BitNet b1.58 2B4T 58.4% Apr 2025 Lab model card
83 LFM2-1.2B 58.3% Jul 2025 Lab model card
84 Llama 2-70B 56.8% Jul 2023 Lab model card
85 Qwen-7B 54.1% Sep 2023 Lab model card
86 Mistral 7B v0.1 52.2% Oct 2023 Lab model card
87 Gemma 7B 46.4% Feb 2024 Lab model card
88 Llama 2-34B 42.2% Jul 2023 Lab model card
89 Llama 2-13B 28.7% Jul 2023 Lab model card
90 Qwen-1_8B 27.8% Sep 2023 Lab model card
91 Gemma 2B 17.7% Mar 2024 Lab model card
92 Llama 2-7B 14.6% Jul 2023 Lab model card
#ModelScoreSettingDateSource
1 davinci-002 56.8% Mar 2022 Stanford HELM, via Epoch AI
2 davinci-003 50.6% Nov 2022 Stanford HELM, via Epoch AI
3 Llama 2-70B 48.4% Jul 2023 Stanford HELM, via Epoch AI
4 GPT-3.5 Turbo Jun 2023 46.9% Jun 2023 Stanford HELM, via Epoch AI
5 LLaMA-65B 46.6% Feb 2023 Stanford HELM, via Epoch AI
6 Mistral 7B v0.1 38.1% Sep 2023 Stanford HELM, via Epoch AI
7 MPT-30B 34.4% Jun 2023 Stanford HELM, via Epoch AI
8 Falcon-40B 33.8% May 2023 Stanford HELM, via Epoch AI
9 LLaMA-33B 32% Feb 2023 Stanford HELM, via Epoch AI
10 Llama 2-13B 24.5% Jul 2023 Stanford HELM, via Epoch AI
11 Vicuna-13B-v1.3 22.6% Jun 2023 Stanford HELM, via Epoch AI
12 LLaMA-13B 15.4% Feb 2023 Stanford HELM, via Epoch AI
13 BLOOM-176B 9.50% Jul 2022 Stanford HELM, via Epoch AI
14 GPT-3 175B (davinci) 9.00% May 2020 Stanford HELM, via Epoch AI
15 LLaMA-7B 8.00% Feb 2023 Stanford HELM, via Epoch AI
16 OPT-175B 4.00% May 2022 Stanford HELM, via Epoch AI
17 Falcon-7B 4.00% Apr 2023 Stanford HELM, via Epoch AI
18 OPT-66B 1.80% May 2022 Stanford HELM, via Epoch AI
19 GPT-3 6.7B 1.60% Jun 2020 Stanford HELM, via Epoch AI
20 GPT-3 XL (babbage) 0.70% Jun 2020 Stanford HELM, via Epoch AI
21 GPT-3 Medium 0.60% Jun 2020 Stanford HELM, via Epoch AI
22 InstructGPT 6B 0.60% Jan 2022 Stanford HELM, via Epoch AI
23 InstructGPT 350M 0.40% Jan 2022 Stanford HELM, via Epoch AI
24 InstructGPT 1.3B 0% Jan 2022 Stanford HELM, via Epoch AI
#ModelScoreSettingDateSource
1 Llama 3.1-405B 96.8% Jul 2024 Lab model card
2 ERNIE-4.5-300B-A47B 96.6% Jun 2025 Lab model card
3 Cogito v1 Preview 70B 96.4% Apr 2025 Lab model card
4 Claude 3.5 Sonnet 96.4% Jun 2024 Lab model card
5 DiffusionGemma 26B A4B 96.3% Jul 2026 Lab model card
6 Cogito v2 Preview 671B MoE 96.2% Jul 2025 Lab model card
7 Qwen2.5-32B 95.9% Dec 2024 Lab model card
8 Gemma 3 27B 95.9% Mar 2025 Lab model card
9 Qwen2.5-72B 95.8% Sep 2024 Lab model card
10 Tulu 3 405B 95.5% Jan 2025 Lab model card
11 Qwen2.5-VL-72B 95.3% Feb 2025 Lab model card
12 Llama 3.1-70B 95.1% Jul 2024 Lab model card
13 Claude 3 Opus 95% Mar 2024 Lab model card
14 Qwen2.5-14B 94.8% Dec 2024 Lab model card
15 MiniMax-Text-01 94.8% Jan 2025 Lab model card
16 Qwen2.5-14B-Instruct-1M 94.8% Jan 2025 Lab model card
17 DeepSeek-Coder-V2 236B 94.5% Jun 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
18 Gemini 1.0 Ultra 94.4% Dec 2023 Lab model card
19 Gemma 3 12B 94.4% Mar 2025 Lab model card
20 Qwen2.5-Coder-14B-Instruct 94.2% Nov 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
21 Granite 4.1 30B 94.2% Apr 2026 Lab model card
22 Sarvam-M 94% May 2025 Lab model card
23 Qwen-Turbo 93.8% Dec 2024 Lab model card
24 Code World Model (CWM) 93.3% Sep 2025 Lab model card
25 Qwen2-72B 93.2% Jul 2024 Lab model card
26 Llama 3-70B 93% Apr 2024 Lab model card
27 Mistral Large 2 Jul 2024 93% Jul 2024 Lab model card
28 Qwen2.5-Coder-32B-Instruct 93% Nov 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
29 DeepSeek V4.1 Flash 93% Sep 2026 Lab model card
30 DeepSeek-V4.1-Flash Base 93% Sep 2026 Lab model card
31 DeepSeek-V4-Pro 92.6% Apr 2026 Lab model card
32 Nemotron 3 Nano 30B-A3B Base 92.3% Dec 2025 Lab model card
33 Claude 3 Sonnet 92.3% Mar 2024 Lab model card
34 DeepSeek-V2 (MoE-236B, May 2024) 92.2% May 2024 Lab model card
35 GPT-4 Mar 2023 92% Mar 2023 Lab model card
36 Nemotron 3.5 Lightning 30B-A3B Base 91.7% Oct 2026 Lab model card
37 Nemotron 3.5 Super 120B-A12B Base 91.7% Oct 2026 Lab model card
38 Gemini 1.5 Pro Feb 2024 91.7% Feb 2024 Lab model card
39 Qwen2.5-7B 91.6% Dec 2024 Lab model card
40 InternVL3.5-241B-A28B 91.6% Aug 2025 Lab model card
41 GPT-4o mini 91.3% Jul 2024 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
42 Mistral Large 91.2% Feb 2024 Lab model card
43 Qwen2.5-Coder-32B 91.1% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
44 PaLM 2-L 91% May 2023 Lab model card
45 phi-3-medium 14B 91% Aug 2024 Lab model card
46 Gemini 1.5 Pro May 2024 90.8% May 2024 Lab model card
47 Grok-1.5 90% Mar 2024 Lab model card
48 GPT-4 Jun 2023 90% Jun 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
49 phi-3-small 7.4B 89.6% Aug 2024 Lab model card
50 DeepSeek-V3 89.3% Dec 2024 Lab model card
51 Gemma 3 4B 89.2% Mar 2025 Lab model card
52 Claude 3 Haiku 88.9% Mar 2024 Lab model card
53 Phi-3.5-MoE 88.7% Aug 2024 Lab model card
54 Qwen2.5-Coder-14B 88.7% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
55 Qwen2.5-Omni-7B 88.7% Mar 2025 Lab model card
56 Phi-4 Mini 88.6% Feb 2025 Lab model card
57 Nemotron 3 Ultra 550B-A55B Base 88.1% Jun 2026 Lab model card
58 Claude 2 88% Jul 2023 Lab model card
59 GLM-4 87.6% Jan 2024 Lab model card
60 DeepSeek-Coder-V2-Lite-Instruct 87.6% Jun 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
61 OLMo 2 32B Instruct 87.6% Mar 2025 Lab model card
62 Granite 4.0 H Small 87.3% Oct 2025 Lab model card
63 Claude Instant 86.7% Aug 2023 Claim in Releasing Claude Instant 1.2, via Epoch AI
64 Qwen2.5-Coder (7B) 86.7% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
65 Phi-3.5-mini 86.2% Aug 2024 Lab model card
66 LFM2.5-1.2B-Thinking 85.6% Jan 2026 Lab model card
67 Gemma 2 9B 84.9% Jun 2024 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
68 Llama 3.1-8B 84.5% Jul 2024 Lab model card
69 LFM2-8B-A1B 84.4% Oct 2025 Lab model card
70 Mistral NeMo 84.2% Jul 2024 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
71 DeepSeek LLM 67B 84.1% Nov 2023 Lab model card
72 Falcon-H1-34B 83.6% May 2025 Lab model card
73 phi-3-mini 3.8B 82.5% Aug 2024 Lab model card
74 Gemini 1.5 Flash May 2024 82.4% May 2024 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
75 Granite 3.2 8B Instruct 81.7% Feb 2025 Lab model card
76 Granite 3.3 8B Instruct 80.9% Apr 2025 Lab model card
77 Mistral Small 3 80.7% Jan 2025 Lab model card
78 Qwen2.5-Coder-3B-Instruct 80.7% Nov 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
79 Llama 3-8B 79.6% Apr 2024 Lab model card
80 Qwen1.5-72B 79.5% Feb 2024 Lab model card
81 GLM-4.5 79.4% Aug 2025 Lab model card
82 davinci-003 78.2% Nov 2022 Claim in Qwen Technical Report, via Epoch AI
83 Apertus 70B 77.6% Sep 2025 Lab model card
84 Moonlight-16B-A3B 77.4% Feb 2025 Lab model card
85 granite-4.0-1b 76.3% Oct 2025 Lab model card
86 Yi-34B 76% Nov 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
87 Qwen2.5-Coder-3B 75.7% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
88 Mixtral 8x7B 74.4% Dec 2023 Claim in Mixtral of Experts, via Epoch AI
89 Gemma 2 27B 74% Jun 2024 Lab model card
90 ChatGLM3-6B 72.3% Oct 2023 Lab model card
91 Llama 2-70B 69.6% Jul 2023 Claim in Mixtral of Experts, via Epoch AI
92 Stable Beluga 2 69.6% Jul 2023 Claim in Qwen Technical Report, via Epoch AI
93 GLM-5 68.8% Feb 2026 Lab model card
94 OLMo 2 1B Instruct 68.3% Apr 2025 Lab model card
95 Falcon-Arabic-7B 67.7% Nov 2025 Lab model card
96 DeepSeek-Coder-V2-Lite-Base 67.1% Jun 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
97 Qwen2.5-Coder (1.5B) 65.8% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
98 internlm-20b 62.9% Sep 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
99 Grok-1 62.9% Nov 2023 Lab model card
100 Gemma 3 1B 62.8% Mar 2025 Lab model card
101 Qwen-14B 61.3% Sep 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
102 Phi-2 61.1% Dec 2023 Lab model card
103 BitNet b1.58 2B4T 58.4% Apr 2025 Lab model card
104 LFM2-1.2B 58.3% Jul 2025 Lab model card
105 GPT-3.5 Turbo Jun 2023 57.8% Jun 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
106 StarCoder 2 15B 57.7% Feb 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
107 davinci-002 56.8% Mar 2022 Stanford HELM, via Epoch AI
108 PaLM (540B) 56.5% Apr 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
109 LLaMA-65B 54.4% Feb 2023 Claim in Qwen Technical Report, via Epoch AI
110 Falcon-180B 54.4% Sep 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
111 Mistral 7B v0.1 54.4% Sep 2023 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
112 Qwen-7B 54.1% Sep 2023 Lab model card
113 Falcon 2 11B 53.8% May 2024 Claim in Falcon2-11B Technical Report, via Epoch AI
114 Baichuan2-13B 52.8% Sep 2023 Claim in Nemotron-4 15B Technical Report, via Epoch AI
115 Gemma 7B 46.4% Feb 2024 Lab model card
116 Nemotron-4 15B 46% Feb 2024 Claim in Nemotron-4 15B Technical Report, via Epoch AI
117 Baichuan2-13B-Chat 45.7% Sep 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
118 Yi 6B 44.9% Nov 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
119 LLaMA-33B 44.1% Feb 2023 Claim in Mixtral of Experts, via Epoch AI
120 Llama 2-34B 42.2% Jul 2023 Lab model card
121 INTELLECT-1 38.6% Nov 2024 Claim in INTELLECT-1 Technical Report, via Epoch AI
122 CodeQwen1.5-7B 37.7% Apr 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
123 Llama 2-13B 36.9% Jul 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
124 DeepSeek Coder 33B 35.4% Nov 2023 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
125 Mistral 7B v0.2 35.4% Dec 2023 Claim in Gemma: Open Models Based on Gemini Research and Technology, via Epoch AI
126 Qwen2.5-Coder-0.5B 34.5% Sep 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
127 MPT-30B 34.4% Jun 2023 Stanford HELM, via Epoch AI
128 Falcon-40B 33.8% May 2023 Stanford HELM, via Epoch AI
129 PaLM 62B 33% Apr 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
130 StarCoder 2 7B 32.7% Feb 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
131 chatglm2-6b 32.4% Jun 2023 Claim in Qwen Technical Report, via Epoch AI
132 internlm-7b 31.2% Jul 2023 Claim in Qwen Technical Report, via Epoch AI
133 vicuna-13b-v1.1 28.1% Apr 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
134 Qwen-1_8B 27.8% Sep 2023 Lab model card
135 Baichuan 1-13B 26.8% Jul 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
136 Baichuan 2-7B 24.6% Sep 2023 Claim in Qwen Technical Report, via Epoch AI
137 Vicuna-13B-v1.3 22.6% Jun 2023 Stanford HELM, via Epoch AI
138 StarCoder 2 3B 21.6% Feb 2024 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
139 DeepSeek Coder 6.7B 21.3% Nov 2023 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
140 LLaMA-13B 20.6% Feb 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
141 Gemma 2B 17.7% Mar 2024 Lab model card
142 Llama 2-7B 16.7% Jul 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
143 internlm-chat-20b 15.7% Sep 2023 Claim in Yi: Open Foundation Models by 01.AI, via Epoch AI
144 LLaMA-7B 11% Feb 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
145 BLOOM-176B 9.50% Jul 2022 Stanford HELM, via Epoch AI
146 Baichuan1-7B 9.20% Jun 2023 Claim in Baichuan 2: Open Large-scale Language Models, via Epoch AI
147 MPT-7B 9.10% May 2023 Claim in Qwen Technical Report, via Epoch AI
148 GPT-3 175B (davinci) 9.00% May 2020 Stanford HELM, via Epoch AI
149 Falcon-7B 6.80% Apr 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
150 DeepSeek Coder 1.3B 4.40% Nov 2023 Claim in Qwen2.5-Coder Technical Report, via Epoch AI
151 OPT-175B 4.00% May 2022 Stanford HELM, via Epoch AI
152 OPT-66B 1.80% May 2022 Stanford HELM, via Epoch AI
153 GPT-3 6.7B 1.60% Jun 2020 Stanford HELM, via Epoch AI
154 GPT-3 XL (babbage) 0.70% Jun 2020 Stanford HELM, via Epoch AI
155 GPT-3 Medium 0.60% Jun 2020 Stanford HELM, via Epoch AI
156 InstructGPT 6B 0.60% Jan 2022 Stanford HELM, via Epoch AI
157 InstructGPT 350M 0.40% Jan 2022 Stanford HELM, via Epoch AI
158 InstructGPT 1.3B 0% Jan 2022 Stanford HELM, via Epoch AI

Epoch AI copied scores for 24 models from Stanford HELM. Epoch AI collected claims for 70 models from labs’ reports and papers. Switch the scores above to rank them together.

5 models have held number one in labs’ own numbers since 2023 Number one over time 6 models have held number one with claims included since 2023

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for Stanford HELM, which Epoch AI copied and what labs claim in their cards.

OpenAI Google DeepMind Anthropic Meta
Stanford HELM, via Epoch AI Not tested since Sep 2023
davinci-002 56.8%
Lab claims Passed 85% in Mar 2023, by GPT-4
Llama 3.1-405B 96.8%

Sources: Stanford HELM, via Epoch AI (CC BY 4.0) and labs’ model cards

OpenAI’s own number for GPT-4 passed 85% in March 2023

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card On another leaderboard, collected by Epoch AI A claim Epoch AI collected from a report or paper Best so far
0% 20% 40% 60% 80% 100% 2020 2021 2022 2023 2024 2025 2026 Near the ceiling (85%) Llama 3.1-405B GPT-4 davinci-003 davinci-002 GPT-3 175B (davinci) 0% 20% 40% 60% 80% 100% 2020 2022 2024 2026 Near the ceiling (85%) Llama 3.1-405B davinci-003

Sources: Other boards, via Epoch AI (CC BY 4.0), labs’ model cards and claims collected by Epoch AI (CC BY 4.0)

Labs headline it less often now, last in September 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

OpenAI
Google DeepMind
Anthropic
Qwen
Mistral AI
Z.ai
xAI
DeepSeek
Microsoft
Meta
MiniMax
Ai2
Moonshot AI
IBM
Baidu
Deep Cogito
Technology Innovation Institute
Sarvam AI
Liquid AI
DateLaunch postScoreAs the post writes it
Sep 10, 2026 DeepSeek V4.1 Flash 93 GSM8K (EM)
Apr 29, 2026 Granite 4.1 30B 94.2 GSM8K (8-shot)
Apr 24, 2026 DeepSeek-V4-Pro 92.6 GSM8K (EM)
Jan 20, 2026 LFM2.5-1.2B-Thinking 85.6 GSM8K
Oct 28, 2025 granite-4.0-1b 76.3 GSM8K (8-shot)
Oct 7, 2025 LFM2-8B-A1B 84.4 GSM8K
Oct 2, 2025 Granite 4.0 H Small 87.3 GSM8K (8-shot)
Jul 31, 2025 Cogito v2 Preview 671B MoE 96.2 GSM8K (thinking)
Jul 10, 2025 LFM2-1.2B 58.3 GSM8K
Jun 30, 2025 ERNIE-4.5-300B-A47B 96.6 GSM8K
May 23, 2025 Sarvam-M 94 GSM-8K (0.94)
May 20, 2025 Falcon-H1-34B 83.6 GSM8k
Apr 29, 2025 OLMo 2 1B Instruct 68.3 GSM8K
Apr 16, 2025 Granite 3.3 8B Instruct 80.9 GSM8K
Apr 15, 2025 BitNet b1.58 2B4T 58.4 GSM8K
Apr 8, 2025 Cogito v1 Preview 70B 96.4 GSM8K (thinking)
Mar 27, 2025 Qwen2.5-Omni-7B 88.7 GSM8K
Mar 16, 2025 ERNIE 4.5 GSM8K
Mar 13, 2025 OLMo 2 32B Instruct 87.6 GSM8k
Mar 12, 2025 Gemma 3 27B 95.9 GSM8K 0-shot (IT); PT 8-shot = 82.6
Feb 26, 2025 Phi-4 Mini 88.6 GSM8K (8-shot, CoT)
Feb 26, 2025 Granite 3.2 8B Instruct 81.7 GSM8K
Feb 22, 2025 Moonlight-16B-A3B 77.4 GSM8K
Jan 30, 2025 Tulu 3 405B 95.5 GSM8K (8 shot, CoT)
Jan 30, 2025 Mistral Small 3 80.7 GSM8K (5-shot, maj@1) [base]
Jan 28, 2025 Qwen2.5-Max GSM8K
Jan 15, 2025 MiniMax-Text-01 94.8 GSM8k*
Dec 26, 2024 DeepSeek-V3 89.3 GSM8K (EM)
Sep 19, 2024 Qwen2.5-72B 95.8 GSM8K
Aug 20, 2024 Phi-3.5-MoE 88.7 GSM8K (8-shot, CoT)
Jul 24, 2024 Mistral Large 2 93 GSM8K (8-shot)
Jul 23, 2024 Llama 3.1-405B 96.8 GSM8K (8-shot, CoT)
Jun 27, 2024 Gemma 2 27B 74 GSM8K (5-shot, maj@1)
Jun 20, 2024 Claude 3.5 Sonnet 96.4 GSM8K (0-shot CoT)
Jun 7, 2024 Qwen2-72B 91.1 GSM8K
May 14, 2024 Gemini 1.5 Pro 90.8 GSM8K, 11-shot
May 7, 2024 DeepSeek-V2 (MoE-236B, May 2024) 92.2 GSM8K
Apr 22, 2024 phi-3-medium 14B 90.3 GSM-8K (0-Shot; CoT)
Apr 18, 2024 Llama 3-70B 93 GSM-8K (8-shot, CoT) [instruct]
Mar 28, 2024 Grok-1.5 90 GSM8K (8-shot, maj@1)
Mar 4, 2024 Claude 3 Opus 95 GSM8K (0-shot CoT)
Feb 26, 2024 Mistral Large 91.2 GSM8K maj@8 (8-shot) [maj@1 (5-shot) = 81.0]
Feb 21, 2024 Gemma 7B 46.4 GSM8K (maj@1)
Feb 15, 2024 Gemini 1.5 Pro 91.7 GSM8K, 11-shot
Feb 5, 2024 Qwen1.5-72B 79.5 GSM8K
Jan 16, 2024 GLM-4 87.6 GSM8K (5-shot)
Dec 12, 2023 Phi-2 61.1 Math (GSM8k, 8-shot)
Dec 11, 2023 Mixtral 8x7B 58.4 GSM-8K (5-shot) [detailed table: 74.4]
Dec 6, 2023 Gemini 1.0 Ultra 94.4 GSM8K, Maj1@32
Nov 29, 2023 DeepSeek LLM 67B 84.1 GSM8K
Nov 3, 2023 Grok-1 62.9 GSM8k (8-shot)
Oct 27, 2023 ChatGLM3-6B 72.3 GSM8K
Sep 27, 2023 Mistral 7B v0.1 52.1 GSM8K (8-shot, maj@8)
Sep 25, 2023 Qwen-14B 60.1 GSM8K (0-shot)
Jul 11, 2023 Claude 2 88 GSM8k (0-shot CoT)
May 10, 2023 PaLM 2-L 91 GSM8K, CoT + self-consistency (greedy CoT = 80.7)
Mar 14, 2023 GPT-4 92 GSM-8K (5-shot chain-of-thought)

72 system cards report it, from 17 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 15, 2026 Nemotron 3.5 Super 120B-A12B Base technical report 91.7 GSM8K, 8-shot CoT exact match, Table 6 Capabilities
Oct 15, 2026 Nemotron 3.5 Lightning 30B-A3B Base technical report 91.7 GSM8K, 8-shot CoT exact match, Table 6 Capabilities
Sep 10, 2026 DeepSeek-V4.1-Flash Base model card 93 GSM8K, EM, 8-shot Capabilities
Jul 31, 2026 DiffusionGemma 26B A4B technical report 96.3 GSM8K, TD, 95.8 no-think (Table 3) Capabilities
Jun 4, 2026 Nemotron 3 Ultra 550B-A55B Base model card 88.1 GSM8K, 8-shot CoT EM Capabilities
Feb 17, 2026 GLM-5 technical report 68.8 GLM-5-Base, GSM8K (EM) (Table 11) Capabilities
Dec 23, 2025 Nemotron 3 Nano 30B-A3B Base technical report 92.3 GSM8K, 8-shot accuracy, base model Table 2 Capabilities
Nov 28, 2025 LFM2-8B-A1B technical report 84.4 GSM8K, LFM2-8B-A1B (Table 7) Capabilities
Nov 28, 2025 LFM2-1.2B technical report 58.3 GSM8K, LFM2-1.2B (Table 6) Capabilities
Nov 8, 2025 Falcon-Arabic-7B technical report 67.7 GSM8K 5-shot, English (Table 8). Arabic-GSM8K exact match 54.89 (Table 7) Capabilities
Oct 8, 2025 OLMo 2 1B Instruct technical report 68.3 GSM8K (Table 7, Table 23) Capabilities
Sep 30, 2025 Code World Model (CWM) technical report 93.3 GSM8k, without reasoning (Table 13) Capabilities
Sep 17, 2025 Apertus 70B technical report 77.6 GSM8K, 8-shot CoT (Table 18) Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 91.6 GSM8K, OpenCompass (Table 14) Capabilities
Aug 8, 2025 GLM-4.5 technical report 79.4 GLM-4.5-Base, GSM8K (EM) (Table 2) Capabilities
Jul 30, 2025 Falcon-H1-34B technical report 83.6 GSM8k, strict match 5-shot (Table 24) Capabilities
Jul 17, 2025 Voxtral Small technical report GSM8K 8-shot, text-only, chart without printed numbers (Figure 6). Speech-synthesized GSM8k* 89.7 (Table 8) Capabilities
Jun 29, 2025 ERNIE-4.5-300B-A47B technical report 96.6 GSM8K (Table 5) Capabilities
Apr 16, 2025 BitNet b1.58 2B4T technical report 58.4 GSM8K (4-shot; EM) (Table 1, Table 2) Capabilities
Mar 26, 2025 Qwen2.5-Omni-7B technical report 88.7 GSM8K (Table 1) Capabilities
Mar 25, 2025 Gemma 3 4B technical report 89.2 GSM8K, IT, 0-shot CoT (Table 18) Capabilities
Mar 25, 2025 Gemma 3 27B technical report 95.9 GSM8K, IT, 0-shot CoT (Table 18) Capabilities
Mar 25, 2025 Gemma 3 1B technical report 62.8 GSM8K, IT, 0-shot CoT (Table 18) Capabilities
Mar 25, 2025 Gemma 3 12B technical report 94.4 GSM8K, IT, 0-shot CoT (Table 18) Capabilities
Mar 3, 2025 Phi-4 Mini technical report 88.6 GSM-8K (8-Shot, CoT) (Table 7) Capabilities
Feb 19, 2025 Qwen2.5-VL-72B technical report 95.3 GSM8K (Table 4) Capabilities
Jan 26, 2025 Qwen2.5-14B-Instruct-1M technical report 94.8 GSM8K (Table 6) Capabilities
Jan 14, 2025 MiniMax-Text-01 technical report 94.8 GSM8k, 0-shot CoT (Table 8) Capabilities
Dec 31, 2024 OLMo 2 32B Instruct technical report 87.6 GSM8K, 8-shot CoT (Tables 7 and 16) Capabilities
Dec 19, 2024 Qwen2.5-7B technical report 91.6 GSM8K, Qwen2.5-7B-Instruct (Table 8) Capabilities
Dec 19, 2024 Qwen2.5-72B technical report 95.8 GSM8K, Qwen2.5-72B-Instruct (Table 6) Capabilities
Dec 19, 2024 Qwen2.5-32B technical report 95.9 GSM8K, Qwen2.5-32B-Instruct (Table 7) Capabilities
Dec 19, 2024 Qwen2.5-14B technical report 94.8 GSM8K, Qwen2.5-14B-Instruct (Table 7) Capabilities
Dec 19, 2024 Qwen-Turbo technical report 93.8 GSM8K, Qwen2.5-Turbo (Table 7) Capabilities
Nov 23, 2024 Llama 3.1-8B technical report 84.5 GSM8K (8-shot, CoT), post-trained (Table 2). Pretrained many-shot GSM8K (16-shot) = 60.0 (Table 14) Capabilities
Nov 23, 2024 Llama 3.1-70B technical report 95.1 GSM8K (8-shot, CoT), post-trained (Table 2). Pretrained many-shot GSM8K (16-shot) = 83.0 (Table 14) Capabilities
Nov 23, 2024 Llama 3.1-405B technical report 96.8 GSM8K (8-shot, CoT), post-trained (Table 2). Pretrained many-shot GSM8K (16-shot) = 90.0 (Table 14) Capabilities
Nov 22, 2024 Tulu 3 405B technical report 95.5 GSM8K (8 shot, CoT), RLVR (Table 4) Capabilities
Aug 30, 2024 phi-3-small 7.4B technical report 89.6 GSM-8K (8-Shot, CoT) (Section 3 table) Capabilities
Aug 30, 2024 phi-3-mini 3.8B technical report 82.5 GSM-8K (8-Shot, CoT) (Section 3 table) Capabilities
Aug 30, 2024 phi-3-medium 14B technical report 91 GSM-8K (8-Shot, CoT) (Section 3 table) Capabilities
Aug 30, 2024 Phi-3.5-MoE technical report 88.7 GSM8K (8-shot, CoT) (Table 3) Capabilities
Aug 30, 2024 Phi-3.5-mini technical report 86.2 GSM8K (8-shot, CoT) (Table 3) Capabilities
Jul 31, 2024 Gemma 2 9B technical report 68.6 GSM8K, PT, 5-shot (Table 13) Capabilities
Jul 31, 2024 Gemma 2 27B technical report 74 GSM8K, PT, 5-shot (Table 13) Capabilities
Jul 23, 2024 Llama 3.1-8B model card 84.5 GSM-8K (CoT), 8-shot, em_maj1@1, instruction tuned (Instruction tuned models table) Capabilities
Jul 23, 2024 Llama 3.1-70B model card 95.1 GSM-8K (CoT), 8-shot, em_maj1@1, instruction tuned (Instruction tuned models table) Capabilities
Jul 23, 2024 Llama 3.1-405B model card 96.8 GSM-8K (CoT), 8-shot, em_maj1@1, instruction tuned (Instruction tuned models table) Capabilities
Jul 15, 2024 Qwen2-72B technical report 93.2 GSM8K, Qwen2-72B-Instruct (Table 6) Capabilities
Jun 20, 2024 Claude 3.5 Sonnet model card 96.4 GSM8K, 0-shot CoT (Table 1) Capabilities
Jun 18, 2024 GLM-4 technical report 87.6 GLM-4 (0116), GSM8K, chain-of-thought (Table 2) Capabilities
Jun 18, 2024 ChatGLM3-6B technical report 72.3 ChatGLM3-6B-Base, GSM8K (Table 1) Capabilities
May 7, 2024 DeepSeek-V2 (MoE-236B, May 2024) technical report 92.2 GSM8K 8-shot, DeepSeek-V2 Chat (RL) (Table 3) Capabilities
Apr 18, 2024 Llama 3-8B model card 79.6 GSM-8K (8-shot, CoT), instruction tuned (Instruction tuned models table) Capabilities
Apr 18, 2024 Llama 3-70B model card 93 GSM-8K (8-shot, CoT), instruction tuned (Instruction tuned models table) Capabilities
Mar 13, 2024 Gemma 7B technical report 46.4 GSM8K, PT, maj@1 (Table 6). HuggingFace H6 run (Table 7): 50.9 Capabilities
Mar 13, 2024 Gemma 2B technical report 17.7 GSM8K, PT, maj@1 (Table 6) Capabilities
Mar 4, 2024 Claude 3 Sonnet model card 92.3 GSM8K, 0-shot CoT (Table 1) Capabilities
Mar 4, 2024 Claude 3 Opus model card 95 GSM8K, 0-shot CoT (Table 1) Capabilities
Mar 4, 2024 Claude 3 Haiku model card 88.9 GSM8K, 0-shot CoT (Table 1) Capabilities
Jan 8, 2024 Mixtral 8x7B technical report 58.4 GSM-8K, 5-shot (Table 3). 8-shot maj@8 74.4 (Table 2) Capabilities
Jan 5, 2024 DeepSeek LLM 67B technical report 84.1 GSM8K 0-shot, chain of thought, DeepSeek 67B Chat (Table 6). 86.7 with tool-integrated reasoning (Table 16) Capabilities
Oct 10, 2023 Mistral 7B v0.1 technical report 52.2 GSM8K, 8-shot maj@8 (Table 2) Capabilities
Sep 28, 2023 Qwen-7B technical report 54.1 GSM8K 8-shot, higher of the two settings, 0-shot 50.3 (Table 5) Capabilities
Sep 28, 2023 Qwen-14B technical report 60.1 GSM8K 0-shot, higher of the two settings, 8-shot 59.3 (Table 5) Capabilities
Sep 28, 2023 Qwen-1_8B technical report 27.8 GSM8K 0-shot, higher of the two settings, 8-shot 19.5 (Table 5) Capabilities
Jul 19, 2023 Llama 2-7B technical report 14.6 GSM8k, 8-shot, maj1@1, pretrained (Table 25) Capabilities
Jul 19, 2023 Llama 2-70B technical report 56.8 GSM8k, 8-shot, maj1@1, pretrained (Table 25) Capabilities
Jul 19, 2023 Llama 2-34B technical report 42.2 GSM8k, 8-shot, maj1@1, pretrained (Table 25) Capabilities
Jul 19, 2023 Llama 2-13B technical report 28.7 GSM8k, 8-shot, maj1@1, pretrained (Table 25) Capabilities
Jul 11, 2023 Claude 2 model card 88 GSM8k (0-shot CoT) (4.3) Capabilities
Mar 14, 2023 GPT-4 technical report 92 GSM-8K, 5-shot chain-of-thought, part of the training set in pre-training mix (Table 2, base model) Capabilities

Versions

Replaced by MATH

More in Math

All 113 in Math
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • MathArena Apex Shortlist In use Models solve 47 hard final-answer problems that MathArena selected from 2025 competitions, where the best models scored about 50% at selection. Scored by accuracy over repeated attempts. GPT-5.5 98.4%