Benchmark Almanac

GSM8K Platinum Initial 268

o1-preview 99.6% leads since February 2025
Not in model cards

Models answer the original 268-question cleaned GSM8K subset. The live matrix reports each model configuration separately.

This initial subset differs from the full 1,209-question GSM8K-Platinum test set.

Released
February 2025
Kind
Word problems
Level
School
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Strongest model tested
Gemini 2.5 Pro Mar 2025 #103 on the capability index
Last model added
April 2025
Scores refreshed
November 12th, 2026
31
Entries on its own leaderboard, 19 models we track

4 models share the top score of 99.6% among models released by April 2025

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in April 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

31 entries. Scroll sideways for the rest.

  1. 99.6 deepseek-r1 (snapshot unspecified)
  2. 99.6 o1-preview
  3. 99.6 o3-mini
  4. 99.6 Claude 3.7 Sonnet
  5. 99.6 GPT-4.5
  6. 99.3 meta-llama/Meta-Llama-3.1-405B-Instruct
  7. 99.3 o1-mini
  8. 99.3 o1
  9. 99.3 Qwen2.5-Max
  10. 99.3 Gemini 2.5 Pro
  11. 98.9 deepseek/deepseek-chat (snapshot unspecified)
  12. 98.9 grok-2-1212
  13. 98.9 mistral-large (snapshot unspecified)
  14. 98.9 GPT-4o
  15. 98.9 Claude 3.5 Sonnet (October 2024)
  16. 98.9 Grok 3
  17. 98.5 gemini-2.0-flash (snapshot unspecified)
  18. 98.5 meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8
  19. 98.5 Grok-3 mini
  20. 98.5 GPT-4.1
  21. 98.1 Claude 3.5 Sonnet
  22. 98.1 GPT-4o
  23. 97.8 gemini-1.5-pro (snapshot unspecified)
  24. 97.8 gpt-4o-mini (snapshot unspecified)
  25. 97.4 meta-llama/Meta-Llama-3.1-70B-Instruct
  26. 97.4 Qwen2.5-72B
  27. 97.4 Llama 3.3 70B Instruct
  28. 97.4 Gemini 2.0 Pro
  29. 96.3 Claude 3.5 Haiku
  30. 95.9 gemini-1.5-flash (snapshot unspecified)
  31. 93.3 mistral-small (snapshot unspecified)
  1. 1 deepseek-r1 (snapshot unspecified) 99.6%
  2. 2 o1-preview 99.6%
  3. 3 o3-mini high 99.6%
  4. 4 Claude 3.7 Sonnet thinking; 16k 99.6%
  5. 5 GPT-4.5 99.6%
  6. 6 meta-llama/Meta-Llama-3.1-405B-Instruct 99.3%
  7. 7 o1-mini 99.3%
  8. 8 o1 medium 99.3%
  9. 9 Qwen2.5-Max 99.3%
  10. 10 Gemini 2.5 Pro Mar 2025 99.3%
  11. 11 deepseek/deepseek-chat (snapshot unspecified) 98.9%
  12. 12 grok-2-1212 98.9%
  13. 13 mistral-large (snapshot unspecified) 98.9%
  14. 14 GPT-4o Aug 2024 98.9%
  15. 15 Claude 3.5 Sonnet (October 2024) 98.9%
  16. 16 Grok 3 98.9%
  17. 17 gemini-2.0-flash (snapshot unspecified) thinking 98.5%
  18. 18 meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 98.5%
  19. 19 Grok-3 mini high 98.5%
  20. 20 GPT-4.1 98.5%
  21. 21 Claude 3.5 Sonnet 98.1%
  22. 22 GPT-4o Nov 2024 98.1%
  23. 23 gemini-1.5-pro (snapshot unspecified) 97.8%
  24. 24 gpt-4o-mini (snapshot unspecified) 97.8%
  25. 25 meta-llama/Meta-Llama-3.1-70B-Instruct 97.4%
  26. 26 Qwen2.5-72B 97.4%
  27. 27 Llama 3.3 70B Instruct 97.4%
  28. 28 Gemini 2.0 Pro 97.4%
  29. 29 Claude 3.5 Haiku 96.3%
  30. 30 gemini-1.5-flash (snapshot unspecified) 95.9%
  31. 31 mistral-small (snapshot unspecified) 93.3%

Source: Official GSM8K Platinum Initial 268 leaderboard

#EntryScoreSettingDateSource
1 deepseek-r1 (snapshot unspecified) 99.6% Official leaderboard
2 o1-preview 99.6% Sep 2024 Official leaderboard
3 o3-mini 99.6% high Jan 2025 Official leaderboard
4 Claude 3.7 Sonnet 99.6% thinking; 16k Feb 2025 Official leaderboard
5 GPT-4.5 99.6% Feb 2025 Official leaderboard
6 meta-llama/Meta-Llama-3.1-405B-Instruct 99.3% Official leaderboard
7 o1-mini 99.3% Sep 2024 Official leaderboard
8 o1 99.3% medium Dec 2024 Official leaderboard
9 Qwen2.5-Max 99.3% Jan 2025 Official leaderboard
10 Gemini 2.5 Pro Mar 2025 99.3% Mar 2025 Official leaderboard
11 deepseek/deepseek-chat (snapshot unspecified) 98.9% Official leaderboard
12 grok-2-1212 98.9% Official leaderboard
13 mistral-large (snapshot unspecified) 98.9% Official leaderboard
14 GPT-4o Aug 2024 98.9% Aug 2024 Official leaderboard
15 Claude 3.5 Sonnet (October 2024) 98.9% Oct 2024 Official leaderboard
16 Grok 3 98.9% Apr 2025 Official leaderboard
17 gemini-2.0-flash (snapshot unspecified) 98.5% thinking Official leaderboard
18 meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 98.5% Official leaderboard
19 Grok-3 mini 98.5% high Apr 2025 Official leaderboard
20 GPT-4.1 98.5% Apr 2025 Official leaderboard
21 Claude 3.5 Sonnet 98.1% Jun 2024 Official leaderboard
22 GPT-4o Nov 2024 98.1% Nov 2024 Official leaderboard
23 gemini-1.5-pro (snapshot unspecified) 97.8% Official leaderboard
24 gpt-4o-mini (snapshot unspecified) 97.8% Official leaderboard
25 meta-llama/Meta-Llama-3.1-70B-Instruct 97.4% Official leaderboard
26 Qwen2.5-72B 97.4% Sep 2024 Official leaderboard
27 Llama 3.3 70B Instruct 97.4% Dec 2024 Official leaderboard
28 Gemini 2.0 Pro 97.4% Feb 2025 Official leaderboard
29 Claude 3.5 Haiku 96.3% Oct 2024 Official leaderboard
30 gemini-1.5-flash (snapshot unspecified) 95.9% Official leaderboard
31 mistral-small (snapshot unspecified) 93.3% Official leaderboard

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Passed 85% in Feb 2025, by o1-preview
o1-preview 99.6%

Source: Official GSM8K Platinum Initial 268 leaderboard

Claude 3.5 Sonnet on the official leaderboard passed 85% in June 2024

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) o1-preview 0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) o1-preview

Source: Official GSM8K Platinum Initial 268 leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

Versions

Replaced by GSM8K-Platinum

More in Math

All 113 in Math
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • MathArena Apex Shortlist In use Models solve 47 hard final-answer problems that MathArena selected from 2025 competitions, where the best models scored about 50% at selection. Scored by accuracy over repeated attempts. GPT-5.5 98.4%