Benchmark Almanac

Step Race

GPT-5 5 leads since August 2025
Not in model cards

Several models talk in public, then secretly pick moves in a race where colliding moves cancel. The game tests cooperation and deception.

The repo was created on January 21st, 2025 and updated on December 9th, 2025.

Made by
Lech Mazur
Released
January 2025
Kind
Multi-agent games
Level
Everyday
Scored
Ranked by matches
Answers
Open-ended
Tasks
All public
Strongest model tested
Gemini 3 Pro #55 on the capability index
Last model added
December 2025
Scores refreshed
November 12th, 2026
68
Entries on its own leaderboard, 59 models we track

GPT-5 leads with 5 among models released by December 2025

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in December 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

68 entries. Scroll sideways for the rest.

  1. 5 GPT-5
  2. 5 o3
  3. 5 GPT-5.1
  4. 5 Gemini 3 Pro
  5. 4 o1
  6. 4 Gemini 2.5 Flash
  7. 4 Gemini 2.5 Pro
  8. 4 Grok 4.1 Fast
  9. 4 Gemini 2.5 Pro
  10. 4 DeepSeek-V3.2-Exp
  11. 4 o3-mini
  12. 4 DeepSeek-V3.1
  13. 3 Grok 4
  14. 3 Claude Sonnet 4.5
  15. 3 Kimi K2 Thinking
  16. 3 Grok-3 mini
  17. 3 QwQ-32B
  18. 3 DeepSeek-R1
  19. 3 DeepSeek-R1
  20. 3 Claude Opus 4.5
  21. 3 Claude Opus 4.1
  22. 3 Qwen3-235B-A22B
  23. 3 Claude Sonnet 4
  24. 3 Silent Greedy
  25. 3 GPT-5 mini
  26. 3 Claude 3.7 Sonnet
  27. 3 o4-mini
  28. 3 Silent Closer
  29. 3 Silent EVMax
  30. 2 Silent Collision Tracker
  31. 2 Qwen3-235B-A22B-Instruct
  32. 2 Silent Blocker
  33. 2 o1-mini
  34. 2 Claude Opus 4
  35. 2 Gemini 2.0 Flash Thinking
  36. 2 GLM-4.6
  37. 2 Qwen3-30B-A3B
  38. 2 GLM-4.5
  39. 2 Claude 3.5 Sonnet (October 2024)
  40. 2 gpt-oss-120b
  41. 2 GPT-4o
  42. 2 Qwen3-Max
  43. 1 Qwen2.5-Max
  44. 1 Gemini 2.0 Pro
  45. 1 Mistral Medium 3.1
  46. 1 Mistral Large 3
  47. 1 Llama 3.3 70B
  48. 1 Grok 3
  49. 1 QwQ-32B-Preview
  50. 1 Qwen2.5-72B
  51. 1 DeepSeek-V3
  52. 1 Llama 4 Maverick
  53. 1 Mistral Large 2
  54. 1 Kimi K2
  55. 1 GPT-4o
  56. 1 GPT-4o Feb 2025
  57. 1 Gemini 2.0 Flash
  58. 1 Gemini 1.5 Pro
  59. 1 GPT-4.5
  60. 1 Gemini 1.5 Flash
  61. 1 Llama 3.1-405B
  62. 1 Grok-2
  63. 1 Gemini 2.0 Flash Think Exp Old
  64. 1 GPT-4o mini
  65. 1 Claude 3.5 Haiku
  66. 1 Gemma 2 27B
  67. 1 Silent UCB
  68. 1 Silent Random
  1. 1 GPT-5 medium 5
  2. 2 o3 medium 5
  3. 3 GPT-5.1 medium 5
  4. 4 Gemini 3 Pro 5
  5. 5 o1 medium 4
  6. 6 Gemini 2.5 Flash Jun 2025 4
  7. 7 Gemini 2.5 Pro Jun 2025 4
  8. 8 Grok 4.1 Fast reasoning 4
  9. 9 Gemini 2.5 Pro Mar 2025 4
  10. 10 DeepSeek-V3.2-Exp 4
  11. 11 o3-mini high 4
  12. 12 DeepSeek-V3.1 thinking 4
  13. 13 Grok 4 3
  14. 14 Claude Sonnet 4.5 thinking 16K 3
  15. 15 Kimi K2 Thinking 3
  16. 16 Grok-3 mini high 3
  17. 17 QwQ-32B 3
  18. 18 DeepSeek-R1 May 2025 3
  19. 19 DeepSeek-R1 3
  20. 20 Claude Opus 4.5 no reasoning 3
  21. 21 Claude Opus 4.1 thinking 16K 3
  22. 22 Qwen3-235B-A22B 3
  23. 23 Claude Sonnet 4 thinking 16K 3
  24. 24 Silent Greedy 3
  25. 25 GPT-5 mini medium 3
  26. 26 Claude 3.7 Sonnet thinking 16K 3
  27. 27 o4-mini medium 3
  28. 28 Silent Closer 3
  29. 29 Silent EVMax 3
  30. 30 Silent Collision Tracker 2
  31. 31 Qwen3-235B-A22B-Instruct Jul 2025 2
  32. 32 Silent Blocker 2
  33. 33 o1-mini 2
  34. 34 Claude Opus 4 thinking 16K 2
  35. 35 Gemini 2.0 Flash Thinking Jan 2025 2
  36. 36 GLM-4.6 2
  37. 37 Qwen3-30B-A3B 2
  38. 38 GLM-4.5 2
  39. 39 Claude 3.5 Sonnet (October 2024) 2
  40. 40 gpt-oss-120b 2
  41. 41 GPT-4o Mar 2025 2
  42. 42 Qwen3-Max 2
  43. 43 Qwen2.5-Max 1
  44. 44 Gemini 2.0 Pro 1
  45. 45 Mistral Medium 3.1 1
  46. 46 Mistral Large 3 1
  47. 47 Llama 3.3 70B 1
  48. 48 Grok 3 no reasoning 1
  49. 49 QwQ-32B-Preview 1
  50. 50 Qwen2.5-72B 1
  51. 51 DeepSeek-V3 1
  52. 52 Llama 4 Maverick 1
  53. 53 Mistral Large 2 Nov 2024 1
  54. 54 Kimi K2 Jul 2025 1
  55. 55 GPT-4o Aug 2024 1
  56. 56 GPT-4o Feb 2025 1
  57. 57 Gemini 2.0 Flash Dec 2024 1
  58. 58 Gemini 1.5 Pro Sep 2024 1
  59. 59 GPT-4.5 1
  60. 60 Gemini 1.5 Flash Sep 2024 1
  61. 61 Llama 3.1-405B 1
  62. 62 Grok-2 Dec 2024 1
  63. 63 Gemini 2.0 Flash Think Exp Old 1
  64. 64 GPT-4o mini 1
  65. 65 Claude 3.5 Haiku 1
  66. 66 Gemma 2 27B 1
  67. 67 Silent UCB 1
  68. 68 Silent Random 1

Source: Official Step Race leaderboard

#EntryScoreSettingDateSource
1 GPT-5 5 medium Aug 2025 Official leaderboard
2 o3 5 medium Apr 2025 Official leaderboard
3 GPT-5.1 5 medium Nov 2025 Official leaderboard
4 Gemini 3 Pro 5 Nov 2025 Official leaderboard
5 o1 4 medium Dec 2024 Official leaderboard
6 Gemini 2.5 Flash Jun 2025 4 Jun 2025 Official leaderboard
7 Gemini 2.5 Pro Jun 2025 4 Jun 2025 Official leaderboard
8 Grok 4.1 Fast 4 reasoning Nov 2025 Official leaderboard
9 Gemini 2.5 Pro Mar 2025 4 Mar 2025 Official leaderboard
10 DeepSeek-V3.2-Exp 4 Sep 2025 Official leaderboard
11 o3-mini 4 high Jan 2025 Official leaderboard
12 DeepSeek-V3.1 4 thinking Aug 2025 Official leaderboard
13 Grok 4 3 Jul 2025 Official leaderboard
14 Claude Sonnet 4.5 3 thinking 16K Sep 2025 Official leaderboard
15 Kimi K2 Thinking 3 Nov 2025 Official leaderboard
16 Grok-3 mini 3 high Apr 2025 Official leaderboard
17 QwQ-32B 3 Mar 2025 Official leaderboard
18 DeepSeek-R1 May 2025 3 May 2025 Official leaderboard
19 DeepSeek-R1 3 Jan 2025 Official leaderboard
20 Claude Opus 4.5 3 no reasoning Nov 2025 Official leaderboard
21 Claude Opus 4.1 3 thinking 16K Aug 2025 Official leaderboard
22 Qwen3-235B-A22B 3 Apr 2025 Official leaderboard
23 Claude Sonnet 4 3 thinking 16K May 2025 Official leaderboard
24 Silent Greedy 3 Official leaderboard
25 GPT-5 mini 3 medium Aug 2025 Official leaderboard
26 Claude 3.7 Sonnet 3 thinking 16K Feb 2025 Official leaderboard
27 o4-mini 3 medium Apr 2025 Official leaderboard
28 Silent Closer 3 Official leaderboard
29 Silent EVMax 3 Official leaderboard
30 Silent Collision Tracker 2 Official leaderboard
31 Qwen3-235B-A22B-Instruct Jul 2025 2 Jul 2025 Official leaderboard
32 Silent Blocker 2 Official leaderboard
33 o1-mini 2 Sep 2024 Official leaderboard
34 Claude Opus 4 2 thinking 16K May 2025 Official leaderboard
35 Gemini 2.0 Flash Thinking Jan 2025 2 Jan 2025 Official leaderboard
36 GLM-4.6 2 Sep 2025 Official leaderboard
37 Qwen3-30B-A3B 2 Apr 2025 Official leaderboard
38 GLM-4.5 2 Aug 2025 Official leaderboard
39 Claude 3.5 Sonnet (October 2024) 2 Oct 2024 Official leaderboard
40 gpt-oss-120b 2 Aug 2025 Official leaderboard
41 GPT-4o Mar 2025 2 Mar 2025 Official leaderboard
42 Qwen3-Max 2 Sep 2025 Official leaderboard
43 Qwen2.5-Max 1 Jan 2025 Official leaderboard
44 Gemini 2.0 Pro 1 Feb 2025 Official leaderboard
45 Mistral Medium 3.1 1 Aug 2025 Official leaderboard
46 Mistral Large 3 1 Dec 2025 Official leaderboard
47 Llama 3.3 70B 1 Dec 2024 Official leaderboard
48 Grok 3 1 no reasoning Apr 2025 Official leaderboard
49 QwQ-32B-Preview 1 Nov 2024 Official leaderboard
50 Qwen2.5-72B 1 Sep 2024 Official leaderboard
51 DeepSeek-V3 1 Dec 2024 Official leaderboard
52 Llama 4 Maverick 1 Apr 2025 Official leaderboard
53 Mistral Large 2 Nov 2024 1 Nov 2024 Official leaderboard
54 Kimi K2 Jul 2025 1 Jul 2025 Official leaderboard
55 GPT-4o Aug 2024 1 Aug 2024 Official leaderboard
56 GPT-4o Feb 2025 1 Official leaderboard
57 Gemini 2.0 Flash Dec 2024 1 Dec 2024 Official leaderboard
58 Gemini 1.5 Pro Sep 2024 1 Sep 2024 Official leaderboard
59 GPT-4.5 1 Feb 2025 Official leaderboard
60 Gemini 1.5 Flash Sep 2024 1 Aug 2024 Official leaderboard
61 Llama 3.1-405B 1 Jul 2024 Official leaderboard
62 Grok-2 Dec 2024 1 Dec 2024 Official leaderboard
63 Gemini 2.0 Flash Think Exp Old 1 Official leaderboard
64 GPT-4o mini 1 Jul 2024 Official leaderboard
65 Claude 3.5 Haiku 1 Oct 2024 Official leaderboard
66 Gemma 2 27B 1 Jun 2024 Official leaderboard
67 Silent UCB 1 Official leaderboard
68 Silent Random 1 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Grok 4.1 Fast
non-reasoni… reasoning
4
o3-mini
medium high
4
DeepSeek-V3.1
non-thinking thinking
4
Claude Opus 4.1
no reasoning thinking 16K
3
Claude Sonnet 4
no reasoning thinking 16K
3
Claude 3.7 Sonnet
thinking 16K
3
Claude Opus 4
no reasoning thinking 16K
2

Source: Official Step Race leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GPT-5 medium 5
o3 medium 5
GPT-5.1 medium 5
Gemini 3 Pro 5
o1 medium 4
Gemini 2.5 Flash Jun 2025 4
Gemini 2.5 Pro Jun 2025 4
Grok 4.1 Fast reasoning 4
Gemini 2.5 Pro Mar 2025 4
DeepSeek-V3.2-Exp 4
o3-mini high 4
DeepSeek-V3.1 thinking 4
Grok 4 3
o3-mini medium 3
Claude Sonnet 4.5 thinking 16K 3
Kimi K2 Thinking 3
Grok-3 mini high 3
QwQ-32B 3
DeepSeek-R1 May 2025 3
DeepSeek-V3.1 non-thinking 3
DeepSeek-R1 3
Claude Opus 4.5 no reasoning 3
Claude Opus 4.1 thinking 16K 3
Qwen3-235B-A22B 3
Claude Sonnet 4 thinking 16K 3
GPT-5 mini medium 3
Claude 3.7 Sonnet thinking 16K 3
o4-mini medium 3
Qwen3-235B-A22B-Instruct Jul 2025 2
o1-mini 2
Claude Opus 4 thinking 16K 2
Gemini 2.0 Flash Thinking Jan 2025 2
GLM-4.6 2
Qwen3-30B-A3B 2
Claude Sonnet 4 no reasoning 2
Grok 4.1 Fast non-reasoning 2
GLM-4.5 2
Claude 3.5 Sonnet (October 2024) 2
gpt-oss-120b 2
Claude Opus 4.1 no reasoning 2
GPT-4o Mar 2025 2
Qwen3-Max 2
Claude Opus 4 no reasoning 2
Qwen2.5-Max 1
Gemini 2.0 Pro 1
Mistral Medium 3.1 1
Mistral Large 3 1
Llama 3.3 70B 1
Grok 3 no reasoning 1
QwQ-32B-Preview 1
Qwen2.5-72B 1
Claude 3.7 Sonnet 1
DeepSeek-V3 1
Llama 4 Maverick 1
Mistral Large 2 Nov 2024 1
Kimi K2 Jul 2025 1
GPT-4o Aug 2024 1
Gemini 2.0 Flash Dec 2024 1
Gemini 1.5 Pro Sep 2024 1
GPT-4.5 1
Gemini 1.5 Flash Sep 2024 1
Llama 3.1-405B 1
Grok-2 Dec 2024 1
GPT-4o mini 1
Claude 3.5 Haiku 1
Gemma 2 27B 1

3 models have held number one on the official leaderboard since its release in January 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Not tested since Dec 2025
GPT-5 5

Source: Official Step Race leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0 2 4 6 2024 2025 2026 0 2 4 6 2024 2025 2026

Source: Official Step Race leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Games and long tasks

All 82 in Games and long tasks
  • ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6 Astra 99.95%
  • Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Each game scores in-game progress or outcomes, and the board ranks models by their average rank across the games. Gemini 2.5 Pro 3.90
  • MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. Seed2.0 Pro 49%
  • TextQuests Only in system cards Agents play 25 classic interactive fiction games. The score is harmful actions per 100 steps, where lower is better. The board also reports game progress. Llama 4 Scout 7.00
  • GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. GPT-6.1 Terra 76.8%
  • Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%