Benchmark Almanac

StarSkirmish Bench

GPT-6 Astra 57.3% leads since October 2026
Not in model cards

Each model gets one hour in its lab's own CLI to write a StarCraft: Brood War bot in C++. The score is its win rate against human-written bots.

v0.2 (October 15th, 2026) replaced v0.1 with new flagship models and first-party harnesses. Each model has five runs.

Made by
Kai McPheeters
Released
October 2026
Kind
Video games
Level
Expert
Scored
Ranked by matches
Answers
Code
Tasks
All public
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
9
Models scored on its own leaderboard

GPT-6 Astra leads with 57.3%

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

  1. 57.3 GPT-6 Astra
  2. 45.8 GPT-6.1 Sol
  3. 39.3 Gemini 4 Argon
  4. 38.6 Claude Opus 5.5
  5. 37.2 Claude Fable 5.1
  6. 25.8 Claude Sonnet 5.5
  7. 21.4 Muse Wonder Full
  8. 21.0 Claude Haiku 5.5
  9. 13.1 GPT-6 Luna
  1. 1 GPT-6 Astra max 57.3%
  2. 2 GPT-6.1 Sol max 45.8%
  3. 3 Gemini 4 Argon high 39.3%
  4. 4 Claude Opus 5.5 max 38.6%
  5. 5 Claude Fable 5.1 max 37.2%
  6. 6 Claude Sonnet 5.5 max 25.8%
  7. 7 Muse Wonder Full max 21.4%
  8. 8 Claude Haiku 5.5 max 21%
  9. 9 GPT-6 Luna max 13.1%

Source: Official StarSkirmish Bench leaderboard

#ModelScoreSettingDateSource
1 GPT-6 Astra 57.3% max Sep 2026 Official leaderboard
2 GPT-6.1 Sol 45.8% max Sep 2026 Official leaderboard
3 Gemini 4 Argon 39.3% high Sep 2026 Official leaderboard
4 Claude Opus 5.5 38.6% max Sep 2026 Official leaderboard
5 Claude Fable 5.1 37.2% max Sep 2026 Official leaderboard
6 Claude Sonnet 5.5 25.8% max Sep 2026 Official leaderboard
7 Muse Wonder Full 21.4% max Oct 2026 Official leaderboard
8 Claude Haiku 5.5 21% max Oct 2026 Official leaderboard
9 GPT-6 Luna 13.1% max Sep 2026 Official leaderboard

Win Rates vs. Practice Against Strong Bots

Each point is one model: the median, over its 5 runs, of the practice games its agent played in the hour against the 7 strongest roster bots, against the model's win rate. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

0% 20% 40% 60% 050100150200 Practice games against the 7 strongest roster bots (median run) (games) Win rate GPT-6 Luna (max): Win rate 13.1%, practice games against the 7 strongest roster bots (median run) 0 games Claude Sonnet 5.5 (max): Win rate 25.8%, practice games against the 7 strongest roster bots (median run) 22 games Claude Haiku 5.5 (max): Win rate 21%, practice games against the 7 strongest roster bots (median run) 0 games Muse Wonder Full (max): Win rate 21.4%, practice games against the 7 strongest roster bots (median run) 2 games Claude Fable 5.1 (max): Win rate 37.2%, practice games against the 7 strongest roster bots (median run) 0 games GPT-6 Astra (max): Win rate 57.3%, practice games against the 7 strongest roster bots (median run) 159 games Claude Opus 5.5 (max): Win rate 38.6%, practice games against the 7 strongest roster bots (median run) 18 games GPT-6.1 Sol (max): Win rate 45.8%, practice games against the 7 strongest roster bots (median run) 87.5 games Gemini 4 Argon (high): Win rate 39.3%, practice games against the 7 strongest roster bots (median run) 54 games GPT-6 Astra (max)GPT-6.1 Sol (max)Gemini 4 Argon (high)Claude Opus 5.5 (max)Claude Fable 5.1 (max) 0% 20% 40% 60% 050100150200 Practice games against the 7 strongest roster bots (median run) (games) Win rate GPT-6 Luna (max): Win rate 13.1%, practice games against the 7 strongest roster bots (median run) 0 games Claude Sonnet 5.5 (max): Win rate 25.8%, practice games against the 7 strongest roster bots (median run) 22 games Claude Haiku 5.5 (max): Win rate 21%, practice games against the 7 strongest roster bots (median run) 0 games Muse Wonder Full (max): Win rate 21.4%, practice games against the 7 strongest roster bots (median run) 2 games Claude Fable 5.1 (max): Win rate 37.2%, practice games against the 7 strongest roster bots (median run) 0 games GPT-6 Astra (max): Win rate 57.3%, practice games against the 7 strongest roster bots (median run) 159 games Claude Opus 5.5 (max): Win rate 38.6%, practice games against the 7 strongest roster bots (median run) 18 games GPT-6.1 Sol (max): Win rate 45.8%, practice games against the 7 strongest roster bots (median run) 87.5 games Gemini 4 Argon (high): Win rate 39.3%, practice games against the 7 strongest roster bots (median run) 54 games GPT-6 Astra (max)GPT-6.1 Sol (max)Gemini 4 Argon (high)Claude Opus 5.5 (max)Claude Fable 5.1 (max)
ModelSettingWin ratePractice games against the 7 strongest roster bots (median run)
GPT-6 Astra max 57.3% 159 games
GPT-6.1 Sol max 45.8% 87.5 games
Gemini 4 Argon high 39.3% 54 games
Claude Opus 5.5 max 38.6% 18 games
Claude Fable 5.1 max 37.2% 0 games
Claude Sonnet 5.5 max 25.8% 22 games
Muse Wonder Full max 21.4% 2 games
Claude Haiku 5.5 max 21% 0 games
GPT-6 Luna max 13.1% 0 games

The best score, 57.3%, costs $18 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% $0.10 $1.00 $10 $100 Cost per task, log scale GPT-6 Astra, max GPT-6.1 Sol, max Claude Haiku 5.5, max GPT-6 Luna, max 0% 20% 40% 60% $0.10 $1.00 $10 $100 Cost per task, log scale GPT-6 Astra, max GPT-6.1 Sol, max

Source: Official StarSkirmish Bench leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GPT-6 Astra max 57.3% $18
GPT-6.1 Sol max 45.8% $2.02
Gemini 4 Argon high 39.3% $3.63
Claude Opus 5.5 max 38.6% $5.12
Claude Fable 5.1 max 37.2% $7.78
Claude Sonnet 5.5 max 25.8% $2.38
Muse Wonder Full max 21.4% $5.87
Claude Haiku 5.5 max 21% $0.98
GPT-6 Luna max 13.1% $0.57

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard 1 leader
GPT-6 Astra 57.3%

Source: Official StarSkirmish Bench leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% Near the ceiling (85%) GPT-6 Astra Claude Fable 5.1 0% 20% 40% 60% 80% 100% Near the ceiling (85%) GPT-6 Astra Claude Fable 5.1

Source: Official StarSkirmish Bench leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Games and long tasks

All 82 in Games and long tasks
  • ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6 Astra 99.95%
  • Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Each game scores in-game progress or outcomes, and the board ranks models by their average rank across the games. Gemini 2.5 Pro 3.90
  • MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. Seed2.0 Pro 49%
  • TextQuests Only in system cards Agents play 25 classic interactive fiction games. The score is harmful actions per 100 steps, where lower is better. The board also reports game progress. Llama 4 Scout 7.00
  • GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. GPT-6.1 Terra 76.8%
  • Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%