Benchmark Almanac

Brood War Bench

GPT-6 Astra 100% leads since September 2026
Not in model cards

Coding agents play StarCraft: Brood War against each other in real time. Systems are ranked by win rate over matches, with cost per game.

Runs agent and model pairs at several effort levels.

Made by
Ben Swerdlow
Released
September 2026
Kind
Video games
Level
Expert
Scored
Ranked by matches
Answers
Agent task
Tasks
All public
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
September 2026
Scores refreshed
November 12th, 2026
9
Entries on its own leaderboard, 6 models we track

GPT-6 Astra leads with 100% among models released by September 2026

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in September 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 100.0 GPT-6 Astra
  2. 83.3 Claude Fable
  3. 72.2 GPT-5.6 Sol
  4. 66.7 Claude Opus 5
  5. 50.0 GPT-5.6 Luna
  6. 50.0 GPT-5.6 Terra
  7. 38.9 Claude Sonnet
  8. 11.1 Grok 4.6
  9. 0.0 Claude Haiku
  1. 1 GPT-6 Astra xhigh 100%
  2. 2 Claude Fable 83.3%
  3. 3 GPT-5.6 Sol medium 72.2%
  4. 4 Claude Opus 5 66.7%
  5. 5 GPT-5.6 Luna low 50%
  6. 6 GPT-5.6 Terra xhigh 50%
  7. 7 Claude Sonnet 38.9%
  8. 8 Grok 4.6 xhigh 11.1%
  9. 9 Claude Haiku 0%

Source: Official Brood War Bench leaderboard

#EntryScoreSettingDateSource
1 GPT-6 Astra 100% xhigh, Codex Sep 2026 Official leaderboard
2 Claude Fable 83.3% Official leaderboard
3 GPT-5.6 Sol 72.2% medium, Codex Jul 2026 Official leaderboard
4 Claude Opus 5 66.7% Jul 2026 Official leaderboard
5 GPT-5.6 Luna 50% low, Codex Jul 2026 Official leaderboard
6 GPT-5.6 Terra 50% xhigh, Codex Jul 2026 Official leaderboard
7 Claude Sonnet 38.9% Official leaderboard
8 Grok 4.6 11.1% xhigh Aug 2026 Official leaderboard
9 Claude Haiku 0% Official leaderboard

The best score, 100%, costs $11 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 $100 Cost per task, log scale GPT-6 Astra, xhigh GPT-5.6 Sol, medium GPT-5.6 Sol, xhigh GPT-5.6 Luna, low GPT-5.6 Luna, xhigh 0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 $100 Cost per task, log scale GPT-6 Astra, xhigh GPT-5.6 Sol, medium

Source: Official Brood War Bench leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GPT-6 Astra xhigh Codex 100% $11
GPT-6 Astra medium Codex 88.9% $15
GPT-6 Astra low Codex 77.8% $21
GPT-5.6 Sol medium Codex 72.2% $5.12
GPT-5.6 Sol low Codex 66.7% $9.23
Claude Opus 5 66.7% $21
GPT-5.6 Sol xhigh Codex 61.1% $3.23
GPT-5.6 Luna low Codex 50% $0.42
GPT-5.6 Terra xhigh Codex 50% $2.10
GPT-5.6 Terra medium Codex 44.4% $3.15
GPT-5.6 Terra low Codex 44.4% $4.65
GPT-5.6 Luna xhigh Codex 38.9% $0.16
GPT-5.6 Luna medium Codex 33.3% $0.30
Grok 4.6 xhigh 11.1% $0.66
Grok 4.6 medium 5.60% $0.79
Grok 4.6 low 0% $1.26

2 models have held number one on the official leaderboard since its release in September 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Passed 85% in Sep 2026, by GPT-6 Astra
GPT-6 Astra 100%

Source: Official Brood War Bench leaderboard

GPT-6 Astra on the official leaderboard passed 85% in September 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% Near the ceiling (85%) GPT-6 Astra GPT-5.6 Sol 0% 20% 40% 60% 80% 100% Near the ceiling (85%) GPT-6 Astra GPT-5.6 Sol

Source: Official Brood War Bench leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Games and long tasks

All 82 in Games and long tasks
  • ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6 Astra 99.95%
  • Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Each game scores in-game progress or outcomes, and the board ranks models by their average rank across the games. Gemini 2.5 Pro 3.90
  • MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. Seed2.0 Pro 49%
  • TextQuests Only in system cards Agents play 25 classic interactive fiction games. The score is harmful actions per 100 steps, where lower is better. The board also reports game progress. Llama 4 Scout 7.00
  • GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. GPT-6.1 Terra 76.8%
  • Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%