Benchmark Almanac

ArtifactArena

GPT-6 Astra 1,539 leads since September 2026
Not in model cards

Models design robot bodies, controllers and strategy for a simulated sumo-style arena. Artifacts fight head-to-head and models get Elo ratings.

Built to resist saturation: new designs face the best existing robots, and community bots can be submitted. Three harnesses: one-shot, verifier-guided refinement, open-ended design lab. Models did worse with full autonomy than with verifier-guided refinement. No arXiv ID yet.

Released
September 2026
Kind
Simulated robots
Level
Expert
Scored
Ranked by matches
Answers
Code
Tasks
All public
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
September 2026
Scores refreshed
November 12th, 2026
21
Models scored on its own leaderboard

GPT-6 Astra leads with 1,539 among models released by September 2026

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best.

The newest model with a score here came out in September 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

21 models. Scroll sideways for the rest. Bars start at 600.

  1. 1,539 GPT-6 Astra
  2. 1,310 Claude Fable 5.1
  3. 1,223 Gemini 3.8 Flash
  4. 1,189 Claude Fable 5
  5. 1,179 GPT-5.4
  6. 1,173 GPT-5.3 Codex
  7. 1,165 GPT-5.6 Sol
  8. 1,164 Kimi K3
  9. 1,136 Grok 4.6
  10. 1,093 GPT-5.6 Terra
  11. 1,089 Gemini 3.1 Pro
  12. 1,055 Qwen3.8-2.4T-A95B
  13. 1,051 GLM-5.2
  14. 986 Claude Sonnet 5
  15. 984 GPT-5.6 Luna
  16. 978 GLM-5.3
  17. 967 GPT-5.5
  18. 957 Grok 4.5
  19. 929 Claude Opus 4.8
  20. 765 DeepSeek-V4-Pro
  21. 745 Grok 4.20

Bars start at 600.

  1. 1 GPT-6 Astra 1,539
  2. 2 Claude Fable 5.1 high 1,310
  3. 3 Gemini 3.8 Flash high 1,223
  4. 4 Claude Fable 5 high 1,189
  5. 5 GPT-5.4 1,179
  6. 6 GPT-5.3 Codex high 1,173
  7. 7 GPT-5.6 Sol 1,165
  8. 8 Kimi K3 high 1,164
  9. 9 Grok 4.6 high 1,136
  10. 10 GPT-5.6 Terra 1,093
  11. 11 Gemini 3.1 Pro high 1,089
  12. 12 Qwen3.8-2.4T-A95B thinking 1,055
  13. 13 GLM-5.2 high 1,051
  14. 14 Claude Sonnet 5 high 986
  15. 15 GPT-5.6 Luna 984
  16. 16 GLM-5.3 high 978
  17. 17 GPT-5.5 967
  18. 18 Grok 4.5 high 957
  19. 19 Claude Opus 4.8 high 929
  20. 20 DeepSeek-V4-Pro thinking 765
  21. 21 Grok 4.20 reasoning 745

Source: Official ArtifactArena leaderboard

#ModelScoreSettingDateSource
1 GPT-6 Astra 1,539 Verifier-Grounded Refinement Sep 2026 Official leaderboard
2 Claude Fable 5.1 1,310 high, Verifier-Grounded Refinement Sep 2026 Official leaderboard
3 Gemini 3.8 Flash 1,223 high, Verifier-Grounded Refinement Sep 2026 Official leaderboard
4 Claude Fable 5 1,189 high, Design Lab Jun 2026 Official leaderboard
5 GPT-5.4 1,179 Sampling Mar 2026 Official leaderboard
6 GPT-5.3 Codex 1,173 high, Verifier-Grounded Refinement Feb 2026 Official leaderboard
7 GPT-5.6 Sol 1,165 Sampling Jul 2026 Official leaderboard
8 Kimi K3 1,164 high, Verifier-Grounded Refinement Jul 2026 Official leaderboard
9 Grok 4.6 1,136 high, Verifier-Grounded Refinement Aug 2026 Official leaderboard
10 GPT-5.6 Terra 1,093 Verifier-Grounded Refinement Jul 2026 Official leaderboard
11 Gemini 3.1 Pro 1,089 high, Sampling Feb 2026 Official leaderboard
12 Qwen3.8-2.4T-A95B 1,055 thinking, Verifier-Grounded Refinement Aug 2026 Official leaderboard
13 GLM-5.2 1,051 high, Verifier-Grounded Refinement Jun 2026 Official leaderboard
14 Claude Sonnet 5 986 high, Verifier-Grounded Refinement Jun 2026 Official leaderboard
15 GPT-5.6 Luna 984 Verifier-Grounded Refinement Jul 2026 Official leaderboard
16 GLM-5.3 978 high, Design Lab Aug 2026 Official leaderboard
17 GPT-5.5 967 Sampling Apr 2026 Official leaderboard
18 Grok 4.5 957 high, Verifier-Grounded Refinement Jul 2026 Official leaderboard
19 Claude Opus 4.8 929 high, Verifier-Grounded Refinement May 2026 Official leaderboard
20 DeepSeek-V4-Pro 765 thinking, Verifier-Grounded Refinement Apr 2026 Official leaderboard
21 Grok 4.20 745 reasoning, Verifier-Grounded Refinement Feb 2026 Official leaderboard

2 models have held number one on the official leaderboard since its release in September 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic OpenAI
Own leaderboard Not tested since Sep 2026
GPT-6 Astra 1,539

Source: Official ArtifactArena leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
500 750 1,000 1,250 1,500 2026 500 750 1,000 1,250 1,500 2026

Source: Official ArtifactArena leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Robotics

All 104 in Robotics
  • ERQA In use 400 multiple-choice visual questions from robot and egocentric footage on spatial, trajectory, action and state reasoning. Accuracy. Gemini Robotics ER 2 78.5%
  • RoboDojo In use Generalist manipulation policies run 42 Isaac Sim tasks and 18 remote real-robot tasks. Scored by task score and success rate. GPT-6 Astra 29%
  • RoboLab In use Zero-shot DROID Franka policies in Isaac Lab on 120 tabletop tasks at three instruction levels, 10 episodes each. Success rate. FLUX 3 Action 42.9%
  • Progress classification (Google DeepMind) In use Models assign each frame of a robot video to one of five task-progress bands from 0 to 100 percent. Scored by accuracy. Gemini Robotics ER 2 57.4%
  • Human proximity (Google DeepMind robotics) In use Models judge from robot camera views whether a human is within 1 meter, a spatial awareness check for safe robot behavior. Scored by accuracy. Gemini Robotics ER 2 93%
  • Moment finding (Google DeepMind) In use Models find the moment in a robot video when an event happens. Scored by accuracy and mean absolute time distance in seconds. Gemini Robotics ER 2 91.3%