Benchmark Almanac

PokéBench: Raw Vision + Buttons

Claude Fable 5.1 10.0 leads since October 2026
Not in model cards

Models play Pokémon Red from screenshots using eight Game Boy buttons. Game memory verifies ten milestones, with fewer turns breaking ties.

The maker ranks 19 models using prompt rvb4. Each model gets up to 1,000 turns to earn the Boulder Badge.

Made by
Cody Harris
Released
October 2026
Kind
Video games
Level
Everyday
Scored
Checked automatically
Answers
Agent task
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Screens → Actions
Interaction
Agentic
Environment
Pokémon Red and Blue
Items from
Written by experts
Refresh
One fixed set
Run by
Anyone
In views
Games, Agents
Strongest model tested
Claude Fable 5.2 #2 on the capability index
Last model added
November 2026
Scores refreshed
November 20th, 2026
19
Entries on its own leaderboard, 18 models we track

8 models share the top score of 10.0 8 models share the top score of 10.0

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

  1. 1 Claude Fable 5.1 rvb4, harness v2, high 10.0
  2. 2 GPT-6 Astra rvb4, harness v2, high 10.0
  3. 3 Claude Opus 5.5 rvb4, harness v2, high 10.0
  4. 4 Claude Sonnet 5.5 rvb4, harness v2, high 10.0
  5. 5 GPT-6.1 Sol rvb4, harness v2, high 10.0
  6. 6 Gemini 4 Argon rvb4, harness v2, high 10.0
  7. 7 GPT-6.1 Terra rvb4, harness v2, high 10.0
  8. 8 Claude Fable 5.2 rvb4, harness v2, high 10.0
  9. 9 Kimi K3.1 Pro rvb4, harness v2, high 7.00
  10. 10 DeepSeek V4.1 Flash 1017 rvb4, harness v2, high 5.00
  11. 11 Tencent Hy4 rvb4, harness v2, high 5.00
  12. 12 Claude Haiku 5.5 rvb4, harness v2, high 4.00
  13. 13 Qwen 3.8 27B rvb4, harness v2, high 1.00
  14. 14 GLM-5.3-Flash rvb4, harness v2, high 1.00
  15. 15 GPT-6 Luna rvb4, harness v2, high 1.00
  16. 16 Gemma 4 26B A4B rvb4, harness v2, high 0.00
  17. 17 Gemma 4 31B IT rvb4, harness v2, high 0.00
  18. 18 Muse Glimmer rvb4, harness v2, high 0.00
  19. 19 Qwen3-VL 30B rvb4, harness v2, high 0.00

Source: Official PokéBench: Raw Vision + Buttons leaderboard

#EntryScoreSettingReleasedRun onLinkSource
1 Claude Fable 5.1 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
2 GPT-6 Astra 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
3 Claude Opus 5.5 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
4 Claude Sonnet 5.5 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
5 GPT-6.1 Sol 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
6 Gemini 4 Argon 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
7 GPT-6.1 Terra 10.0 rvb4, harness v2, high Oct 2026 Oct 2026 Official leaderboard
8 Claude Fable 5.2 10.0 rvb4, harness v2, high Nov 2026 Nov 2026 youtube.com Official leaderboard
9 Kimi K3.1 Pro 7.00 rvb4, harness v2, high Oct 2026 Oct 2026 youtube.com Official leaderboard
10 DeepSeek V4.1 Flash 1017 5.00 rvb4, harness v2, high Oct 2026 Oct 2026 youtube.com Official leaderboard
11 Tencent Hy4 5.00 rvb4, harness v2, high Nov 2026 Nov 2026 Official leaderboard
12 Claude Haiku 5.5 4.00 rvb4, harness v2, high Oct 2026 Oct 2026 youtube.com Official leaderboard
13 Qwen 3.8 27B 1.00 rvb4, harness v2, high Aug 2026 Oct 2026 youtube.com Official leaderboard
14 GLM-5.3-Flash 1.00 rvb4, harness v2, high Aug 2026 Oct 2026 youtube.com Official leaderboard
15 GPT-6 Luna 1.00 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
16 Gemma 4 26B A4B 0.00 rvb4, harness v2, high Apr 2026 Oct 2026 youtube.com Official leaderboard
17 Gemma 4 31B IT 0.00 rvb4, harness v2, high Apr 2026 Oct 2026 youtube.com Official leaderboard
18 Muse Glimmer 0.00 rvb4, harness v2, high Aug 2026 Oct 2026 youtube.com Official leaderboard
19 Qwen3-VL 30B 0.00 rvb4, harness v2, high Oct 2026 youtube.com Official leaderboard
#ModelScoreSettingReleasedRun onLinkSource
1 Claude Fable 5.1 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
2 GPT-6 Astra 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
3 Claude Opus 5.5 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
4 Claude Sonnet 5.5 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
5 GPT-6.1 Sol 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
6 Gemini 4 Argon 10.0 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
7 GPT-6.1 Terra 10.0 rvb4, harness v2, high Oct 2026 Oct 2026 Official leaderboard
8 Claude Fable 5.2 10.0 rvb4, harness v2, high Nov 2026 Nov 2026 youtube.com Official leaderboard
9 Kimi K3.1 Pro 7.00 rvb4, harness v2, high Oct 2026 Oct 2026 youtube.com Official leaderboard
10 DeepSeek V4.1 Flash 1017 5.00 rvb4, harness v2, high Oct 2026 Oct 2026 youtube.com Official leaderboard
11 Tencent Hy4 5.00 rvb4, harness v2, high Nov 2026 Nov 2026 Official leaderboard
12 Claude Haiku 5.5 4.00 rvb4, harness v2, high Oct 2026 Oct 2026 youtube.com Official leaderboard
13 Qwen 3.8 27B 1.00 rvb4, harness v2, high Aug 2026 Oct 2026 youtube.com Official leaderboard
14 GLM-5.3-Flash 1.00 rvb4, harness v2, high Aug 2026 Oct 2026 youtube.com Official leaderboard
15 GPT-6 Luna 1.00 rvb4, harness v2, high Sep 2026 Oct 2026 youtube.com Official leaderboard
16 Gemma 4 26B A4B 0.00 rvb4, harness v2, high Apr 2026 Oct 2026 youtube.com Official leaderboard
17 Gemma 4 31B IT 0.00 rvb4, harness v2, high Apr 2026 Oct 2026 youtube.com Official leaderboard
18 Muse Glimmer 0.00 rvb4, harness v2, high Aug 2026 Oct 2026 youtube.com Official leaderboard

Rankings by part

We read 10 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as Left the house, Got a starter and Reached Route 1. Gemini 4 Argon leads 6 of them, and Claude Fable 5.2 leads the other 4. Arrows show how far a model moved from its overall rank among the models we track.

Left the house

15 models we track have a score here.

  1. 1 Claude Fable 5.2 ↑7 12.0
  2. 2 GPT-6.1 Terra ↑5 13.0
  3. 3 GPT-6.1 Sol ↑2 14.0
  4. 4 Claude Opus 5.5 ↓1 15.0
  5. 5 Gemini 4 Argon ↑1 15.0
  6. 6 Claude Sonnet 5.5 ↓2 17.0
  7. 7 GPT-6 Astra ↓5 19.0
  8. 8 Tencent Hy4 ↑3 19.0
  9. 9 Claude Fable 5.1 ↓8 22.0
  10. 10 Kimi K3.1 Pro ↓1 25.0

The best score, 10.0, costs $7.62 for the full run

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Anthropic DeepSeek Cost frontier One model at each effort level
0.00 2.00 4.00 6.00 8.00 10.0 $1.00 $2.00 $5.00 $10 $20 $50 $100 Cost of the full run, log scale Claude Opus 5.5, rvb4, harness v2, high DeepSeek V4.1 Flash 1017, rvb4, harness v2, high Claude Haiku 5.5, rvb4, harness v2, high 0.00 2.00 4.00 6.00 8.00 10.0 $1.00 $10 $100 Cost of the full run, log scale Claude Opus 5.5, rvb4, harness v2, high DeepSeek V4.1 Flash 1017, rvb4, harness v2, high

Source: Official PokéBench: Raw Vision + Buttons leaderboard

ModelEffortHarnessScoreCost of the full runOutput tokens
Claude Fable 5.1 rvb4, harness v2, high 10.0 $50 762,779
GPT-6 Astra rvb4, harness v2, high 10.0 $58 54,495
Claude Opus 5.5 rvb4, harness v2, high 10.0 $7.62 224,834
Claude Sonnet 5.5 rvb4, harness v2, high 10.0 $14 515,015
GPT-6.1 Sol rvb4, harness v2, high 10.0 $33 154,134
Gemini 4 Argon rvb4, harness v2, high 10.0 $11 264,789
GPT-6.1 Terra rvb4, harness v2, high 10.0 $14 113,814
Claude Fable 5.2 rvb4, harness v2, high 10.0 $11 208,525
Kimi K3.1 Pro rvb4, harness v2, high 7.00 $15 800,087
DeepSeek V4.1 Flash 1017 rvb4, harness v2, high 5.00 $1.87 2,787,675
Tencent Hy4 rvb4, harness v2, high 5.00 $33 22,199,749
Claude Haiku 5.5 rvb4, harness v2, high 4.00 $1.42 1,096,238
Qwen 3.8 27B rvb4, harness v2, high 1.00 209,979
GLM-5.3-Flash rvb4, harness v2, high 1.00 $4.57 1,093,386
GPT-6 Luna rvb4, harness v2, high 1.00 $3.59 491,193
Gemma 4 26B A4B rvb4, harness v2, high 0.00 224,504
Gemma 4 31B IT rvb4, harness v2, high 0.00 75,162
Muse Glimmer rvb4, harness v2, high 0.00 857,362

The best score, 10.0, costs $7.62 for the full run

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Anthropic DeepSeek Cost frontier One model at each effort level
0.00 2.00 4.00 6.00 8.00 10.0 $1.00 $2.00 $5.00 $10 $20 $50 $100 Cost of the full run, log scale Claude Opus 5.5, rvb4, harness v2, high DeepSeek V4.1 Flash 1017, rvb4, harness v2, high Claude Haiku 5.5, rvb4, harness v2, high 0.00 2.00 4.00 6.00 8.00 10.0 $1.00 $10 $100 Cost of the full run, log scale Claude Opus 5.5, rvb4, harness v2, high DeepSeek V4.1 Flash 1017, rvb4, harness v2, high

Source: Official PokéBench: Raw Vision + Buttons leaderboard

ModelEffortHarnessScoreCost of the full runOutput tokens
Claude Fable 5.1 rvb4, harness v2, high 10.0 $50 762,779
GPT-6 Astra rvb4, harness v2, high 10.0 $58 54,495
Claude Opus 5.5 rvb4, harness v2, high 10.0 $7.62 224,834
Claude Sonnet 5.5 rvb4, harness v2, high 10.0 $14 515,015
GPT-6.1 Sol rvb4, harness v2, high 10.0 $33 154,134
Gemini 4 Argon rvb4, harness v2, high 10.0 $11 264,789
GPT-6.1 Terra rvb4, harness v2, high 10.0 $14 113,814
Claude Fable 5.2 rvb4, harness v2, high 10.0 $11 208,525
Kimi K3.1 Pro rvb4, harness v2, high 7.00 $15 800,087
DeepSeek V4.1 Flash 1017 rvb4, harness v2, high 5.00 $1.87 2,787,675
Tencent Hy4 rvb4, harness v2, high 5.00 $33 22,199,749
Claude Haiku 5.5 rvb4, harness v2, high 4.00 $1.42 1,096,238
Qwen 3.8 27B rvb4, harness v2, high 1.00 209,979
GLM-5.3-Flash rvb4, harness v2, high 1.00 $4.57 1,093,386
GPT-6 Luna rvb4, harness v2, high 1.00 $3.59 491,193
Gemma 4 26B A4B rvb4, harness v2, high 0.00 224,504
Gemma 4 31B IT rvb4, harness v2, high 0.00 75,162
Muse Glimmer rvb4, harness v2, high 0.00 857,362

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard 1 leader

Source: Official PokéBench: Raw Vision + Buttons leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
0.00 2.00 4.00 6.00 8.00 10.0 2026 2027 0.00 2.00 4.00 6.00 8.00 10.0 2026 2027

Source: Official PokéBench: Raw Vision + Buttons leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Games and long tasks

All 261 in Games and long tasks
  • ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6.2 Astra 100%
  • Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Each game scores in-game progress or outcomes, and the board ranks models by their average rank across the games. Gemini 2.5 Pro 3.90
  • MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. Seed2.0 Pro 49%
  • TextQuests Only in system cards Agents play 25 classic interactive fiction games. The score is harmful actions per 100 steps, where lower is better. The board also reports game progress. Llama 4 Scout 7.00
  • GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. GPT-6.1 Terra 76.8%
  • Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (19 entries).
  • Pass Models scored 18 models we track, plus 1 other entry on its leaderboard.
  • Pass Newest score The newest is from November 2026, for Claude Fable 5.2.
  • Pass Frontier models 8 of the top 10 have a score, led by Claude Fable 5.2 (#2).
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on pokebench.tv: 18 rows for models we track and 1 other entry at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check