Benchmark Almanac

FrogsGame post-training task

Claude Fable 5 34% leads since June 2026
Not in model cards

Agent has 20 hours to post-train Qwen3-8B to solve FrogsGame puzzle boards. A hidden verifier scores the best checkpoint on unseen boards.

Verifier is hidden and only the best submitted checkpoint counts.

Released
April 2026
Kind
Machine learning engineering
Level
Research frontier
Scored
Checked automatically
Answers
Agent task
Tasks
All public
Strongest model tested
Claude Fable 5 #11 on the capability index
Last model added
August 2026
Scores refreshed
November 12th, 2026
17
Models scored on its own leaderboard

Claude Fable 5 leads with 34% among models released by August 2026

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best.

The newest model with a score here came out in August 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

17 models. Scroll sideways for the rest.

  1. 34.0 Claude Fable 5
  2. 33.0 GLM-5.3
  3. 9.30 Grok 4.6
  4. 3.40 Grok 4.5
  5. 3.00 Claude Opus 4.8
  6. 2.40 GLM-5.2
  7. 2.00 GPT-5.4
  8. 1.40 GPT-5.5
  9. 1.40 Composer 2.5
  10. 1.00 Claude Opus 4.6
  11. 1.00 Claude Opus 4.7
  12. 0.40 GLM-5.1
  13. 0.40 Kimi K2.6
  14. 0.40 DeepSeek-V4-Pro
  15. 0.0 Kimi K2.5
  16. 0.0 Gemini 3.1 Pro
  17. 0.0 Qwen 3.6 Plus
  1. 1 Claude Fable 5 34%
  2. 2 GLM-5.3 33%
  3. 3 Grok 4.6 9.30%
  4. 4 Grok 4.5 3.40%
  5. 5 Claude Opus 4.8 3.00%
  6. 6 GLM-5.2 2.40%
  7. 7 GPT-5.4 2.00%
  8. 8 GPT-5.5 1.40%
  9. 9 Composer 2.5 1.40%
  10. 10 Claude Opus 4.6 1.00%
  11. 11 Claude Opus 4.7 1.00%
  12. 12 GLM-5.1 0.40%
  13. 13 Kimi K2.6 0.40%
  14. 14 DeepSeek-V4-Pro 0.40%
  15. 15 Kimi K2.5 0%
  16. 16 Gemini 3.1 Pro 0%
  17. 17 Qwen 3.6 Plus 0%

Source: Official FrogsGame post-training task leaderboard

#ModelScoreSettingDateSource
1 Claude Fable 5 34% Claude Code Jun 2026 Official leaderboard
2 GLM-5.3 33% Claude Code Aug 2026 Official leaderboard
3 Grok 4.6 9.30% Grok CLI Aug 2026 Official leaderboard
4 Grok 4.5 3.40% Grok CLI Jul 2026 Official leaderboard
5 Claude Opus 4.8 3.00% Claude Code May 2026 Official leaderboard
6 GLM-5.2 2.40% Claude Code Jun 2026 Official leaderboard
7 GPT-5.4 2.00% Codex Mar 2026 Official leaderboard
8 GPT-5.5 1.40% Codex Apr 2026 Official leaderboard
9 Composer 2.5 1.40% Cursor CLI May 2026 Official leaderboard
10 Claude Opus 4.6 1.00% Claude Code Feb 2026 Official leaderboard
11 Claude Opus 4.7 1.00% Claude Code Apr 2026 Official leaderboard
12 GLM-5.1 0.40% Claude Code Apr 2026 Official leaderboard
13 Kimi K2.6 0.40% Kimi CLI Apr 2026 Official leaderboard
14 DeepSeek-V4-Pro 0.40% Claude Code Apr 2026 Official leaderboard
15 Kimi K2.5 0% Kimi CLI Jan 2026 Official leaderboard
16 Gemini 3.1 Pro 0% Gemini CLI Feb 2026 Official leaderboard
17 Qwen 3.6 Plus 0% Qwen Code Mar 2026 Official leaderboard

Average score vs. tokens

The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

0% 10% 20% 30% 40% 100K1M10M100M1000M Average total tokens, log scale, lower is better Average score Kimi K2.5: Average score 0%, average total tokens 12.3M Claude Opus 4.6: Average score 1%, average total tokens 16.2M Gemini 3.1 Pro: Average score 0%, average total tokens 6.9M GPT-5.4: Average score 2%, average total tokens 36M Qwen 3.6 Plus: Average score 0%, average total tokens 12.6M GLM-5.1: Average score 0.4%, average total tokens 21.3M Claude Opus 4.7: Average score 1%, average total tokens 14.1M Kimi K2.6: Average score 0.4%, average total tokens 48.5M GPT-5.5: Average score 1.4%, average total tokens 9.7M DeepSeek-V4-Pro: Average score 0.4%, average total tokens 25.5M Claude Opus 4.8: Average score 3%, average total tokens 12.7M GLM-5.2: Average score 2.4%, average total tokens 154.3M Grok 4.5: Average score 3.4%, average total tokens 12.7M GLM-5.3: Average score 33%, average total tokens 104.5M Composer 2.5: Average score 1.4%, average total tokens 391K Claude Fable 5: Average score 34%, average total tokens 25.4M Grok 4.6: Average score 9.3%, average total tokens 12.4M Claude Fable 5Grok 4.6Composer 2.5 0% 10% 20% 30% 40% 100K1M10M100M1000M Average total tokens, log scale, lower is better Average score Kimi K2.5: Average score 0%, average total tokens 12.3M Claude Opus 4.6: Average score 1%, average total tokens 16.2M Gemini 3.1 Pro: Average score 0%, average total tokens 6.9M GPT-5.4: Average score 2%, average total tokens 36M Qwen 3.6 Plus: Average score 0%, average total tokens 12.6M GLM-5.1: Average score 0.4%, average total tokens 21.3M Claude Opus 4.7: Average score 1%, average total tokens 14.1M Kimi K2.6: Average score 0.4%, average total tokens 48.5M GPT-5.5: Average score 1.4%, average total tokens 9.7M DeepSeek-V4-Pro: Average score 0.4%, average total tokens 25.5M Claude Opus 4.8: Average score 3%, average total tokens 12.7M GLM-5.2: Average score 2.4%, average total tokens 154.3M Grok 4.5: Average score 3.4%, average total tokens 12.7M GLM-5.3: Average score 33%, average total tokens 104.5M Composer 2.5: Average score 1.4%, average total tokens 391K Claude Fable 5: Average score 34%, average total tokens 25.4M Grok 4.6: Average score 9.3%, average total tokens 12.4M Claude Fable 5Grok 4.6Composer 2.5
ModelSettingAverage scoreAverage total tokens
Claude Fable 5 Claude Code 34% 25,400,000
GLM-5.3 Claude Code 33% 104,500,000
Grok 4.6 Grok CLI 9.3% 12,400,000
Grok 4.5 Grok CLI 3.4% 12,700,000
Claude Opus 4.8 Claude Code 3% 12,700,000
GLM-5.2 Claude Code 2.4% 154,300,000
GPT-5.4 Codex 2% 36,000,000
Composer 2.5 Cursor CLI 1.4% 391,000
GPT-5.5 Codex 1.4% 9,700,000
Claude Opus 4.7 Claude Code 1% 14,100,000
Claude Opus 4.6 Claude Code 1% 16,200,000
GLM-5.1 Claude Code 0.4% 21,300,000
DeepSeek-V4-Pro Claude Code 0.4% 25,500,000
Kimi K2.6 Kimi CLI 0.4% 48,500,000
Gemini 3.1 Pro Gemini CLI 0% 6,900,000
Kimi K2.5 Kimi CLI 0% 12,300,000
Qwen 3.6 Plus Qwen Code 0% 12,600,000

3 models have held number one on the official leaderboard since its release in April 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Anthropic
Own leaderboard Not tested since Aug 2026
Claude Fable 5 34%

Source: Official FrogsGame post-training task leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Fable 5 Claude Opus 4.8 GPT-5.4 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Fable 5 Claude Opus 4.8

Source: Official FrogsGame post-training task leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in AI research

All 104 in AI research
  • PaperBench In use Agent replicates ICML 2024 papers from scratch. LLM judge grades against author-written hierarchical rubrics. Qwen 3.8 Max 93%
  • MLS-Bench In use Agent must invent one algorithmic change in an ML research scaffold whose gain transfers across settings. Scored relative to human baselines. Claude Fable 5.1 50.3%
  • PostTrainBench 1.2 In use CLI agent gets a base model, one H100 and a time budget to post-train it for a target benchmark. Scored on normalized gain. Claude Fable 5.1 44.6%
  • MLE-bench In use Kaggle-style ML engineering competitions. Agent trains models and is scored against competition leaderboards. GPT-6 Astra 93.8%
  • KernelBench In use Replace PyTorch operators with faster custom GPU kernels. Scored on correctness and speedup. MiniMax-M3 28.8%
  • PMPP-Hard In use Agents write GPU kernels for 69 tasks that must be correct, and on 48 of them also run within 1.25 times the reference time. The score is the area under the solved-tasks curve over a 10-hour budget, out of 690 task-hours. Inkling 2 Small 494.3