Benchmark Almanac

OpenHands Index

Claude Fable 5 81% leads since June 2026
Not in model cards

Models run in the same OpenHands agent setup on five areas: issue resolution, greenfield builds, frontend work, test writing and information gathering. Accuracy and cost are both reported.

Built from SWE-bench Verified, commit0, SWE-bench Multimodal, SWT-Bench and GAIA. A public API serves the leaderboard.

Released
January 2026
Kind
Fixing real repositories
Level
Expert
Scored
Checked automatically
Answers
Agent task
Tasks
All public
Strongest model tested
Claude Fable 5 #11 on the capability index
Last model added
June 2026
Scores refreshed
November 14th, 2026
33
Models scored on its own leaderboard

Claude Fable 5 leads with 81% among models released by June 2026

Best score per model from the benchmark's own leaderboard, read on November 14th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

The newest model with a score here came out in June 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

33 models. Scroll sideways for the rest.

  1. 81.0 Claude Fable 5
  2. 71.9 Claude Opus 4.8
  3. 69.7 Claude Opus 4.7
  4. 66.7 Claude Opus 4.6
  5. 65.9 GPT-5.5
  6. 64.3 GPT-5.4
  7. 62.6 Gemini 3.5 Flash
  8. 60.6 Claude Opus 4.5
  9. 60.6 Gemini 3.1 Pro
  10. 58.8 GPT-5.2
  11. 58.3 GPT-5.2 Codex
  12. 58.2 GLM-5.1
  13. 57.2 MiniMax-M3
  14. 57.1 Kimi K2.6
  15. 53.0 Claude Sonnet 4.5
  16. 52.9 Qwen 3.6 Plus
  17. 49.4 GLM-5
  18. 49.2 Kimi K2.5
  19. 49.0 Gemini 3 Pro
  20. 49.0 Gemini 3 Flash
  21. 45.7 DeepSeek-V3.2
  22. 45.2 MiniMax-M2.5
  23. 44.5 Claude Sonnet 4.6
  24. 43.8 Qwen3-Coder-Next
  25. 43.4 MiniMax-M2.7
  26. 42.3 GLM-4.7
  27. 41.2 MiniMax-M2.1
  28. 41.0 Kimi K2 Thinking
  29. 40.7 DeepSeek-V4-Pro
  30. 38.1 Qwen3.5-Flash
  31. 36.2 Nemotron 3 Super
  32. 32.1 Trinity Large Thinking
  33. 30.9 Qwen3-Coder-480B-A35B
  1. 1 Claude Fable 5 81%
  2. 2 Claude Opus 4.8 71.9%
  3. 3 Claude Opus 4.7 69.7%
  4. 4 Claude Opus 4.6 66.7%
  5. 5 GPT-5.5 65.9%
  6. 6 GPT-5.4 64.3%
  7. 7 Gemini 3.5 Flash 62.6%
  8. 8 Claude Opus 4.5 60.6%
  9. 9 Gemini 3.1 Pro 60.6%
  10. 10 GPT-5.2 58.8%
  11. 11 GPT-5.2 Codex 58.3%
  12. 12 GLM-5.1 58.2%
  13. 13 MiniMax-M3 57.2%
  14. 14 Kimi K2.6 57.1%
  15. 15 Claude Sonnet 4.5 53%
  16. 16 Qwen 3.6 Plus 52.9%
  17. 17 GLM-5 49.4%
  18. 18 Kimi K2.5 49.2%
  19. 19 Gemini 3 Pro 49%
  20. 20 Gemini 3 Flash 49%
  21. 21 DeepSeek-V3.2 45.7%
  22. 22 MiniMax-M2.5 45.2%
  23. 23 Claude Sonnet 4.6 44.5%
  24. 24 Qwen3-Coder-Next 43.8%
  25. 25 MiniMax-M2.7 43.4%
  26. 26 GLM-4.7 42.3%
  27. 27 MiniMax-M2.1 41.2%
  28. 28 Kimi K2 Thinking 41%
  29. 29 DeepSeek-V4-Pro 40.7%
  30. 30 Qwen3.5-Flash 38.1%
  31. 31 Nemotron 3 Super 36.2%
  32. 32 Trinity Large Thinking 32.1%
  33. 33 Qwen3-Coder-480B-A35B 30.9%

Source: Official OpenHands Index leaderboard

#ModelScoreSettingDateSource
1 Claude Fable 5 81% OpenHands Jun 2026 Official leaderboard
2 Claude Opus 4.8 71.9% OpenHands May 2026 Official leaderboard
3 Claude Opus 4.7 69.7% OpenHands Apr 2026 Official leaderboard
4 Claude Opus 4.6 66.7% OpenHands Feb 2026 Official leaderboard
5 GPT-5.5 65.9% OpenHands Apr 2026 Official leaderboard
6 GPT-5.4 64.3% OpenHands Mar 2026 Official leaderboard
7 Gemini 3.5 Flash 62.6% OpenHands May 2026 Official leaderboard
8 Claude Opus 4.5 60.6% OpenHands Nov 2025 Official leaderboard
9 Gemini 3.1 Pro 60.6% OpenHands Feb 2026 Official leaderboard
10 GPT-5.2 58.8% OpenHands Dec 2025 Official leaderboard
11 GPT-5.2 Codex 58.3% OpenHands Dec 2025 Official leaderboard
12 GLM-5.1 58.2% OpenHands Apr 2026 Official leaderboard
13 MiniMax-M3 57.2% OpenHands Jun 2026 Official leaderboard
14 Kimi K2.6 57.1% OpenHands Apr 2026 Official leaderboard
15 Claude Sonnet 4.5 53% OpenHands Sep 2025 Official leaderboard
16 Qwen 3.6 Plus 52.9% OpenHands Mar 2026 Official leaderboard
17 GLM-5 49.4% OpenHands Feb 2026 Official leaderboard
18 Kimi K2.5 49.2% OpenHands Jan 2026 Official leaderboard
19 Gemini 3 Pro 49% OpenHands Nov 2025 Official leaderboard
20 Gemini 3 Flash 49% OpenHands Dec 2025 Official leaderboard
21 DeepSeek-V3.2 45.7% OpenHands Dec 2025 Official leaderboard
22 MiniMax-M2.5 45.2% OpenHands Feb 2026 Official leaderboard
23 Claude Sonnet 4.6 44.5% OpenHands Feb 2026 Official leaderboard
24 Qwen3-Coder-Next 43.8% OpenHands Feb 2026 Official leaderboard
25 MiniMax-M2.7 43.4% OpenHands Mar 2026 Official leaderboard
26 GLM-4.7 42.3% OpenHands Dec 2025 Official leaderboard
27 MiniMax-M2.1 41.2% OpenHands Dec 2025 Official leaderboard
28 Kimi K2 Thinking 41% OpenHands Nov 2025 Official leaderboard
29 DeepSeek-V4-Pro 40.7% OpenHands Apr 2026 Official leaderboard
30 Qwen3.5-Flash 38.1% OpenHands Feb 2026 Official leaderboard
31 Nemotron 3 Super 36.2% OpenHands Mar 2026 Official leaderboard
32 Trinity Large Thinking 32.1% OpenHands Apr 2026 Official leaderboard
33 Qwen3-Coder-480B-A35B 30.9% OpenHands Jul 2025 Official leaderboard

OpenHands Index Overall Runtime/Performance

Average score against average runtime per problem on a log scale, with the efficiency frontier of models that are faster or score higher than any other. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

0% 20% 40% 60% 80% 100% 1001,00010K Average runtime per problem (s), log scale, lower is better Average score Qwen3-Coder-480B-A35B: Average score 30.9%, average runtime per problem 502 s Claude Sonnet 4.5: Average score 53%, average runtime per problem 565 s Kimi K2 Thinking: Average score 41%, average runtime per problem 1,268 s Gemini 3 Pro: Average score 49%, average runtime per problem 1,091 s Claude Opus 4.5: Average score 60.6%, average runtime per problem 525 s DeepSeek-V3.2: Average score 45.7%, average runtime per problem 1,254 s GPT-5.2: Average score 58.8%, average runtime per problem 833 s Gemini 3 Flash: Average score 49%, average runtime per problem 768 s GPT-5.2 Codex: Average score 58.3%, average runtime per problem 915 s GLM-4.7: Average score 42.3%, average runtime per problem 1,219 s MiniMax-M2.1: Average score 41.2%, average runtime per problem 818 s Kimi K2.5: Average score 49.2%, average runtime per problem 899 s Qwen3-Coder-Next: Average score 43.8%, average runtime per problem 1,082 s Claude Opus 4.6: Average score 66.7%, average runtime per problem 501 s GLM-5: Average score 49.4%, average runtime per problem 1,343 s MiniMax-M2.5: Average score 45.2%, average runtime per problem 780 s Claude Sonnet 4.6: Average score 44.5%, average runtime per problem 737 s Gemini 3.1 Pro: Average score 60.6%, average runtime per problem 786 s Qwen3.5-Flash: Average score 38.1%, average runtime per problem 2,910 s GPT-5.4: Average score 64.3%, average runtime per problem 473 s Nemotron 3 Super: Average score 36.2%, average runtime per problem 1,329 s MiniMax-M2.7: Average score 43.4%, average runtime per problem 852 s Qwen 3.6 Plus: Average score 52.9%, average runtime per problem 700 s Trinity Large Thinking: Average score 32.1%, average runtime per problem 1,028 s GLM-5.1: Average score 58.2%, average runtime per problem 1,367 s Kimi K2.6: Average score 57.1%, average runtime per problem 856 s GPT-5.5: Average score 65.9%, average runtime per problem 423 s DeepSeek-V4-Pro: Average score 40.7%, average runtime per problem 920 s Gemini 3.5 Flash: Average score 62.6%, average runtime per problem 784 s MiniMax-M3: Average score 57.2%, average runtime per problem 1,150 s Claude Opus 4.7: Average score 69.7%, average runtime per problem 303 s Claude Opus 4.8: Average score 71.9%, average runtime per problem 359 s Claude Fable 5: Average score 81%, average runtime per problem 459 s Claude Fable 5Claude Opus 4.8Claude Opus 4.7 0% 20% 40% 60% 80% 100% 1001,00010K Average runtime per problem (s), log scale, lower is better Average score Qwen3-Coder-480B-A35B: Average score 30.9%, average runtime per problem 502 s Claude Sonnet 4.5: Average score 53%, average runtime per problem 565 s Kimi K2 Thinking: Average score 41%, average runtime per problem 1,268 s Gemini 3 Pro: Average score 49%, average runtime per problem 1,091 s Claude Opus 4.5: Average score 60.6%, average runtime per problem 525 s DeepSeek-V3.2: Average score 45.7%, average runtime per problem 1,254 s GPT-5.2: Average score 58.8%, average runtime per problem 833 s Gemini 3 Flash: Average score 49%, average runtime per problem 768 s GPT-5.2 Codex: Average score 58.3%, average runtime per problem 915 s GLM-4.7: Average score 42.3%, average runtime per problem 1,219 s MiniMax-M2.1: Average score 41.2%, average runtime per problem 818 s Kimi K2.5: Average score 49.2%, average runtime per problem 899 s Qwen3-Coder-Next: Average score 43.8%, average runtime per problem 1,082 s Claude Opus 4.6: Average score 66.7%, average runtime per problem 501 s GLM-5: Average score 49.4%, average runtime per problem 1,343 s MiniMax-M2.5: Average score 45.2%, average runtime per problem 780 s Claude Sonnet 4.6: Average score 44.5%, average runtime per problem 737 s Gemini 3.1 Pro: Average score 60.6%, average runtime per problem 786 s Qwen3.5-Flash: Average score 38.1%, average runtime per problem 2,910 s GPT-5.4: Average score 64.3%, average runtime per problem 473 s Nemotron 3 Super: Average score 36.2%, average runtime per problem 1,329 s MiniMax-M2.7: Average score 43.4%, average runtime per problem 852 s Qwen 3.6 Plus: Average score 52.9%, average runtime per problem 700 s Trinity Large Thinking: Average score 32.1%, average runtime per problem 1,028 s GLM-5.1: Average score 58.2%, average runtime per problem 1,367 s Kimi K2.6: Average score 57.1%, average runtime per problem 856 s GPT-5.5: Average score 65.9%, average runtime per problem 423 s DeepSeek-V4-Pro: Average score 40.7%, average runtime per problem 920 s Gemini 3.5 Flash: Average score 62.6%, average runtime per problem 784 s MiniMax-M3: Average score 57.2%, average runtime per problem 1,150 s Claude Opus 4.7: Average score 69.7%, average runtime per problem 303 s Claude Opus 4.8: Average score 71.9%, average runtime per problem 359 s Claude Fable 5: Average score 81%, average runtime per problem 459 s Claude Fable 5Claude Opus 4.8Claude Opus 4.7
ModelSettingAverage scoreAverage runtime per problem
Claude Fable 5 OpenHands 81% 459 s
Claude Opus 4.8 OpenHands 71.9% 359 s
Claude Opus 4.7 OpenHands 69.7% 303 s
Claude Opus 4.6 OpenHands 66.7% 501 s
GPT-5.5 OpenHands 65.9% 423 s
GPT-5.4 OpenHands 64.3% 473 s
Gemini 3.5 Flash OpenHands 62.6% 784 s
Claude Opus 4.5 OpenHands 60.6% 525 s
Gemini 3.1 Pro OpenHands 60.6% 786 s
GPT-5.2 OpenHands 58.8% 833 s
GPT-5.2 Codex OpenHands 58.3% 915 s
GLM-5.1 OpenHands 58.2% 1,367 s
MiniMax-M3 OpenHands 57.2% 1,150 s
Kimi K2.6 OpenHands 57.1% 856 s
Claude Sonnet 4.5 OpenHands 53% 565 s
Qwen 3.6 Plus OpenHands 52.9% 700 s
GLM-5 OpenHands 49.4% 1,343 s
Kimi K2.5 OpenHands 49.2% 899 s
Gemini 3 Pro OpenHands 49% 1,091 s
Gemini 3 Flash OpenHands 49% 768 s
DeepSeek-V3.2 OpenHands 45.7% 1,254 s
MiniMax-M2.5 OpenHands 45.2% 780 s
Claude Sonnet 4.6 OpenHands 44.5% 737 s
Qwen3-Coder-Next OpenHands 43.8% 1,082 s
MiniMax-M2.7 OpenHands 43.4% 852 s
GLM-4.7 OpenHands 42.3% 1,219 s
MiniMax-M2.1 OpenHands 41.2% 818 s
Kimi K2 Thinking OpenHands 41% 1,268 s
DeepSeek-V4-Pro OpenHands 40.7% 920 s
Qwen3.5-Flash OpenHands 38.1% 2,910 s
Nemotron 3 Super OpenHands 36.2% 1,329 s
Trinity Large Thinking OpenHands 32.1% 1,028 s
Qwen3-Coder-480B-A35B OpenHands 30.9% 502 s

The best score, 81%, costs $5.54 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

DeepSeek OpenAI Anthropic MiniMax Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.10 $0.20 $0.50 $1.00 $2.00 $5.00 $10 Cost per task, log scale Claude Fable 5 Claude Opus 4.8 GPT-5.4 MiniMax-M3 DeepSeek-V3.2 DeepSeek-V4-Pro 0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 Cost per task, log scale Claude Fable 5 Claude Opus 4.8

Source: Official OpenHands Index leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5 OpenHands 81% $5.54
Claude Opus 4.8 OpenHands 71.9% $2.46
Claude Opus 4.7 OpenHands 69.7% $2.31
Claude Opus 4.6 OpenHands 66.7% $2.34
GPT-5.5 OpenHands 65.9% $2.31
GPT-5.4 OpenHands 64.3% $1.44
Gemini 3.5 Flash OpenHands 62.6% $3.02
Claude Opus 4.5 OpenHands 60.6% $3.08
Gemini 3.1 Pro OpenHands 60.6% $1.54
GPT-5.2 OpenHands 58.8% $1.87
GPT-5.2 Codex OpenHands 58.3% $2.12
GLM-5.1 OpenHands 58.2% $3.00
MiniMax-M3 OpenHands 57.2% $0.32
Kimi K2.6 OpenHands 57.1% $0.72
Claude Sonnet 4.5 OpenHands 53% $1.63
Qwen 3.6 Plus OpenHands 52.9% $2.75
GLM-5 OpenHands 49.4% $1.03
Kimi K2.5 OpenHands 49.2% $0.82
Gemini 3 Pro OpenHands 49% $1.42
Gemini 3 Flash OpenHands 49% $0.90
DeepSeek-V3.2 OpenHands 45.7% $0.22
MiniMax-M2.5 OpenHands 45.2% $0.34
Claude Sonnet 4.6 OpenHands 44.5% $2.21
Qwen3-Coder-Next OpenHands 43.8% $1.03
MiniMax-M2.7 OpenHands 43.4% $0.36
GLM-4.7 OpenHands 42.3% $0.68
MiniMax-M2.1 OpenHands 41.2% $0.66
Kimi K2 Thinking OpenHands 41% $1.38
DeepSeek-V4-Pro OpenHands 40.7% $0.15
Qwen3.5-Flash OpenHands 38.1% $2.23
Nemotron 3 Super OpenHands 36.2% $0.58
Trinity Large Thinking OpenHands 32.1% $0.76
Qwen3-Coder-480B-A35B OpenHands 30.9% $0.92

5 models have held number one on the official leaderboard since its release in January 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard Not tested since Jun 2026
Claude Fable 5 81%

Source: Official OpenHands Index leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Fable 5 Claude Opus 4.8 Claude Opus 4.6 Claude Opus 4.5 Claude Sonnet 4.5 Qwen3-Coder-480B-A35B 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Fable 5 Claude Opus 4.8

Source: Official OpenHands Index leaderboard

No lab model card reports it yet

We read the headline results of 523 launch posts from 52 labs, and none of them list this benchmark.

More in Coding

All 282 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Fable 5.2 96.6%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471