Benchmark Almanac

Brokk Power Ranking

Claude Opus 5.5 39.7 leads since September 2026
Not in model cards

Models solve 181 real tickets from a 1.1 million line Rust codebase. Model judges compare solutions in pairs, and a Bradley-Terry fit gives each model a strength score.

Also reports speed and value per model. Updated October 29th, 2026. Release month is the earliest Reddit post found, so check it.

Made by
Brokk (SlopCop)
Released
August 2025
Kind
Fixing real repositories
Level
Expert
Scored
Graded by a model
Answers
Code
Tasks
Kept private
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
19
Models scored on its own leaderboard

Claude Opus 5.5 leads with 39.7

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

19 models. Scroll sideways for the rest.

  1. 39.7 Claude Opus 5.5
  2. 29.6 GPT-6.1 Terra
  3. 26.9 Claude Sonnet 5.5
  4. 19.9 GPT-6.1 Sol
  5. 19.9 Claude Haiku 5.5
  6. 14.5 GPT-6 Astra
  7. 10.7 GPT-6 Luna
  8. 10.6 Claude Opus 5
  9. 8.80 GPT-5.6 Luna
  10. 8.30 Kimi K3
  11. 8.20 GPT-6 Sol
  12. 7.80 DeepSeek V4.1 Flash
  13. 7.60 GPT-5.6 Sol
  14. 5.40 Grok 4.7
  15. 4.60 Muse Spark 1.3
  16. 3.50 Kimi K3.1 Pro
  17. 3.00 GLM-5.3
  18. 1.70 GLM-5.3-Flash
  19. 1.50 Grok 4.6
  1. 1 Claude Opus 5.5 xhigh 39.7
  2. 2 GPT-6.1 Terra max 29.6
  3. 3 Claude Sonnet 5.5 xhigh 26.9
  4. 4 GPT-6.1 Sol high 19.9
  5. 5 Claude Haiku 5.5 max 19.9
  6. 6 GPT-6 Astra low 14.5
  7. 7 GPT-6 Luna max 10.7
  8. 8 Claude Opus 5 xhigh 10.6
  9. 9 GPT-5.6 Luna max 8.80
  10. 10 Kimi K3 max 8.30
  11. 11 GPT-6 Sol xhigh 8.20
  12. 12 DeepSeek V4.1 Flash max 7.80
  13. 13 GPT-5.6 Sol xhigh 7.60
  14. 14 Grok 4.7 high 5.40
  15. 15 Muse Spark 1.3 ultra 4.60
  16. 16 Kimi K3.1 Pro max 3.50
  17. 17 GLM-5.3 max 3.00
  18. 18 GLM-5.3-Flash max 1.70
  19. 19 Grok 4.6 xhigh 1.50

Source: Official Brokk Power Ranking leaderboard

#ModelScoreSettingDateSource
1 Claude Opus 5.5 39.7 xhigh Sep 2026 Official leaderboard
2 GPT-6.1 Terra 29.6 max Oct 2026 Official leaderboard
3 Claude Sonnet 5.5 26.9 xhigh Sep 2026 Official leaderboard
4 GPT-6.1 Sol 19.9 high Sep 2026 Official leaderboard
5 Claude Haiku 5.5 19.9 max Oct 2026 Official leaderboard
6 GPT-6 Astra 14.5 low Sep 2026 Official leaderboard
7 GPT-6 Luna 10.7 max Sep 2026 Official leaderboard
8 Claude Opus 5 10.6 xhigh Jul 2026 Official leaderboard
9 GPT-5.6 Luna 8.80 max Jul 2026 Official leaderboard
10 Kimi K3 8.30 max Jul 2026 Official leaderboard
11 GPT-6 Sol 8.20 xhigh Sep 2026 Official leaderboard
12 DeepSeek V4.1 Flash 7.80 max Sep 2026 Official leaderboard
13 GPT-5.6 Sol 7.60 xhigh Jul 2026 Official leaderboard
14 Grok 4.7 5.40 high Sep 2026 Official leaderboard
15 Muse Spark 1.3 4.60 ultra Sep 2026 Official leaderboard
16 Kimi K3.1 Pro 3.50 max Oct 2026 Official leaderboard
17 GLM-5.3 3.00 max Aug 2026 Official leaderboard
18 GLM-5.3-Flash 1.70 max Aug 2026 Official leaderboard
19 Grok 4.6 1.50 xhigh Aug 2026 Official leaderboard

Intelligence vs speed

Working minutes per task for a task of typical size, against the Bradley-Terry strength from pairwise judging (0 is the average model). The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

-2 0 2 4 02,0004,0006,0008,00010K Working time per task (s), lower is better Intelligence (strength) GPT-5.6 Luna (max): Intelligence (strength) -0.91, working time per task 6,522 s GPT-5.6 Sol (xhigh): Intelligence (strength) -0.03, working time per task 5,664 s Kimi K3 (max): Intelligence (strength) 1.25, working time per task 9,774 s Claude Opus 5 (xhigh): Intelligence (strength) 1.02, working time per task 8,436 s Grok 4.6 (xhigh): Intelligence (strength) -3.01, working time per task 4,044 s GLM-5.3 (max): Intelligence (strength) -1.33, working time per task 6,498 s GLM-5.3-Flash (max): Intelligence (strength) -2.96, working time per task 6,852 s Muse Spark 1.3 (ultra): Intelligence (strength) -1.6, working time per task 6,342 s DeepSeek V4.1 Flash (max): Intelligence (strength) 0.13, working time per task 8,958 s DeepSeek V4.1 Flash (high): Intelligence (strength) -0.79, working time per task 8,748 s Grok 4.7 (xhigh): Intelligence (strength) -0.54, working time per task 5,130 s Grok 4.7 (high): Intelligence (strength) -0.61, working time per task 5,490 s Claude Opus 5.5 (medium): Intelligence (strength) 0.71, working time per task 5,022 s GPT-6 Luna (max): Intelligence (strength) -0.82, working time per task 6,348 s GPT-6 Luna (xhigh): Intelligence (strength) -1.4, working time per task 5,676 s GPT-6 Sol (xhigh): Intelligence (strength) -0.16, working time per task 3,738 s Claude Sonnet 5.5 (xhigh): Intelligence (strength) 2.13, working time per task 7,290 s Claude Sonnet 5.5 (high): Intelligence (strength) 0.76, working time per task 5,178 s Claude Sonnet 5.5 (medium): Intelligence (strength) -2.45, working time per task 2,868 s GPT-6.1 Sol (high): Intelligence (strength) 1.2, working time per task 4,164 s GPT-6.1 Sol (medium): Intelligence (strength) 1, working time per task 4,458 s GPT-6.1 Sol (xhigh): Intelligence (strength) 0.97, working time per task 4,230 s Claude Haiku 5.5 (max): Intelligence (strength) 0.86, working time per task 5,358 s Claude Haiku 5.5 (xhigh): Intelligence (strength) 0.09, working time per task 5,076 s Claude Haiku 5.5 (high): Intelligence (strength) -1.18, working time per task 5,964 s Kimi K3.1 Pro (max): Intelligence (strength) -0.88, working time per task 7,476 s Kimi K3.1 Pro (high): Intelligence (strength) -2.15, working time per task 6,900 s GPT-6.1 Terra (xhigh): Intelligence (strength) 0.34, working time per task 3,912 s GPT-6 Astra (low): Intelligence (strength) 1.26, working time per task 3,312 s GPT-6 Astra (medium): Intelligence (strength) 1.19, working time per task 3,264 s Claude Opus 5.5 (xhigh): Intelligence (strength) 3.12, working time per task 7,164 s Claude Opus 5.5 (high): Intelligence (strength) 2.05, working time per task 5,016 s GPT-6.1 Terra (max): Intelligence (strength) 2.35, working time per task 5,676 s GPT-6.1 Terra (high): Intelligence (strength) -1.05, working time per task 2,502 s Claude Opus 5.5 (xhigh)GPT-6.1 Terra (max)Claude Opus 5.5 (high)GPT-6 Astra (low)GPT-6 Astra (medium)GPT-6.1 Terra (high) -2 0 2 4 02,0004,0006,0008,00010K Working time per task (s), lower is better Intelligence (strength) GPT-5.6 Luna (max): Intelligence (strength) -0.91, working time per task 6,522 s GPT-5.6 Sol (xhigh): Intelligence (strength) -0.03, working time per task 5,664 s Kimi K3 (max): Intelligence (strength) 1.25, working time per task 9,774 s Claude Opus 5 (xhigh): Intelligence (strength) 1.02, working time per task 8,436 s Grok 4.6 (xhigh): Intelligence (strength) -3.01, working time per task 4,044 s GLM-5.3 (max): Intelligence (strength) -1.33, working time per task 6,498 s GLM-5.3-Flash (max): Intelligence (strength) -2.96, working time per task 6,852 s Muse Spark 1.3 (ultra): Intelligence (strength) -1.6, working time per task 6,342 s DeepSeek V4.1 Flash (max): Intelligence (strength) 0.13, working time per task 8,958 s DeepSeek V4.1 Flash (high): Intelligence (strength) -0.79, working time per task 8,748 s Grok 4.7 (xhigh): Intelligence (strength) -0.54, working time per task 5,130 s Grok 4.7 (high): Intelligence (strength) -0.61, working time per task 5,490 s Claude Opus 5.5 (medium): Intelligence (strength) 0.71, working time per task 5,022 s GPT-6 Luna (max): Intelligence (strength) -0.82, working time per task 6,348 s GPT-6 Luna (xhigh): Intelligence (strength) -1.4, working time per task 5,676 s GPT-6 Sol (xhigh): Intelligence (strength) -0.16, working time per task 3,738 s Claude Sonnet 5.5 (xhigh): Intelligence (strength) 2.13, working time per task 7,290 s Claude Sonnet 5.5 (high): Intelligence (strength) 0.76, working time per task 5,178 s Claude Sonnet 5.5 (medium): Intelligence (strength) -2.45, working time per task 2,868 s GPT-6.1 Sol (high): Intelligence (strength) 1.2, working time per task 4,164 s GPT-6.1 Sol (medium): Intelligence (strength) 1, working time per task 4,458 s GPT-6.1 Sol (xhigh): Intelligence (strength) 0.97, working time per task 4,230 s Claude Haiku 5.5 (max): Intelligence (strength) 0.86, working time per task 5,358 s Claude Haiku 5.5 (xhigh): Intelligence (strength) 0.09, working time per task 5,076 s Claude Haiku 5.5 (high): Intelligence (strength) -1.18, working time per task 5,964 s Kimi K3.1 Pro (max): Intelligence (strength) -0.88, working time per task 7,476 s Kimi K3.1 Pro (high): Intelligence (strength) -2.15, working time per task 6,900 s GPT-6.1 Terra (xhigh): Intelligence (strength) 0.34, working time per task 3,912 s GPT-6 Astra (low): Intelligence (strength) 1.26, working time per task 3,312 s GPT-6 Astra (medium): Intelligence (strength) 1.19, working time per task 3,264 s Claude Opus 5.5 (xhigh): Intelligence (strength) 3.12, working time per task 7,164 s Claude Opus 5.5 (high): Intelligence (strength) 2.05, working time per task 5,016 s GPT-6.1 Terra (max): Intelligence (strength) 2.35, working time per task 5,676 s GPT-6.1 Terra (high): Intelligence (strength) -1.05, working time per task 2,502 s Claude Opus 5.5 (xhigh)GPT-6.1 Terra (max)Claude Opus 5.5 (high)GPT-6 Astra (low)GPT-6 Astra (medium)GPT-6.1 Terra (high)
ModelSettingIntelligence (strength)Working time per task
Claude Opus 5.5 xhigh 3.12 7,164 s
GPT-6.1 Terra max 2.35 5,676 s
Claude Sonnet 5.5 xhigh 2.13 7,290 s
Claude Opus 5.5 high 2.05 5,016 s
GPT-6 Astra low 1.26 3,312 s
Kimi K3 max 1.25 9,774 s
GPT-6.1 Sol high 1.2 4,164 s
GPT-6 Astra medium 1.19 3,264 s
Claude Opus 5 xhigh 1.02 8,436 s
GPT-6.1 Sol medium 1 4,458 s
GPT-6.1 Sol xhigh 0.97 4,230 s
Claude Haiku 5.5 max 0.86 5,358 s
Claude Sonnet 5.5 high 0.76 5,178 s
Claude Opus 5.5 medium 0.71 5,022 s
GPT-6.1 Terra xhigh 0.34 3,912 s
DeepSeek V4.1 Flash max 0.13 8,958 s
Claude Haiku 5.5 xhigh 0.09 5,076 s
GPT-5.6 Sol xhigh -0.03 5,664 s
GPT-6 Sol xhigh -0.16 3,738 s
Grok 4.7 xhigh -0.54 5,130 s
Grok 4.7 high -0.61 5,490 s
DeepSeek V4.1 Flash high -0.79 8,748 s
GPT-6 Luna max -0.82 6,348 s
Kimi K3.1 Pro max -0.88 7,476 s
GPT-5.6 Luna max -0.91 6,522 s
GPT-6.1 Terra high -1.05 2,502 s
Claude Haiku 5.5 high -1.18 5,964 s
GLM-5.3 max -1.33 6,498 s
GPT-6 Luna xhigh -1.4 5,676 s
Muse Spark 1.3 ultra -1.6 6,342 s
Kimi K3.1 Pro high -2.15 6,900 s
Claude Sonnet 5.5 medium -2.45 2,868 s
GLM-5.3-Flash max -2.96 6,852 s
Grok 4.6 xhigh -3.01 4,044 s

The best score, 39.7, costs $20 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Meta DeepSeek Anthropic OpenAI Cost frontier One model at each effort level
0.00 10.0 20.0 30.0 40.0 $0.10 $1.00 $10 $100 Cost per task, log scale Claude Opus 5.5, xhigh Claude Opus 5.5, high GPT-6.1 Sol, high Claude Sonnet 5.5, high Claude Haiku 5.5, xhigh GPT-6 Luna, max GPT-6 Luna, xhigh 0.00 10.0 20.0 30.0 40.0 $0.10 $1.00 $10 $100 Cost per task, log scale Claude Opus 5.5, xhigh Claude Opus 5.5, high

Source: Official Brokk Power Ranking leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 xhigh 39.7 $20
Claude Opus 5.5 high 29.7 $11
GPT-6.1 Terra max 29.6 $6.82
Claude Sonnet 5.5 xhigh 26.9 $9.28
GPT-6.1 Sol high 19.9 $4.57
Claude Haiku 5.5 max 19.9 $5.12
Claude Sonnet 5.5 high 18.2 $4.04
GPT-6.1 Sol medium 16.9 $5.44
GPT-6.1 Sol xhigh 16.9 $5.51
Claude Opus 5.5 medium 16.6 $7.42
Claude Haiku 5.5 xhigh 15.1 $3.48
GPT-6 Astra low 14.5 $23
GPT-6 Astra medium 13.7 $26
GPT-6.1 Terra xhigh 13.7 $3.84
GPT-6 Luna max 10.7 $0.62
Claude Opus 5 xhigh 10.6 $47
GPT-6.1 Terra high 9.00 $1.98
GPT-5.6 Luna max 8.80 $1.54
GPT-6 Luna xhigh 8.50 $0.56
Claude Haiku 5.5 high 8.50 $2.28
Kimi K3 max 8.30 $24
GPT-6 Sol xhigh 8.20 $11
DeepSeek V4.1 Flash max 7.80 $0.54
GPT-5.6 Sol xhigh 7.60 $18
Claude Sonnet 5.5 medium 5.50 $1.41
Grok 4.7 high 5.40 $18
DeepSeek V4.1 Flash high 5.30 $0.43
Grok 4.7 xhigh 4.90 $32
Muse Spark 1.3 ultra 4.60 $0.21
Kimi K3.1 Pro max 3.50 $13
GLM-5.3 max 3.00 $14
Kimi K3.1 Pro high 2.00 $12
GLM-5.3-Flash max 1.70 $4.12
Grok 4.6 xhigh 1.50 $21

4 models have held number one on the official leaderboard since its release in August 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Anthropic
Own leaderboard 4 leaders
Claude Opus 5.5 39.7

Source: Official Brokk Power Ranking leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0.00 10.0 20.0 30.0 40.0 2027 0.00 10.0 20.0 30.0 40.0 2027

Source: Official Brokk Power Ranking leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Coding

All 282 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Fable 5.2 96.6%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471