Benchmark Almanac

FrontierCode

Claude Opus 4.8 34.3% leads since June 2026
Claude Opus 5.5 54.4% leads since September 2026
In use

Real-repo coding tasks graded for mergeability: correctness, regressions, lint, tests, scope and code quality. Failing any blocker scores zero.

Tasks written by 20+ open-source maintainers, 40+ hours each. Only sample tasks are public.

Made by
Cognition
Released
June 2026
Kind
Fixing real repositories
Level
Expert
Scored
Checked automatically
Answers
Code
Tasks
Kept private
Strongest model tested
Claude Opus 5.5 #4 on the capability index
Last in a launch post
September 2026
Last model added
September 2026
Scores refreshed
November 12th, 2026
12
Models scored on its own leaderboard
4
Launch posts list it, from 1 lab
2
System cards report it, from 1 lab

Claude Opus 4.8 leads with 34.3% among models released by May 2026

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

Claude Opus 5.5 leads with 54.4%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

The newest model with a score here came out in May 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 34.3 Claude Opus 4.8
  2. 25.5 GPT-5.5
  3. 23.0 Claude Opus 4.7
  4. 17.8 GPT-5.4 Mini
  5. 16.7 Gemini 3.1 Pro
  6. 16.0 Kimi K2.6
  7. 15.1 Claude Sonnet 4.6
  8. 6.86 Kimi K2.5
  9. 5.98 MiniMax-M2.7
  10. 5.50 SWE-1.6
  11. 5.34 MiniMax-M2.5
  12. 4.84 Gemini 3.1 Flash-Lite
  1. 1 Claude Opus 4.8 xhigh 34.3%
  2. 2 GPT-5.5 xhigh 25.5%
  3. 3 Claude Opus 4.7 xhigh 23%
  4. 4 GPT-5.4 Mini xhigh 17.8%
  5. 5 Gemini 3.1 Pro high 16.7%
  6. 6 Kimi K2.6 none 16%
  7. 7 Claude Sonnet 4.6 high 15.1%
  8. 8 Kimi K2.5 none 6.86%
  9. 9 MiniMax-M2.7 none 5.98%
  10. 10 SWE-1.6 none 5.50%
  11. 11 MiniMax-M2.5 none 5.34%
  12. 12 Gemini 3.1 Flash-Lite low 4.84%

Source: Official FrontierCode leaderboard

Sources: Labs’ model cards and official FrontierCode leaderboard

#ModelScoreSettingDateSource
1 Claude Opus 4.8 34.3% xhigh, Claude Code May 2026 Official leaderboard
2 GPT-5.5 25.5% xhigh, Codex Apr 2026 Official leaderboard
3 Claude Opus 4.7 23% xhigh, Claude Code Apr 2026 Official leaderboard
4 GPT-5.4 Mini 17.8% xhigh, Codex Mar 2026 Official leaderboard
5 Gemini 3.1 Pro 16.7% high, Gemini CLI Feb 2026 Official leaderboard
6 Kimi K2.6 16% none, mini-swe-agent Apr 2026 Official leaderboard
7 Claude Sonnet 4.6 15.1% high, Claude Code Feb 2026 Official leaderboard
8 Kimi K2.5 6.86% none, mini-swe-agent Jan 2026 Official leaderboard
9 MiniMax-M2.7 5.98% none, mini-swe-agent Mar 2026 Official leaderboard
10 SWE-1.6 5.50% none, Devin Apr 2026 Official leaderboard
11 MiniMax-M2.5 5.34% none, mini-swe-agent Feb 2026 Official leaderboard
12 Gemini 3.1 Flash-Lite 4.84% low, Gemini CLI Mar 2026 Official leaderboard
#ModelScoreSettingDateSource
1 Claude Opus 5.5 54.4% Sep 2026 Lab model card
2 Claude Opus 5 53.4% Jul 2026 Lab model card
3 Claude Sonnet 5 38.8% Jun 2026 Lab model card
4 Claude Opus 4.8 34.3% xhigh, Claude Code May 2026 Official leaderboard
5 Claude Fable 5 29.3% Jun 2026 Lab model card
6 GPT-5.5 25.5% xhigh, Codex Apr 2026 Official leaderboard
7 Claude Opus 4.7 23% xhigh, Claude Code Apr 2026 Official leaderboard
8 GPT-5.4 Mini 17.8% xhigh, Codex Mar 2026 Official leaderboard
9 Gemini 3.1 Pro 16.7% high, Gemini CLI Feb 2026 Official leaderboard
10 Kimi K2.6 16% none, mini-swe-agent Apr 2026 Official leaderboard
11 Claude Sonnet 4.6 15.1% high, Claude Code Feb 2026 Official leaderboard
12 Kimi K2.5 6.86% none, mini-swe-agent Jan 2026 Official leaderboard
13 MiniMax-M2.7 5.98% none, mini-swe-agent Mar 2026 Official leaderboard
14 SWE-1.6 5.50% none, Devin Apr 2026 Official leaderboard
15 MiniMax-M2.5 5.34% none, mini-swe-agent Feb 2026 Official leaderboard
16 Gemini 3.1 Flash-Lite 4.84% low, Gemini CLI Mar 2026 Official leaderboard

Labs report scores for 4 models in their cards. Switch the scores above to rank them together.

Score against time

Each point is one model at one reasoning effort. The post's chart can also put output tokens, cost, tool calls, agent steps or OTE on the x-axis. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

0% 10% 20% 30% 40% 1001,00010K Avg wall-clock time per rollout (s), log scale, lower is better Score Kimi K2.5 (none): Score 6.86%, avg wall-clock time per rollout 641 s MiniMax-M2.5 (none): Score 5.34%, avg wall-clock time per rollout 605 s Claude Sonnet 4.6 (medium): Score 13.6%, avg wall-clock time per rollout 908 s Claude Sonnet 4.6 (high): Score 15.1%, avg wall-clock time per rollout 1,056 s Claude Sonnet 4.6 (xhigh): Score 13.4%, avg wall-clock time per rollout 1,162 s Claude Sonnet 4.6 (max): Score 13.2%, avg wall-clock time per rollout 1,074 s Gemini 3.1 Pro (low): Score 16.5%, avg wall-clock time per rollout 930 s Gemini 3.1 Pro (high): Score 16.7%, avg wall-clock time per rollout 773 s Gemini 3.1 Flash-Lite (minimal): Score 2.51%, avg wall-clock time per rollout 446 s Gemini 3.1 Flash-Lite (medium): Score 1.84%, avg wall-clock time per rollout 484 s Gemini 3.1 Flash-Lite (high): Score 4.44%, avg wall-clock time per rollout 301 s GPT-5.4 Mini (medium): Score 12.1%, avg wall-clock time per rollout 390 s GPT-5.4 Mini (high): Score 12.4%, avg wall-clock time per rollout 475 s GPT-5.4 Mini (xhigh): Score 17.8%, avg wall-clock time per rollout 827 s MiniMax-M2.7 (none): Score 5.98%, avg wall-clock time per rollout 517 s SWE-1.6 (none): Score 5.5%, avg wall-clock time per rollout 445 s Claude Opus 4.7 (low): Score 17.9%, avg wall-clock time per rollout 456 s Claude Opus 4.7 (medium): Score 21.3%, avg wall-clock time per rollout 752 s Claude Opus 4.7 (high): Score 22.1%, avg wall-clock time per rollout 907 s Claude Opus 4.7 (xhigh): Score 23%, avg wall-clock time per rollout 1,030 s Kimi K2.6 (none): Score 16%, avg wall-clock time per rollout 869 s Claude Opus 4.8 (low): Score 25.3%, avg wall-clock time per rollout 613 s Claude Opus 4.8 (max): Score 31.3%, avg wall-clock time per rollout 1,552 s Gemini 3.1 Flash-Lite (low): Score 4.84%, avg wall-clock time per rollout 283 s GPT-5.5 (low): Score 21.1%, avg wall-clock time per rollout 349 s GPT-5.5 (medium): Score 22.4%, avg wall-clock time per rollout 466 s GPT-5.5 (high): Score 25.3%, avg wall-clock time per rollout 570 s GPT-5.5 (xhigh): Score 25.5%, avg wall-clock time per rollout 724 s Claude Opus 4.8 (medium): Score 26.9%, avg wall-clock time per rollout 865 s Claude Opus 4.8 (high): Score 30.3%, avg wall-clock time per rollout 869 s Claude Opus 4.8 (xhigh): Score 34.3%, avg wall-clock time per rollout 1,232 s Claude Opus 4.8 (xhigh)Claude Opus 4.8 (high)Claude Opus 4.8 (medium)GPT-5.5 (xhigh)GPT-5.5 (high)GPT-5.5 (medium)GPT-5.5 (low)Gemini 3.1 Flash-Lite (low) 0% 10% 20% 30% 40% 1001,00010K Avg wall-clock time per rollout (s), log scale, lower is better Score Kimi K2.5 (none): Score 6.86%, avg wall-clock time per rollout 641 s MiniMax-M2.5 (none): Score 5.34%, avg wall-clock time per rollout 605 s Claude Sonnet 4.6 (medium): Score 13.6%, avg wall-clock time per rollout 908 s Claude Sonnet 4.6 (high): Score 15.1%, avg wall-clock time per rollout 1,056 s Claude Sonnet 4.6 (xhigh): Score 13.4%, avg wall-clock time per rollout 1,162 s Claude Sonnet 4.6 (max): Score 13.2%, avg wall-clock time per rollout 1,074 s Gemini 3.1 Pro (low): Score 16.5%, avg wall-clock time per rollout 930 s Gemini 3.1 Pro (high): Score 16.7%, avg wall-clock time per rollout 773 s Gemini 3.1 Flash-Lite (minimal): Score 2.51%, avg wall-clock time per rollout 446 s Gemini 3.1 Flash-Lite (medium): Score 1.84%, avg wall-clock time per rollout 484 s Gemini 3.1 Flash-Lite (high): Score 4.44%, avg wall-clock time per rollout 301 s GPT-5.4 Mini (medium): Score 12.1%, avg wall-clock time per rollout 390 s GPT-5.4 Mini (high): Score 12.4%, avg wall-clock time per rollout 475 s GPT-5.4 Mini (xhigh): Score 17.8%, avg wall-clock time per rollout 827 s MiniMax-M2.7 (none): Score 5.98%, avg wall-clock time per rollout 517 s SWE-1.6 (none): Score 5.5%, avg wall-clock time per rollout 445 s Claude Opus 4.7 (low): Score 17.9%, avg wall-clock time per rollout 456 s Claude Opus 4.7 (medium): Score 21.3%, avg wall-clock time per rollout 752 s Claude Opus 4.7 (high): Score 22.1%, avg wall-clock time per rollout 907 s Claude Opus 4.7 (xhigh): Score 23%, avg wall-clock time per rollout 1,030 s Kimi K2.6 (none): Score 16%, avg wall-clock time per rollout 869 s Claude Opus 4.8 (low): Score 25.3%, avg wall-clock time per rollout 613 s Claude Opus 4.8 (max): Score 31.3%, avg wall-clock time per rollout 1,552 s Gemini 3.1 Flash-Lite (low): Score 4.84%, avg wall-clock time per rollout 283 s GPT-5.5 (low): Score 21.1%, avg wall-clock time per rollout 349 s GPT-5.5 (medium): Score 22.4%, avg wall-clock time per rollout 466 s GPT-5.5 (high): Score 25.3%, avg wall-clock time per rollout 570 s GPT-5.5 (xhigh): Score 25.5%, avg wall-clock time per rollout 724 s Claude Opus 4.8 (medium): Score 26.9%, avg wall-clock time per rollout 865 s Claude Opus 4.8 (high): Score 30.3%, avg wall-clock time per rollout 869 s Claude Opus 4.8 (xhigh): Score 34.3%, avg wall-clock time per rollout 1,232 s Claude Opus 4.8 (xhigh)Claude Opus 4.8 (high)Claude Opus 4.8 (medium)GPT-5.5 (xhigh)GPT-5.5 (high)GPT-5.5 (medium)GPT-5.5 (low)Gemini 3.1 Flash-Lite (low)
ModelSettingScoreAvg wall-clock time per rollout
Claude Opus 4.8 xhigh, Claude Code 34.3% 1,232 s
Claude Opus 4.8 max, Claude Code 31.3% 1,552 s
Claude Opus 4.8 high, Claude Code 30.3% 869.4 s
Claude Opus 4.8 medium, Claude Code 26.9% 865.2 s
GPT-5.5 xhigh, Codex 25.5% 723.6 s
GPT-5.5 high, Codex 25.3% 570 s
Claude Opus 4.8 low, Claude Code 25.3% 613.2 s
Claude Opus 4.7 xhigh, Claude Code 23% 1,030 s
GPT-5.5 medium, Codex 22.4% 466.2 s
Claude Opus 4.7 high, Claude Code 22.1% 906.6 s
Claude Opus 4.7 medium, Claude Code 21.3% 752.4 s
GPT-5.5 low, Codex 21.1% 349.2 s
Claude Opus 4.7 low, Claude Code 17.9% 456 s
GPT-5.4 Mini xhigh, Codex 17.8% 826.8 s
Gemini 3.1 Pro high, Gemini CLI 16.7% 773.4 s
Gemini 3.1 Pro low, Gemini CLI 16.5% 930 s
Kimi K2.6 none, mini-swe-agent 16% 868.8 s
Claude Sonnet 4.6 high, Claude Code 15.1% 1,056 s
Claude Sonnet 4.6 medium, Claude Code 13.6% 908.4 s
Claude Sonnet 4.6 xhigh, Claude Code 13.4% 1,162 s
Claude Sonnet 4.6 max, Claude Code 13.2% 1,074 s
GPT-5.4 Mini high, Codex 12.5% 475.2 s
GPT-5.4 Mini medium, Codex 12.1% 390 s
Kimi K2.5 none, mini-swe-agent 6.9% 640.8 s
MiniMax-M2.7 none, mini-swe-agent 6% 517.2 s
SWE-1.6 none, Devin 5.5% 444.6 s
MiniMax-M2.5 none, mini-swe-agent 5.3% 605.4 s
Gemini 3.1 Flash-Lite low, Gemini CLI 4.8% 282.6 s
Gemini 3.1 Flash-Lite high, Gemini CLI 4.4% 300.6 s
Gemini 3.1 Flash-Lite minimal, Gemini CLI 2.5% 446.4 s
Gemini 3.1 Flash-Lite medium, Gemini CLI 1.8% 484.2 s

The best score, 34.3%, costs $6.69 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Google DeepMind OpenAI Anthropic Cost frontier One model at each effort level
0% 10% 20% 30% 40% $0.10 $0.20 $0.50 $1.00 $2.00 $5.00 $10 Cost per task, log scale Claude Opus 4.8, xhigh Claude Opus 4.8, high Claude Opus 4.8, medium Claude Opus 4.8, low GPT-5.5, low GPT-5.4 Mini, high Gemini 3.1 Flash-Lite, low 0% 10% 20% 30% 40% $0.10 $1.00 $10 Cost per task, log scale Claude Opus 4.8, xhigh Claude Opus 4.8, high

Source: Official FrontierCode leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 4.8 xhigh Claude Code 34.3% $6.69 62,445
Claude Opus 4.8 max Claude Code 31.3% $9.03 88,169
Claude Opus 4.8 high Claude Code 30.3% $3.94 37,280
Claude Opus 4.8 medium Claude Code 26.9% $3.30 31,701
GPT-5.5 xhigh Codex 25.5% $5.59 25,576
GPT-5.5 high Codex 25.3% $4.38 19,002
Claude Opus 4.8 low Claude Code 25.3% $2.68 23,239
Claude Opus 4.7 xhigh Claude Code 23% $5.93 35,077
GPT-5.5 medium Codex 22.4% $3.29 13,912
Claude Opus 4.7 high Claude Code 22.1% $4.43 28,180
Claude Opus 4.7 medium Claude Code 21.3% $2.85 20,017
GPT-5.5 low Codex 21.1% $1.61 8,482
Claude Opus 4.7 low Claude Code 17.9% $2.43 16,932
GPT-5.4 Mini xhigh Codex 17.8% $1.67 88,550
Gemini 3.1 Pro high Gemini CLI 16.7% $2.41 18,997
Gemini 3.1 Pro low Gemini CLI 16.5% $2.52 19,651
Kimi K2.6 none mini-swe-agent 16% 41,815
Claude Sonnet 4.6 high Claude Code 15.1% $2.40 34,334
Claude Sonnet 4.6 medium Claude Code 13.6% $2.09 28,176
Claude Sonnet 4.6 xhigh Claude Code 13.4% $2.48 34,856
Claude Sonnet 4.6 max Claude Code 13.2% $2.57 38,004
GPT-5.4 Mini high Codex 12.4% $1.05 45,131
GPT-5.4 Mini medium Codex 12.1% $0.89 34,720
Kimi K2.5 none mini-swe-agent 6.86% $2.55 19,108
MiniMax-M2.7 none mini-swe-agent 5.98% 30,587
SWE-1.6 none Devin 5.50%
MiniMax-M2.5 none mini-swe-agent 5.34% 21,691
Gemini 3.1 Flash-Lite low Gemini CLI 4.84% $0.17 13,559
Gemini 3.1 Flash-Lite high Gemini CLI 4.44% $0.18 14,679
Gemini 3.1 Flash-Lite minimal Gemini CLI 2.51% $0.37 11,463
Gemini 3.1 Flash-Lite medium Gemini CLI 1.84% $0.26 10,006

Number one over time 4 models have held number one with claims included since its release in June 2026

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Anthropic
Own leaderboard Replaced by 1.1, Jul 2026
Claude Opus 4.8 34.3%
Lab claims Replaced by 1.1, Sep 2026
Claude Opus 5.5 54.4%

Sources: Official FrontierCode leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Opus 5.5 Claude Opus 5 Claude Sonnet 5 Claude Opus 4.8 GPT-5.5 GPT-5.4 Mini Kimi K2.5 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Opus 5.5 Claude Sonnet 5

Sources: Official FrontierCode leaderboard and labs’ model cards

One lab put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Anthropic
DateLaunch postScoreAs the post writes it
Sep 22, 2026 Claude Opus 5.5 54.4 Agentic coding FrontierCode v1.1 (Main)
Jul 24, 2026 Claude Opus 5 53.4 Agentic coding FrontierCode v1.1, Main
Jun 30, 2026 Claude Sonnet 5 38.8 FrontierCode v1
Jun 9, 2026 Claude Fable 5 29.3 Agentic coding FrontierCode (Diamond) (xhigh)

2 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Jun 30, 2026 Claude Sonnet 5 system card 38.8 FrontierCode v1, 150 tasks, max effort (xhigh 34.0%, high 28.9%) (Table 8.1.A, Figure 8.4.A) Capabilities
Jun 9, 2026 Claude Fable 5 system card 29.3 FrontierCode (Diamond subset), score at xhigh effort, mean@5, 150-task benchmark by Cognition; pass rate 30.2%; Main subset score 46.3% and pass rate 48.8% (Table 8.1.A, 8.4) Capabilities

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic
FrontierCode 2.0 4 leaders
Claude Fable 5.2 50.2%
FrontierCode Replaced by 1.1, Jul 2026
Claude Opus 4.8 34.3%
FrontierCode 1.1 Replaced by 2.0, Oct 2026
Claude Fable 5 53.5%
FrontierCode 2.0 2 leaders
Claude Fable 5.2 50.2%

Source: Official leaderboards

Replaced by FrontierCode 1.1

More in Coding

All 282 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Fable 5.2 96.6%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471