Benchmark Almanac

Hard12

Claude Opus 5.5 83.3% leads since November 2026
Not in model cards

Agents solve 12 selected coding tasks from five benchmarks. Hidden tests grade each task. The leaderboard averages three runs per task.

On November 10, 2026, GPT-6.1 Terra and Claude Opus 5.5 tied at 83.3% pass@1. Each used medium reasoning effort.

Released
November 2026
Kind
Fixing real repositories
Level
Expert
Scored
Checked automatically
Answers
Agent task
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Code → Code, Actions
Interaction
Agentic
Environment
Terminal
Items from
Another benchmark
Refresh
One fixed set
Run by
Anyone
In views
Agents
Strongest model tested
Claude Opus 5.5 #5 on the capability index
Last model added
October 2026
Scores refreshed
November 20th, 2026
26
Entries on its own leaderboard, 11 models we track

2 models share the top score of 83.3% 2 models share the top score of 83.3%

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran on its own with several settings, we keep its best. Every run is in the next chart. Agents other companies built around a model get a pale shade of that model's lab color. Gray marks blends and entries not in our model list, such as fine-tunes and older models. None of these count toward leaders. Agents built around a model, such as OpenCode (Claude Opus 5.5 + DeepSeek V4.1 Flash), are their own entries. We don't count them as the model's score. This view shows only models we track, each at its best.

  1. 1 Claude Opus 5.5 Opus 5.5 medium 83.3%
  2. 2 GPT-6.1 Terra GPT-6.1 Terra medium (pi) 83.3%
  3. 3 OpenCode (Claude Opus 5.5 + DeepSeek V4.1 Flash) Opus 5.5 orchestrator + DeepSeek V4.1 Flash (OpenCode) 80.6%
  4. 4 pi (Claude Opus 5.5 + Qwen 3.8 27B) Opus 5.5 orchestrator + Qwen3.8 27B (pi) 80.6%
  5. 5 Porch (Claude Opus 5.5 + DeepSeek V4.1 Flash 1017) Opus 5.5 high + DeepSeek V4.1 Flash 1017 (Porch) 80.6%
  6. 6 GPT-6.1 Sol GPT-6.1 Sol medium (pi) 80.6%
  7. 7 OpenCode (Claude Opus 5.5 + DeepSeek V4.1 Flash 1017) Opus 5.5 orchestrator + DeepSeek V4.1 Flash 1017 (OpenCode) 77.8%
  8. 8 pi (Claude Opus 5.5 + GPT-6 Luna) Opus 5.5 orchestrator + GPT-6 Luna (pi) 75%
  9. 9 Kimi K3.1 Pro Kimi K3.1 Pro solo (pi) 75%
  10. 10 Orchestrator (Claude Opus 5.5 + DeepSeek V4.1 Flash 1017) Opus 5.5 + DeepSeek V4.1 Flash 1017, delegation by prompt 72.2%
  11. 11 Review (Claude Opus 5.5 + Sol) Opus 5.5 low + Sol cross-review + fix 72.2%
  12. 12 OpenCode (DeepSeek V4.1 Flash 1017 + Sol) DeepSeek V4.1 Flash 1017 (OpenCode) + Sol review loop 72.2%
  13. 13 pi (GPT-6.1 Sol + Opus) GPT-6.1 Sol medium (pi) + Opus cross-review + fix 69.4%
  14. 14 Claude Code subagent (Claude Opus 5.5 + Claude Haiku 5.5) Opus 5.5 orchestrator + Haiku 5.5 (Claude Code subagent) 66.7%
  15. 15 DeepSeek V4.1 Flash 1017 DeepSeek V4.1 Flash 1017 (OpenCode) 63.9%
  16. 16 Kimi K3 Kimi K3 (pi) 61.1%
  17. 17 GPT-6 Luna GPT-6 Luna (pi) 61.1%
  18. 18 Team (Claude Opus 5.5 + GPT-6 Luna) PAUSE: Opus 5.5 spec + GPT-6 Luna 58.3%
  19. 19 Team (GPT-6 Luna + Claude Opus 5.5) Advisor: GPT-6 Luna + Opus 5.5 high 55.6%
  20. 20 Claude Haiku 5.5 Haiku 5.5 (Claude Code) 50%
  21. 21 pi (Qwen 3.8 27B + Sol) Qwen3.8 27B (pi) + Sol review loop 44.4%
  22. 22 DeepSeek V4.1 Flash DeepSeek V4.1 Flash (OpenCode) 44.4%
  23. 23 Team (Claude Opus 5.5 + Qwen 3.8 27B) PAUSE: Opus 5.5 spec + Qwen3.8 27B 44.4%
  24. 24 Qwen 3.8 27B Qwen3.8 27B (pi), loop run without review 33.3%
  25. 25 DeepSeek V4 Flash 0731 DeepSeek V4 Flash 0731 (pi) 16.7%
  26. 26 reflection/beam reflection/beam (pi) 11.1%

Source: Official Hard12 leaderboard

#EntryScore ±SettingDateSource
1 Claude Opus 5.5 83.3% Opus 5.5 medium, Single model Sep 2026 Official leaderboard
2 GPT-6.1 Terra 83.3% GPT-6.1 Terra medium (pi), pi Oct 2026 Official leaderboard
3 OpenCode (Claude Opus 5.5 + DeepSeek V4.1 Flash) 80.6% Opus 5.5 orchestrator + DeepSeek V4.1 Flash (OpenCode) Sep 2026 Official leaderboard
4 pi (Claude Opus 5.5 + Qwen 3.8 27B) 80.6% Opus 5.5 orchestrator + Qwen3.8 27B (pi) Sep 2026 Official leaderboard
5 Porch (Claude Opus 5.5 + DeepSeek V4.1 Flash 1017) 80.6% 58.3 to 100 Opus 5.5 high + DeepSeek V4.1 Flash 1017 (Porch) Sep 2026 Official leaderboard
6 GPT-6.1 Sol 80.6% GPT-6.1 Sol medium (pi), pi Sep 2026 Official leaderboard
7 OpenCode (Claude Opus 5.5 + DeepSeek V4.1 Flash 1017) 77.8% Opus 5.5 orchestrator + DeepSeek V4.1 Flash 1017 (OpenCode) Sep 2026 Official leaderboard
8 pi (Claude Opus 5.5 + GPT-6 Luna) 75% Opus 5.5 orchestrator + GPT-6 Luna (pi) Sep 2026 Official leaderboard
9 Kimi K3.1 Pro 75% Kimi K3.1 Pro solo (pi), pi Oct 2026 Official leaderboard
10 Orchestrator (Claude Opus 5.5 + DeepSeek V4.1 Flash 1017) 72.2% Opus 5.5 + DeepSeek V4.1 Flash 1017, delegation by prompt Sep 2026 Official leaderboard
11 Review (Claude Opus 5.5 + Sol) 72.2% Opus 5.5 low + Sol cross-review + fix Sep 2026 Official leaderboard
12 OpenCode (DeepSeek V4.1 Flash 1017 + Sol) 72.2% DeepSeek V4.1 Flash 1017 (OpenCode) + Sol review loop Oct 2026 Official leaderboard
13 pi (GPT-6.1 Sol + Opus) 69.4% GPT-6.1 Sol medium (pi) + Opus cross-review + fix Sep 2026 Official leaderboard
14 Claude Code subagent (Claude Opus 5.5 + Claude Haiku 5.5) 66.7% Opus 5.5 orchestrator + Haiku 5.5 (Claude Code subagent) Sep 2026 Official leaderboard
15 DeepSeek V4.1 Flash 1017 63.9% DeepSeek V4.1 Flash 1017 (OpenCode), OpenCode Oct 2026 Official leaderboard
16 Kimi K3 61.1% Kimi K3 (pi), pi Jul 2026 Official leaderboard
17 GPT-6 Luna 61.1% GPT-6 Luna (pi), pi Sep 2026 Official leaderboard
18 Team (Claude Opus 5.5 + GPT-6 Luna) 58.3% PAUSE: Opus 5.5 spec + GPT-6 Luna Sep 2026 Official leaderboard
19 Team (GPT-6 Luna + Claude Opus 5.5) 55.6% Advisor: GPT-6 Luna + Opus 5.5 high Sep 2026 Official leaderboard
20 Claude Haiku 5.5 50% Haiku 5.5 (Claude Code), Claude Code Oct 2026 Official leaderboard
21 pi (Qwen 3.8 27B + Sol) 44.4% Qwen3.8 27B (pi) + Sol review loop Aug 2026 Official leaderboard
22 DeepSeek V4.1 Flash 44.4% DeepSeek V4.1 Flash (OpenCode), OpenCode Sep 2026 Official leaderboard
23 Team (Claude Opus 5.5 + Qwen 3.8 27B) 44.4% PAUSE: Opus 5.5 spec + Qwen3.8 27B Sep 2026 Official leaderboard
24 Qwen 3.8 27B 33.3% Qwen3.8 27B (pi), loop run without review, pi Aug 2026 Official leaderboard
25 DeepSeek V4 Flash 0731 16.7% DeepSeek V4 Flash 0731 (pi), pi Jul 2026 Official leaderboard
26 reflection/beam 11.1% reflection/beam (pi), pi Official leaderboard
#ModelScore ±SettingDateSource
1 Claude Opus 5.5 83.3% Opus 5.5 medium, Single model Sep 2026 Official leaderboard
2 GPT-6.1 Terra 83.3% GPT-6.1 Terra medium (pi), pi Oct 2026 Official leaderboard
3 GPT-6.1 Sol 80.6% GPT-6.1 Sol medium (pi), pi Sep 2026 Official leaderboard
4 Kimi K3.1 Pro 75% Kimi K3.1 Pro solo (pi), pi Oct 2026 Official leaderboard
5 DeepSeek V4.1 Flash 1017 63.9% DeepSeek V4.1 Flash 1017 (OpenCode), OpenCode Oct 2026 Official leaderboard
6 Kimi K3 61.1% Kimi K3 (pi), pi Jul 2026 Official leaderboard
7 GPT-6 Luna 61.1% GPT-6 Luna (pi), pi Sep 2026 Official leaderboard
8 Claude Haiku 5.5 50% Haiku 5.5 (Claude Code), Claude Code Oct 2026 Official leaderboard
9 DeepSeek V4.1 Flash 44.4% DeepSeek V4.1 Flash (OpenCode), OpenCode Sep 2026 Official leaderboard
10 Qwen 3.8 27B 33.3% Qwen3.8 27B (pi), loop run without review, pi Aug 2026 Official leaderboard
11 DeepSeek V4 Flash 0731 16.7% DeepSeek V4 Flash 0731 (pi), pi Jul 2026 Official leaderboard

The best score, 83.3%, costs $0.17 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI DeepSeek Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-6.1 Terra, GPT-6.1 Terra medium (pi) DeepSeek V4.1 Flash 1017, DeepSeek V4.1 Flash 1017 (OpenCode) GPT-6 Luna, GPT-6 Luna (pi) 0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-6.1 Terra, GPT-6.1 Terra medium (pi) DeepSeek V4.1 Flash 1017, DeepSeek V4.1 Flash 1017 (OpenCode)

Source: Official Hard12 leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 Opus 5.5 medium Single model 83.3% $1.20
GPT-6.1 Terra GPT-6.1 Terra medium (pi) pi 83.3% $0.17
GPT-6.1 Sol GPT-6.1 Sol medium (pi) pi 80.6% $0.17
GPT-6.1 Sol GPT-6.1 Sol high (Codex) Codex 80.6% $0.64
GPT-6.1 Terra GPT-6.1 Terra high (pi, pre-fix task version) pi 80.6% $0.30
OpenCode (Claude Opus 5.5 + DeepSeek V4.1 Flash) Opus 5.5 orchestrator + DeepSeek V4.1 Flash (OpenCode) OpenCode 80.6% $1.32
pi (Claude Opus 5.5 + Qwen 3.8 27B) Opus 5.5 orchestrator + Qwen3.8 27B (pi) pi 80.6% $1.39
Porch (Claude Opus 5.5 + DeepSeek V4.1 Flash 1017) Opus 5.5 high + DeepSeek V4.1 Flash 1017 (Porch) Porch 80.6% $1.76
GPT-6.1 Sol GPT-6.1 Sol medium (pi) + Sol self-review + fix pi 77.8% $0.32
GPT-6.1 Sol GPT-6.1 Sol medium (Codex) Codex 77.8% $0.51
OpenCode (Claude Opus 5.5 + DeepSeek V4.1 Flash 1017) Opus 5.5 orchestrator + DeepSeek V4.1 Flash 1017 (OpenCode) OpenCode 77.8% $1.27
Kimi K3.1 Pro Kimi K3.1 Pro solo (pi) pi 75% $1.04
pi (Claude Opus 5.5 + GPT-6 Luna) Opus 5.5 orchestrator + GPT-6 Luna (pi) pi 75% $1.29
GPT-6.1 Sol GPT-6.1 Sol low (Codex) Codex 72.2% $0.33
Orchestrator (Claude Opus 5.5 + DeepSeek V4.1 Flash 1017) Opus 5.5 + DeepSeek V4.1 Flash 1017, delegation by prompt Orchestrator 72.2% $0.69
Review (Claude Opus 5.5 + Sol) Opus 5.5 low + Sol cross-review + fix Review 72.2% $1.39
OpenCode (DeepSeek V4.1 Flash 1017 + Sol) DeepSeek V4.1 Flash 1017 (OpenCode) + Sol review loop OpenCode 72.2% $0.28
pi (GPT-6.1 Sol + Opus) GPT-6.1 Sol medium (pi) + Opus cross-review + fix pi 69.4% $0.51
Claude Code subagent (Claude Opus 5.5 + Claude Haiku 5.5) Opus 5.5 orchestrator + Haiku 5.5 (Claude Code subagent) Claude Code subagent 66.7% $1.74
Claude Opus 5.5 Opus 5.5 low Single model 63.9% $0.58
DeepSeek V4.1 Flash 1017 DeepSeek V4.1 Flash 1017 (OpenCode) OpenCode 63.9% $0.14
Kimi K3 Kimi K3 (pi) pi 61.1% $1.58
Claude Opus 5.5 Opus 5.5 low, review run without review Review 61.1% $0.59
Claude Opus 5.5 Opus 5.5 low + Opus self-review + fix Review 61.1% $1.72
GPT-6 Luna GPT-6 Luna (pi) pi 61.1% $0.06
DeepSeek V4.1 Flash 1017 DeepSeek V4.1 Flash 1017 (OpenCode), loop run without review OpenCode 61.1% $0.15
Team (Claude Opus 5.5 + GPT-6 Luna) PAUSE: Opus 5.5 spec + GPT-6 Luna Team 58.3% $0.57
Team (Claude Opus 5.5 + GPT-6 Luna) Pipeline: Opus 5.5 medium + GPT-6 Luna Team 58.3% $0.84
Team (Claude Opus 5.5 + GPT-6 Luna) Pipeline: Opus 5.5 high + GPT-6 Luna Team 55.6% $1.22
Team (GPT-6 Luna + Claude Opus 5.5) Advisor: GPT-6 Luna + Opus 5.5 high Team 55.6% $2.12
Claude Haiku 5.5 Haiku 5.5 (Claude Code) Claude Code 50% $0.37
Team (GPT-6 Luna + Claude Opus 5.5) Advisor: GPT-6 Luna + Opus 5.5 medium Team 47.2% $1.93
DeepSeek V4.1 Flash DeepSeek V4.1 Flash (OpenCode) OpenCode 44.4% $0.07
GPT-6 Luna GPT-6 Luna (Codex) Codex 44.4% $0.08
pi (Qwen 3.8 27B + Sol) Qwen3.8 27B (pi) + Sol review loop pi 44.4% $0.07
Team (Claude Opus 5.5 + Qwen 3.8 27B) PAUSE: Opus 5.5 spec + Qwen3.8 27B Team 44.4% $0.51
Qwen 3.8 27B Qwen3.8 27B (pi), loop run without review pi 33.3%
Qwen 3.8 27B Qwen3.8 27B (team harness) team harness 30.6%
Qwen 3.8 27B Qwen3.8 27B (pi) pi 19.4%
DeepSeek V4 Flash 0731 DeepSeek V4 Flash 0731 (pi) pi 16.7% $0.10

The best score, 83.3%, costs $0.17 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI DeepSeek Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-6.1 Terra, GPT-6.1 Terra medium (pi) DeepSeek V4.1 Flash 1017, DeepSeek V4.1 Flash 1017 (OpenCode) GPT-6 Luna, GPT-6 Luna (pi) 0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-6.1 Terra, GPT-6.1 Terra medium (pi) DeepSeek V4.1 Flash 1017, DeepSeek V4.1 Flash 1017 (OpenCode)

Source: Official Hard12 leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 Opus 5.5 medium Single model 83.3% $1.20
GPT-6.1 Terra GPT-6.1 Terra medium (pi) pi 83.3% $0.17
GPT-6.1 Sol GPT-6.1 Sol medium (pi) pi 80.6% $0.17
GPT-6.1 Sol GPT-6.1 Sol high (Codex) Codex 80.6% $0.64
GPT-6.1 Terra GPT-6.1 Terra high (pi, pre-fix task version) pi 80.6% $0.30
GPT-6.1 Sol GPT-6.1 Sol medium (pi) + Sol self-review + fix pi 77.8% $0.32
GPT-6.1 Sol GPT-6.1 Sol medium (Codex) Codex 77.8% $0.51
Kimi K3.1 Pro Kimi K3.1 Pro solo (pi) pi 75% $1.04
GPT-6.1 Sol GPT-6.1 Sol low (Codex) Codex 72.2% $0.33
Claude Opus 5.5 Opus 5.5 low Single model 63.9% $0.58
DeepSeek V4.1 Flash 1017 DeepSeek V4.1 Flash 1017 (OpenCode) OpenCode 63.9% $0.14
Kimi K3 Kimi K3 (pi) pi 61.1% $1.58
Claude Opus 5.5 Opus 5.5 low, review run without review Review 61.1% $0.59
Claude Opus 5.5 Opus 5.5 low + Opus self-review + fix Review 61.1% $1.72
GPT-6 Luna GPT-6 Luna (pi) pi 61.1% $0.06
DeepSeek V4.1 Flash 1017 DeepSeek V4.1 Flash 1017 (OpenCode), loop run without review OpenCode 61.1% $0.15
Claude Haiku 5.5 Haiku 5.5 (Claude Code) Claude Code 50% $0.37
DeepSeek V4.1 Flash DeepSeek V4.1 Flash (OpenCode) OpenCode 44.4% $0.07
GPT-6 Luna GPT-6 Luna (Codex) Codex 44.4% $0.08
Qwen 3.8 27B Qwen3.8 27B (pi), loop run without review pi 33.3%
Qwen 3.8 27B Qwen3.8 27B (team harness) team harness 30.6%
Qwen 3.8 27B Qwen3.8 27B (pi) pi 19.4%
DeepSeek V4 Flash 0731 DeepSeek V4 Flash 0731 (pi) pi 16.7% $0.10

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard 1 leader

Source: Official Hard12 leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) Claude Opus 5.5 Kimi K3 0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) Claude Opus 5.5 Kimi K3

Source: Official Hard12 leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Coding

All 523 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Fable 5.2 96.6%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471

Data audit

All 11 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 5 pass
  • Pass Independent runs Scores from the official leaderboard (26 entries).
  • Pass Models scored 11 models we track, plus 15 other entries on its leaderboard.
  • Pass Newest score The newest is from October 2026, for GPT-6.1 Terra.
  • Pass Frontier models 3 of the top 10 have a score, led by Claude Opus 5.5 (#5).
  • Pass Headroom The best is 83.3%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on ibragim.dev: 40 rows for models we track and 1 other entry at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check