Benchmark Almanac

CWE-bench 2

Claude Fable 5.2 75% leads since November 2026
In use
Version 2

Agents audit and patch vulnerabilities in 250 codebases. The score is pass@1 under deterministic verification.

The board publishes pass@4 and average API cost per rollout. Runs use high reasoning and a one-hour task limit.

Made by
Collinear AI
Released
November 2026
Kind
Secure code and review
Level
Expert
Scored
Checked automatically
Answers
Code
Tasks
Public, plus a private test set
Measures
Capability
Language
English
In and out
Text, Code → Code, Actions
Interaction
Agentic
Environment
Terminal
Items from
Real work and use
Run by
A host
Based on
CWE-bench
In views
Agents
Strongest model tested
GPT-6.2 Astra #1 on the capability index
Last in a launch post
November 2026
Last model added
November 2026
Scores refreshed
November 19th, 2026
20
Models scored on its own leaderboard
1
Launch post lists it, from 1 lab
1
System card reports it, from 1 lab

Claude Fable 5.2 leads with 75%

Best score per model from the benchmark's own leaderboard, read on November 19th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

Source: Official CWE-bench 2 leaderboard

#ModelScoreSettingDateSource
1 Claude Fable 5.2 75% high reasoning; 1h limit, Claude Code Nov 2026 Official leaderboard
2 Gemini 4 Neon 70.7% high reasoning; 1h limit, Antigravity Nov 2026 Official leaderboard
3 Gemini 4 Hydrogen 68.1% high reasoning; 1h limit, Antigravity Nov 2026 Official leaderboard
4 Gemini 4 Carbon 67.3% high reasoning; 1h limit, Antigravity Oct 2026 Official leaderboard
5 Claude Opus 5.5 67.1% high reasoning; 1h limit, Claude Code Sep 2026 Official leaderboard
6 GPT-6.2 Astra 66.8% high reasoning; 1h limit, Codex Nov 2026 Official leaderboard
7 Muse Wonder 66.1% high reasoning; 1h limit, opencode Oct 2026 Official leaderboard
8 GPT-6.2 Sol 66% high reasoning; 1h limit, Codex Nov 2026 Official leaderboard
9 Gemini 4 Argon 65% high reasoning; 1h limit, Antigravity Sep 2026 Official leaderboard
10 GPT-6 Astra 64.2% high reasoning; 1h limit, Codex Sep 2026 Official leaderboard
11 Tencent Hy4 64.1% high reasoning; 1h limit, Claude Code Nov 2026 Official leaderboard
12 Claude Sonnet 5.5 61.4% high reasoning; 1h limit, Claude Code Sep 2026 Official leaderboard
13 GPT-6.1 Terra 60% high reasoning; 1h limit, Codex Oct 2026 Official leaderboard
14 DeepSeek V4.1 Flash 1017 57.6% high reasoning; 1h limit, opencode Oct 2026 Official leaderboard
15 Grok 4.7 54.7% high reasoning; 1h limit, opencode Sep 2026 Official leaderboard
16 Tencent Hy4 preview 47.3% high reasoning; 1h limit, opencode Aug 2026 Official leaderboard
17 Mistral Large 4 45.6% high reasoning; 1h limit, opencode Oct 2026 Official leaderboard
18 Qwen3.8 Max (0902) 45.4% high reasoning; 1h limit, opencode Sep 2026 Official leaderboard
19 Kimi K3.1 Pro 45.3% high reasoning; 1h limit, opencode Oct 2026 Official leaderboard
20 GLM-5.3 42% high reasoning; 1h limit, opencode Aug 2026 Official leaderboard

Labs report scores for 1 model in their cards. We plot every source in the charts below.

Pass@1 and average cost per rollout

The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

40% 50% 60% 70% 80% $0$2$4$6$8 Mean cost per rollout ($), lower is better Pass@1 Claude Fable 5.2 (high reasoning; 1h limit)Gemini 4 Neon (high reasoning; 1h limit)Gemini 4 Hydrogen (high reasoning; 1h limit)Claude Opus 5.5 (high reasoning; 1h limit)GPT-6.2 Sol (high reasoning; 1h limit)Claude Sonnet 5.5 (high reasoning; 1h limit)GPT-6.1 Terra (high reasoning; 1h limit)DeepSeek V4.1 Flash 1017 (high reasoning; 1h limit) 40% 50% 60% 70% 80% $0$2$4$6$8 Mean cost per rollout ($), lower is better Pass@1 Claude Fable 5.2 (high reasoning; 1h limit)Gemini 4 Neon (high reasoning; 1h limit)Gemini 4 Hydrogen (high reasoning; 1h limit)Claude Opus 5.5 (high reasoning; 1h limit)GPT-6.2 Sol (high reasoning; 1h limit)Claude Sonnet 5.5 (high reasoning; 1h limit)GPT-6.1 Terra (high reasoning; 1h limit)DeepSeek V4.1 Flash 1017 (high reasoning; 1h limit)
ModelSettingPass@1Mean cost per rollout
Claude Fable 5.2 high reasoning; 1h limit, Claude Code 75% $7.85
Gemini 4 Neon high reasoning; 1h limit, Antigravity 70.7% $6.52
Gemini 4 Hydrogen high reasoning; 1h limit, Antigravity 68.1% $2.63
Gemini 4 Carbon high reasoning; 1h limit, Antigravity 67.3% $4.27
Claude Opus 5.5 high reasoning; 1h limit, Claude Code 67.1% $1.85
GPT-6.2 Astra high reasoning; 1h limit, Codex 66.8% $4.32
Muse Wonder high reasoning; 1h limit, opencode 66.1% $2.73
GPT-6.2 Sol high reasoning; 1h limit, Codex 66% $0.81
Gemini 4 Argon high reasoning; 1h limit, Antigravity 65% $3.7
GPT-6 Astra high reasoning; 1h limit, Codex 64.2% $3.87
Tencent Hy4 high reasoning; 1h limit, Claude Code 64.1% $1.35
Claude Sonnet 5.5 high reasoning; 1h limit, Claude Code 61.4% $0.58
GPT-6.1 Terra high reasoning; 1h limit, Codex 60% $0.44
DeepSeek V4.1 Flash 1017 high reasoning; 1h limit, opencode 57.6% $0.04
Grok 4.7 high reasoning; 1h limit, opencode 54.7% $2.97
Tencent Hy4 preview high reasoning; 1h limit, opencode 47.3% $0.43
Mistral Large 4 high reasoning; 1h limit, opencode 45.6% $0.37
Qwen3.8 Max (0902) high reasoning; 1h limit, opencode 45.4% $2.06
Kimi K3.1 Pro high reasoning; 1h limit, opencode 45.3% $0.96
GLM-5.3 high reasoning; 1h limit, opencode 42% $0.58

The best score, 75%, costs $7.85 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Anthropic DeepSeek OpenAI Google DeepMind Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Fable 5.2, high reasoning; 1h limit Gemini 4 Neon, high reasoning; 1h limit GPT-6.2 Sol, high reasoning; 1h limit Claude Sonnet 5.5, high reasoning; 1h limit 0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Fable 5.2, high reasoning; 1h limit Gemini 4 Hydrogen, high reasoning; 1h limit

Source: Official CWE-bench 2 leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5.2 high reasoning; 1h limit Claude Code 75% $7.85
Gemini 4 Neon high reasoning; 1h limit Antigravity 70.7% $6.52
Gemini 4 Hydrogen high reasoning; 1h limit Antigravity 68.1% $2.63
Gemini 4 Carbon high reasoning; 1h limit Antigravity 67.3% $4.27
Claude Opus 5.5 high reasoning; 1h limit Claude Code 67.1% $1.85
GPT-6.2 Astra high reasoning; 1h limit Codex 66.8% $4.32
Muse Wonder high reasoning; 1h limit opencode 66.1% $2.73
GPT-6.2 Sol high reasoning; 1h limit Codex 66% $0.81
Gemini 4 Argon high reasoning; 1h limit Antigravity 65% $3.70
GPT-6 Astra high reasoning; 1h limit Codex 64.2% $3.87
Tencent Hy4 high reasoning; 1h limit Claude Code 64.1% $1.35
Claude Sonnet 5.5 high reasoning; 1h limit Claude Code 61.4% $0.58
GPT-6.1 Terra high reasoning; 1h limit Codex 60% $0.44
DeepSeek V4.1 Flash 1017 high reasoning; 1h limit opencode 57.6% $0.04
Grok 4.7 high reasoning; 1h limit opencode 54.7% $2.97
Tencent Hy4 preview high reasoning; 1h limit opencode 47.3% $0.43
Mistral Large 4 high reasoning; 1h limit opencode 45.6% $0.37
Qwen3.8 Max (0902) high reasoning; 1h limit opencode 45.4% $2.06
Kimi K3.1 Pro high reasoning; 1h limit opencode 45.3% $0.96
GLM-5.3 high reasoning; 1h limit opencode 42% $0.58

2 models have held number one on the official leaderboard since its release in November 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind Anthropic
Own leaderboard 2 leaders

Source: Official CWE-bench 2 leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) Claude Fable 5.2 Gemini 4 Carbon GPT-6 Astra Tencent Hy4 preview GLM-5.3 0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) Claude Fable 5.2 Gemini 4 Carbon

Sources: Official CWE-bench 2 leaderboard and labs’ model cards

One lab put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
DateLaunch postScoreAs the post writes it
Nov 18, 2026 Gemini 4 Neon 70.7 CWE-Bench v2

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Nov 18, 2026 Gemini 4 Neon model card 70.7 CWE-Bench v2 (results table, p.4) Cyber risk

One lab claim next to the leaderboard

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 1 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
Gemini 4 Neon 70.7% CWE-Bench v2 70.7% Leaderboard, high reasoning; 1h limit, Antigravity Same

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic OpenAI Google DeepMind
CWE-bench 2 5 leaders
CWE-bench 3 leaders

Source: Official leaderboards

More in Cybersecurity

All 138 in Cybersecurity
  • ExploitGym In use 869 real-vulnerability exploit-development tasks (userspace, V8, Linux kernel). Success means capturing a flag through the intended vulnerability. GPT-6 Astra 42.4%
  • ExploitBench In use 41 recent V8 JavaScript-engine N-day vulnerabilities. Progress measured by 16 grader-verified capability flags across five tiers. GPT-6 Astra 100%
  • ExploitBench - Internal Port In use Agents develop working exploits for recently disclosed vulnerabilities to limit contamination. Scored as arbitrary code execution success rate. GPT-6.2 Astra 53.5%
  • Atomic Challenges In use Agents solve 22 skill-level offensive security challenges in vulnerability research, network attack simulation and evasion. Scored as challenges solved. GPT-6 Astra 90.9%
  • MiMo Cyber Bench In use Agents do in-house cybersecurity tasks that add to public vulnerability reproduction and exploitation benchmarks. The report gives no task count or grading details. MiMo-V2.6 Pro 81.7%
  • PatchBench (OpenAI) In use Models patch security flaws. Automated tests and model-based review check that each patch fixes the flaw, keeps the intended behavior and is ready to merge. GPT-6.2 Astra 79.7%

Data audit

All 12 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access Public tasks, plus a private test set.
  • Pass Confidence Checked against a primary source.
Scores 5 pass
  • Pass Independent runs Scores from the official leaderboard (20 entries), launch posts (1 score) and system cards (1 score).
  • Pass Models scored 20 models we track.
  • Pass Newest score The newest is from November 2026, for Gemini 4 Neon.
  • Pass Frontier models 8 of the top 10 have a score, led by GPT-6.2 Astra (#1).
  • Pass Headroom The best is 75%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on cwe-bench.com: 20 rows for models we track at the last read.
  • Pass Last read November 19th, 2026.
  • Pass Read worked The last read worked.
Consistency 2 pass
  • Pass Lab claims The one pair agrees within 10 points.
  • Pass Known issues No published audit found problems.

How we check