CRSBench

Claude Opus 4.6 82.3% leads since February 2026

Only in system cards

Official leaderboard Include claimed scores

Cyber reasoning systems find and patch bugs in 124 C, C++ and Java projects with 315 vulnerabilities. A patch passes only if no ground-truth proof of vulnerability still triggers and functionality tests pass.

The GitHub repo was created on October 31st, 2025, and the public release month is not stated. The dataset is gated behind a data use agreement.

Made by: Georgia Tech SSLab (OpenSSF OSS-CRS)
Kind: Finding and exploiting real bugs
Level: Expert
Scored: Checked automatically
Answers: Agent task
Tasks: All public

Strongest model tested: Claude Opus 4.6 #37 on the capability index
Last in a system card: July 2026
Last model added: July 2026
Scores refreshed: November 12th, 2026

Models scored on its own leaderboard

System card reports it, from 1 lab

Claude Opus 4.6 leads with 82.3% among models released by April 2026

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best.

Claude Opus 4.6 leads with 82.3%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

Bars Table

The newest model with a score here came out in April 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

1 Claude Opus 4.6 82.3%
2 GLM-5.1 68.6%
3 Gemini 3 Flash 56.9%
4 GPT-5.4 Mini 54.9%
5 Claude Haiku 4.5 31.4%

Source: Official CRSBench leaderboard

Sources: Official CRSBench leaderboard and labs’ model cards

#	Model	Score	Setting	Date	Source
1	Claude Opus 4.6	82.3%	Claude Code	Feb 2026	Official leaderboard
2	GLM-5.1	68.6%	Opencode	Apr 2026	Official leaderboard
3	Gemini 3 Flash	56.9%	Gemini CLI	Dec 2025	Official leaderboard
4	GPT-5.4 Mini	54.9%	Codex	Mar 2026	Official leaderboard
5	Claude Haiku 4.5	31.4%	Claude Code	Oct 2025	Official leaderboard

#	Model	Score	Setting	Date	Source
1	Claude Opus 4.6	82.3%	Claude Code	Feb 2026	Official leaderboard
2	GLM-5.1	68.6%	Opencode	Apr 2026	Official leaderboard
3	MAI-Cyber-1-Flash	65.1%		Jul 2026	Lab model card
4	Gemini 3 Flash	56.9%	Gemini CLI	Dec 2025	Official leaderboard
5	GPT-5.4 Mini	54.9%	Codex	Mar 2026	Official leaderboard
6	Claude Haiku 4.5	31.4%	Claude Code	Oct 2025	Official leaderboard

Labs report scores for 1 model in their cards. Switch the scores above to rank them together.

3 models have held number one on the official leaderboard since 2023 3 models have held number one with claims included since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic Google DeepMind

Own leaderboard Not tested since Apr 2026

Claude Opus 4.6 82.3%

Source: Official CRSBench leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far

Sources: Official CRSBench leaderboard and labs’ model cards

Labs report it only in system cards

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark. Some system cards do, below.

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

Date	Document	Score	As the document writes it	Section
Jul 27, 2026	MAI-Cyber-1-Flash model card	65.1	CRSBench, printed 0.651 (POV=1200), metric not defined in the card (External Cyber Benchmark Results)	Cyber risk