Benchmark Almanac

SEC-bench Pro

Gemini 4 Neon 90.5% leads since November 2026
GPT-5.5 46.4% leads since May 2026
Gemini 4 Neon 90.5% leads since November 2026
In use
Version 260505

The official leaderboard hasn’t scored a model released after GPT-5.5 (April 2026). Labs report their own numbers for 14 newer models, such as Gemini 4 Neon at 90.5%.

Agents hunt and reproduce security bugs in large software projects from limited information. The first release covers V8 and SpiderMonkey.

The 260505 snapshot of 5 May 2026 holds 103 V8 and 80 SpiderMonkey instances, 183 in all. A later 260617 snapshot adds Linux and more SpiderMonkey, reaching 344. The maker calls the benchmark self-evolving.

Released
May 2026
Kind
Finding and exploiting real bugs
Level
Expert
Scored
Checked automatically
Answers
Agent task
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Code → Code, Actions, Text
Interaction
Agentic
Environment
Terminal
Items from
Real work and use
Run by
Anyone
Strongest model tested
GPT-6.2 Astra #1 on the capability index
Last in a launch post
November 2026
Last model added
November 2026
Scores refreshed
November 20th, 2026

In the authors’ words

Finding a real vulnerability in complicated systems is a challenging, long-horizon task that demands reasoning across an entire codebase to produce a working proof-of-concept (PoC). However, such critical security problems remain understudied. We present SEC-bench Pro, a benchmark that measures how well frontier models hunt real vulnerabilities by reproducing working PoC inputs from disclosed reports, where each task pairs a concrete bug with the instructions for triggering it. We also demonstrate the limitations of existing rule-based judges for grading generated PoCs, and propose a novel LLM-based judge for more precise grading. We instantiate SEC-bench Pro with 344 validated vulnerabilities across three targets, the V8 and SpiderMonkey browser engines and the Linux kernel, covering critical vulnerability families including memory-safety, sandbox, JIT, race-condition, and kernel-subsystem bugs. Across six frontier commercial and open-weight models and three coding agents, the strongest, Codex with GPT-5.5, solves 58% of instances overall. We also observe that Claude Code with Opus 4.6 tends to time out but solves most instances it completes. In contrast, open-weight models struggle; for example, GLM-5 solves only 13 of the 344 instances. During construction and evaluation, SEC-bench Pro also surfaced three vulnerabilities in V8 and SpiderMonkey, including a sandbox escape that was fixed and earned a $20,000 Google Vulnerability Reward Program bounty. More recently, SEC-bench Pro has been adopted by OpenAI to evaluate the long-horizon security capabilities of its newest models. Overall, SEC-bench Pro exposes where long-horizon vulnerability discovery succeeds, where it fails, and how different grading choices change the evaluation landscape, offering insights for security-centric model evaluation and training. Our artifact is available at https://github.com/SEC-bench/SEC-bench-Pro.

Abstract of SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks? (arXiv:2605.26548, CC0)
14
Models scored in lab cards
7
Launch posts list it, from 4 labs
21
System cards report it, from 4 labs

Gemini 4 Neon leads with 90.5%

Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.

GPT-5.5 leads with 46.4% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.

Gemini 4 Neon leads with 90.5%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.

  1. 1 Gemini 4 Neon 90.5%
  2. 2 GPT-6.2 Sol 88.3%
  3. 3 Gemini 4 Carbon 88.2%
  4. 4 GPT-6 Cyber 87%
  5. 5 GPT-6.2 Astra max effort 86.8%
  6. 6 GPT-6 Astra 85.4%
  7. 7 GPT-6.1 Terra 83.3%
  8. 8 GPT-6.1 Sol 78.8%
  9. 9 GPT-6 Sol 68.8%
  10. 10 MiMo-V2.6 Pro 66.3%
  11. 11 DeepSeek V4.1 Flash 62.8%
  12. 12 GPT-5.6 Terra 57.7%
  13. 13 GPT-6 Luna 51.8%
  14. 14 GPT-5.6 Luna 48.9%

Source: Labs’ model cards

Source: Official SEC-bench Pro leaderboard

#ModelScoreSettingDateSource
1 Gemini 4 Neon 90.5% Nov 2026 Lab model card
2 GPT-6.2 Sol 88.3% Nov 2026 Lab model card
3 Gemini 4 Carbon 88.2% Oct 2026 Lab model card
4 GPT-6 Cyber 87% Nov 2026 Lab model card
5 GPT-6.2 Astra 86.8% max effort Nov 2026 Lab model card
6 GPT-6 Astra 85.4% Sep 2026 Lab model card
7 GPT-6.1 Terra 83.3% Oct 2026 Lab model card
8 GPT-6.1 Sol 78.8% Sep 2026 Lab model card
9 GPT-6 Sol 68.8% Oct 2026 Lab model card
10 MiMo-V2.6 Pro 66.3% Sep 2026 Lab model card
11 DeepSeek V4.1 Flash 62.8% Sep 2026 Lab model card
12 GPT-5.6 Terra 57.7% Jul 2026 Lab model card
13 GPT-6 Luna 51.8% Oct 2026 Lab model card
14 GPT-5.6 Luna 48.9% Jul 2026 Lab model card
#ModelScoreSettingDateSource
1 GPT-5.5 46.4% xhigh, Codex Apr 2026 Official leaderboard
2 GPT-5.4 31.7% xhigh, Codex Mar 2026 Official leaderboard
3 Claude Opus 4.6 25.7% max, Claude Code Feb 2026 Official leaderboard
#ModelScoreSettingDateSource
1 Gemini 4 Neon 90.5% Nov 2026 Lab model card
2 GPT-6.2 Sol 88.3% Nov 2026 Lab model card
3 Gemini 4 Carbon 88.2% Oct 2026 Lab model card
4 GPT-6 Cyber 87% Nov 2026 Lab model card
5 GPT-6.2 Astra 86.8% max effort Nov 2026 Lab model card
6 GPT-6 Astra 85.4% Sep 2026 Lab model card
7 GPT-6.1 Terra 83.3% Oct 2026 Lab model card
8 GPT-6.1 Sol 78.8% Sep 2026 Lab model card
9 GPT-6 Sol 68.8% Oct 2026 Lab model card
10 MiMo-V2.6 Pro 66.3% Sep 2026 Lab model card
11 DeepSeek V4.1 Flash 62.8% Sep 2026 Lab model card
12 GPT-5.6 Terra 57.7% Jul 2026 Lab model card
13 GPT-6 Luna 51.8% Oct 2026 Lab model card
14 GPT-5.6 Luna 48.9% Jul 2026 Lab model card
15 GPT-5.5 46.4% xhigh, Codex Apr 2026 Official leaderboard
16 GPT-5.4 31.7% xhigh, Codex Mar 2026 Official leaderboard
17 Claude Opus 4.6 25.7% max, Claude Code Feb 2026 Official leaderboard

The benchmark's own leaderboard has 3 models. Switch the scores above to rank them together.

Rankings by part

We read 2 parts from the benchmark’s own leaderboard: V8 and Firefox. Each ranks models on one part of the test. GPT-5.5 leads both. Arrows show how far a model moved from its overall rank among the models we track.

V8

3 models we track have a score here.

  1. 1 GPT-5.5 47.6%
  2. 2 GPT-5.4 35%
  3. 3 Claude Opus 4.6 22.3%

5 models have held number one in labs’ own numbers since its release in May 2026 Number one over time 6 models have held number one with claims included since its release in May 2026

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

OpenAI Google DeepMind
Own leaderboard Replaced by 260617, Jun 2026
GPT-5.5 46.4%
Lab claims Passed 85% in Sep 2026, by GPT-6 Astra

Sources: Official SEC-bench Pro leaderboard and labs’ model cards

OpenAI’s own number for GPT-6 Astra passed 85% in September 2026

That was 4 months after its release. By the same age, 24% of the benchmarks from 2026 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Gemini 4 Neon GPT-6 Astra GPT-5.6 Terra GPT-5.5 GPT-5.4 Claude Opus 4.6 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Gemini 4 Neon GPT-5.6 Terra

Sources: Official SEC-bench Pro leaderboard and labs’ model cards

4 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out. OpenAI accounts for 11 of the 15 releases that named it since 2024.

OpenAI
DeepSeek
Xiaomi Corp
Google DeepMind
DateLaunch postScoreAs the post writes it
Nov 18, 2026 Gemini 4 Neon 90.5 SEC-bench Pro
Nov 5, 2026 GPT-6 Cyber 87 SEC-Bench Pro, pass@1
Oct 28, 2026 Gemini 4 Carbon 88.2 SEC-bench Pro
Sep 22, 2026 MiMo-V2.6 Pro 66.3 SEC Bench Pro
Sep 10, 2026 DeepSeek V4.1 Flash 62.8 SEC-Bench Pro (Pass@1)
Jul 9, 2026 GPT-5.6 Luna 48.9 SEC-Bench Pro
Jul 9, 2026 GPT-5.6 Terra 57.7 SEC-Bench Pro

21 system cards report it, from 4 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Nov 18, 2026 Gemini 4 Neon model card 90.5 SEC-bench Pro (results table, p.4) Cyber risk
Nov 17, 2026 GPT-6.2 Sol system card 88.3 SEC-Bench Pro, vulnerability reproduction pass@1 (Table 4) Cyber risk
Nov 17, 2026 GPT-6.2 Astra system card 86.8 SEC-Bench Pro, vulnerability reproduction pass@1, maximum reasoning effort (Table 4) Cyber risk
Nov 10, 2026 Gemini 4 Carbon model card 88.2 SEC-bench Pro (results table) Cyber risk
Nov 5, 2026 GPT-6 Cyber system card 87 SEC-Bench Pro, pass@1 Cyber risk
Oct 29, 2026 GPT-6 Luna system card addendum 51.8 SEC-Bench Pro score (8.1.2.2) Cyber risk
Oct 26, 2026 GPT-6.1 Terra system card addendum 83.3 SEC-Bench Pro, pass@1 (Figure 27, 9.0.2.2) Cyber risk
Oct 7, 2026 GPT-6 Sol system card addendum 68.8 SEC-Bench Pro score (8.1.2.2) Cyber risk
Oct 7, 2026 GPT-6 Luna system card addendum 48.5 SEC-Bench Pro score (8.1.2.2) Cyber risk
Sep 29, 2026 GPT-6.1 Sol system card addendum 78.8 SEC-Bench Pro, pass@1 (9.1.2.2) Cyber risk
Sep 22, 2026 MiMo-V2.6 Pro technical report 66.3 SEC Bench Pro (Table 3) Cyber risk
Sep 22, 2026 GPT-6 Sol system card 66.3 SEC-Bench Pro score (A.8.1.2.2) Cyber risk
Sep 22, 2026 GPT-6 Luna system card 34.2 SEC-Bench Pro score (A.8.1.2.2) Cyber risk
Sep 10, 2026 DeepSeek V4.1 Flash technical report 62.8 SEC-Bench Pro version 260505 (Pass@1), Claude Code harness (Table 3) Cyber risk
Sep 3, 2026 GPT-6 Astra system card 85.4 SEC-bench Pro, May 2026 version with improved grader, printed chart label (Figure 48) Cyber risk
Jul 9, 2026 GPT-5.6 Terra system card SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) Cyber risk
Jul 9, 2026 GPT-5.6 Sol system card SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) Cyber risk
Jul 9, 2026 GPT-5.6 Luna system card SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) Cyber risk
Jun 25, 2026 GPT-5.6 Terra system card SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) Cyber risk
Jun 25, 2026 GPT-5.6 Sol system card SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) Cyber risk
Jun 25, 2026 GPT-5.6 Luna system card SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) Cyber risk

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

OpenAI Google DeepMind
SEC-bench Pro 260617 Not tested since Jun 2026
GPT-5.5 58.4%
SEC-bench Pro Passed 85% in Sep 2026, by GPT-6 Astra
SEC-bench Pro 260617 Not tested since Jun 2026
GPT-5.5 58.4%

Sources: Official leaderboards and labs’ model cards

More in Cybersecurity

All 138 in Cybersecurity
  • ExploitGym In use 869 real-vulnerability exploit-development tasks (userspace, V8, Linux kernel). Success means capturing a flag through the intended vulnerability. GPT-6 Astra 42.4%
  • ExploitBench In use 41 recent V8 JavaScript-engine N-day vulnerabilities. Progress measured by 16 grader-verified capability flags across five tiers. GPT-6 Astra 100%
  • ExploitBench - Internal Port In use Agents develop working exploits for recently disclosed vulnerabilities to limit contamination. Scored as arbitrary code execution success rate. GPT-6.2 Astra 53.5%
  • Atomic Challenges In use Agents solve 22 skill-level offensive security challenges in vulnerability research, network attack simulation and evasion. Scored as challenges solved. GPT-6 Astra 90.9%
  • MiMo Cyber Bench In use Agents do in-house cybersecurity tasks that add to public vulnerability reproduction and exploitation benchmarks. The report gives no task count or grading details. MiMo-V2.6 Pro 81.7%
  • PatchBench (OpenAI) In use Models patch security flaws. Automated tests and model-based review check that each patch fixes the flaw, keeps the intended behavior and is ready to merge. GPT-6.2 Astra 79.7%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (6 entries), launch posts (7 scores) and system cards (15 scores).
  • Pass Models scored 17 models we track, plus 3 other entries on its leaderboard.
  • Pass Newest score The newest is from November 2026, for Gemini 4 Neon.
  • Pass Frontier models 5 of the top 10 have a score, led by GPT-6.2 Astra (#1).
  • Note Headroom Saturated. The best is 90.5%, past 85% since September 2026.
Leaderboard import 3 pass
  • Pass Importer We read the board on sec-bench.github.io: 6 rows for models we track at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check