SEC-bench Pro
Version 260505
The official leaderboard hasn’t scored a model released after GPT-5.5 (April 2026). Labs report their own numbers for 14 newer models, such as Gemini 4 Neon at 90.5%.
Agents hunt and reproduce security bugs in large software projects from limited information. The first release covers V8 and SpiderMonkey.
The 260505 snapshot of 5 May 2026 holds 103 V8 and 80 SpiderMonkey instances, 183 in all. A later 260617 snapshot adds Linux and more SpiderMonkey, reaching 344. The maker calls the benchmark self-evolving.
- Released
- May 2026
- Kind
- Finding and exploiting real bugs
- Level
- Expert
- Scored
- Checked automatically
- Answers
- Agent task
- Tasks
- All public
- Measures
- Capability
- Risk
- Cyber offence
- Language
- English
- Interaction
- Agentic
- Environment
- Terminal
- Items from
- Real work and use
- Refresh
- New versions replace it
- Run by
- Anyone
- In views
- Agents, Dangerous capabilities
- Strongest model tested
- GPT-6.2 Astra #1 on the capability index
- Last in a launch post
- November 2026
- Last model added
- November 2026
- Scores refreshed
- November 20th, 2026
In the authors’ words
Finding a real vulnerability in complicated systems is a challenging, long-horizon task that demands reasoning across an entire codebase to produce a working proof-of-concept (PoC). However, such critical security problems remain understudied. We present SEC-bench Pro, a benchmark that measures how well frontier models hunt real vulnerabilities by reproducing working PoC inputs from disclosed reports, where each task pairs a concrete bug with the instructions for triggering it. We also demonstrate the limitations of existing rule-based judges for grading generated PoCs, and propose a novel LLM-based judge for more precise grading. We instantiate SEC-bench Pro with 344 validated vulnerabilities across three targets, the V8 and SpiderMonkey browser engines and the Linux kernel, covering critical vulnerability families including memory-safety, sandbox, JIT, race-condition, and kernel-subsystem bugs. Across six frontier commercial and open-weight models and three coding agents, the strongest, Codex with GPT-5.5, solves 58% of instances overall. We also observe that Claude Code with Opus 4.6 tends to time out but solves most instances it completes. In contrast, open-weight models struggle; for example, GLM-5 solves only 13 of the 344 instances. During construction and evaluation, SEC-bench Pro also surfaced three vulnerabilities in V8 and SpiderMonkey, including a sandbox escape that was fixed and earned a $20,000 Google Vulnerability Reward Program bounty. More recently, SEC-bench Pro has been adopted by OpenAI to evaluate the long-horizon security capabilities of its newest models. Overall, SEC-bench Pro exposes where long-horizon vulnerability discovery succeeds, where it fails, and how different grading choices change the evaluation landscape, offering insights for security-centric model evaluation and training. Our artifact is available at https://github.com/SEC-bench/SEC-bench-Pro.
Gemini 4 Neon leads with 90.5%
Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.
GPT-5.5 leads with 46.4% across all independent runs
Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.
Gemini 4 Neon leads with 90.5%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
- 1
Gemini 4 Neon 90.5%
- 2 GPT-6.2 Sol 88.3%
- 3
Gemini 4 Carbon 88.2%
- 4 GPT-6 Cyber 87%
- 5 GPT-6.2 Astra max effort 86.8%
- 6 GPT-6 Astra 85.4%
- 7 GPT-6.1 Terra 83.3%
- 8 GPT-6.1 Sol 78.8%
- 9 GPT-6 Sol 68.8%
- 10 MiMo-V2.6 Pro 66.3%
- 11
DeepSeek V4.1 Flash 62.8%
- 12 GPT-5.6 Terra 57.7%
- 13 GPT-6 Luna 51.8%
- 14 GPT-5.6 Luna 48.9%
Source: Labs’ model cards
- 1 GPT-5.5 xhigh 46.4%
- 2 GPT-5.4 xhigh 31.7%
- 3 Claude Opus 4.6 max 25.7%
Source: Official SEC-bench Pro leaderboard
- 90.5
Gemini 4 Neon
- 88.3 GPT-6.2 Sol
- 88.2
Gemini 4 Carbon
- 87.0 GPT-6 Cyber
- 86.8 GPT-6.2 Astra
- 85.4 GPT-6 Astra
- 83.3 GPT-6.1 Terra
- 78.8 GPT-6.1 Sol
- 68.8 GPT-6 Sol
- 66.3 MiMo-V2.6 Pro
- 62.8
DeepSeek V4.1 Flash
- 57.7 GPT-5.6 Terra
- 51.8 GPT-6 Luna
- 48.9 GPT-5.6 Luna
- 46.4 GPT-5.5
- 31.7 GPT-5.4
- 25.7 Claude Opus 4.6
- 1
Gemini 4 Neon 90.5%
- 2 GPT-6.2 Sol 88.3%
- 3
Gemini 4 Carbon 88.2%
- 4 GPT-6 Cyber 87%
- 5 GPT-6.2 Astra max effort 86.8%
- 6 GPT-6 Astra 85.4%
- 7 GPT-6.1 Terra 83.3%
- 8 GPT-6.1 Sol 78.8%
- 9 GPT-6 Sol 68.8%
- 10 MiMo-V2.6 Pro 66.3%
- 11
DeepSeek V4.1 Flash 62.8%
- 12 GPT-5.6 Terra 57.7%
- 13 GPT-6 Luna 51.8%
- 14 GPT-5.6 Luna 48.9%
- 15 GPT-5.5 xhigh 46.4%
- 16 GPT-5.4 xhigh 31.7%
- 17 Claude Opus 4.6 max 25.7%
Sources: Labs’ model cards and official SEC-bench Pro leaderboard
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 90.5% | Nov 2026 | Lab model card | |
| 2 | GPT-6.2 Sol | 88.3% | Nov 2026 | Lab model card | |
| 3 | | 88.2% | Oct 2026 | Lab model card | |
| 4 | GPT-6 Cyber | 87% | Nov 2026 | Lab model card | |
| 5 | GPT-6.2 Astra | 86.8% | max effort | Nov 2026 | Lab model card |
| 6 | GPT-6 Astra | 85.4% | Sep 2026 | Lab model card | |
| 7 | GPT-6.1 Terra | 83.3% | Oct 2026 | Lab model card | |
| 8 | GPT-6.1 Sol | 78.8% | Sep 2026 | Lab model card | |
| 9 | GPT-6 Sol | 68.8% | Oct 2026 | Lab model card | |
| 10 | MiMo-V2.6 Pro | 66.3% | Sep 2026 | Lab model card | |
| 11 | | 62.8% | Sep 2026 | Lab model card | |
| 12 | GPT-5.6 Terra | 57.7% | Jul 2026 | Lab model card | |
| 13 | GPT-6 Luna | 51.8% | Oct 2026 | Lab model card | |
| 14 | GPT-5.6 Luna | 48.9% | Jul 2026 | Lab model card |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-5.5 | 46.4% | xhigh, Codex | Apr 2026 | Official leaderboard |
| 2 | GPT-5.4 | 31.7% | xhigh, Codex | Mar 2026 | Official leaderboard |
| 3 | Claude Opus 4.6 | 25.7% | max, Claude Code | Feb 2026 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | 90.5% | Nov 2026 | Lab model card | ||
| 2 | GPT-6.2 Sol | 88.3% | Nov 2026 | Lab model card | |
| 3 | 88.2% | Oct 2026 | Lab model card | ||
| 4 | GPT-6 Cyber | 87% | Nov 2026 | Lab model card | |
| 5 | GPT-6.2 Astra | 86.8% | max effort | Nov 2026 | Lab model card |
| 6 | GPT-6 Astra | 85.4% | Sep 2026 | Lab model card | |
| 7 | GPT-6.1 Terra | 83.3% | Oct 2026 | Lab model card | |
| 8 | GPT-6.1 Sol | 78.8% | Sep 2026 | Lab model card | |
| 9 | GPT-6 Sol | 68.8% | Oct 2026 | Lab model card | |
| 10 | MiMo-V2.6 Pro | 66.3% | Sep 2026 | Lab model card | |
| 11 | 62.8% | Sep 2026 | Lab model card | ||
| 12 | GPT-5.6 Terra | 57.7% | Jul 2026 | Lab model card | |
| 13 | GPT-6 Luna | 51.8% | Oct 2026 | Lab model card | |
| 14 | GPT-5.6 Luna | 48.9% | Jul 2026 | Lab model card | |
| 15 | GPT-5.5 | 46.4% | xhigh, Codex | Apr 2026 | Official leaderboard |
| 16 | GPT-5.4 | 31.7% | xhigh, Codex | Mar 2026 | Official leaderboard |
| 17 | Claude Opus 4.6 | 25.7% | max, Claude Code | Feb 2026 | Official leaderboard |
The benchmark's own leaderboard has 3 models. Switch the scores above to rank them together.
Rankings by part
We read 2 parts from the benchmark’s own leaderboard: V8 and Firefox. Each ranks models on one part of the test. GPT-5.5 leads both. Arrows show how far a model moved from its overall rank among the models we track.
Firefox
3 models we track have a score here.
- 1 GPT-5.5 45%
- 2 Claude Opus 4.6 ↑1 30%
- 3 GPT-5.4 ↓1 27.5%
5 models have held number one in labs’ own numbers since its release in May 2026 Number one over time 6 models have held number one with claims included since its release in May 2026
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Sources: Official SEC-bench Pro leaderboard and labs’ model cards
OpenAI’s own number for GPT-6 Astra passed 85% in September 2026
That was 4 months after its release. By the same age, 24% of the benchmarks from 2026 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official SEC-bench Pro leaderboard and labs’ model cards
4 labs put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out. OpenAI accounts for 11 of the 15 releases that named it since 2024.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Nov 18, 2026 | 90.5 | SEC-bench Pro | |
| Nov 5, 2026 | GPT-6 Cyber | 87 | SEC-Bench Pro, pass@1 |
| Oct 28, 2026 | 88.2 | SEC-bench Pro | |
| Sep 22, 2026 | MiMo-V2.6 Pro | 66.3 | SEC Bench Pro |
| Sep 10, 2026 | 62.8 | SEC-Bench Pro (Pass@1) | |
| Jul 9, 2026 | GPT-5.6 Luna | 48.9 | SEC-Bench Pro |
| Jul 9, 2026 | GPT-5.6 Terra | 57.7 | SEC-Bench Pro |
21 system cards report it, from 4 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Nov 18, 2026 | | 90.5 | SEC-bench Pro (results table, p.4) | Cyber risk |
| Nov 17, 2026 | GPT-6.2 Sol system card | 88.3 | SEC-Bench Pro, vulnerability reproduction pass@1 (Table 4) | Cyber risk |
| Nov 17, 2026 | GPT-6.2 Astra system card | 86.8 | SEC-Bench Pro, vulnerability reproduction pass@1, maximum reasoning effort (Table 4) | Cyber risk |
| Nov 10, 2026 | | 88.2 | SEC-bench Pro (results table) | Cyber risk |
| Nov 5, 2026 | GPT-6 Cyber system card | 87 | SEC-Bench Pro, pass@1 | Cyber risk |
| Oct 29, 2026 | GPT-6 Luna system card addendum | 51.8 | SEC-Bench Pro score (8.1.2.2) | Cyber risk |
| Oct 26, 2026 | GPT-6.1 Terra system card addendum | 83.3 | SEC-Bench Pro, pass@1 (Figure 27, 9.0.2.2) | Cyber risk |
| Oct 7, 2026 | GPT-6 Sol system card addendum | 68.8 | SEC-Bench Pro score (8.1.2.2) | Cyber risk |
| Oct 7, 2026 | GPT-6 Luna system card addendum | 48.5 | SEC-Bench Pro score (8.1.2.2) | Cyber risk |
| Sep 29, 2026 | GPT-6.1 Sol system card addendum | 78.8 | SEC-Bench Pro, pass@1 (9.1.2.2) | Cyber risk |
| Sep 22, 2026 | MiMo-V2.6 Pro technical report | 66.3 | SEC Bench Pro (Table 3) | Cyber risk |
| Sep 22, 2026 | GPT-6 Sol system card | 66.3 | SEC-Bench Pro score (A.8.1.2.2) | Cyber risk |
| Sep 22, 2026 | GPT-6 Luna system card | 34.2 | SEC-Bench Pro score (A.8.1.2.2) | Cyber risk |
| Sep 10, 2026 | | 62.8 | SEC-Bench Pro version 260505 (Pass@1), Claude Code harness (Table 3) | Cyber risk |
| Sep 3, 2026 | GPT-6 Astra system card | 85.4 | SEC-bench Pro, May 2026 version with improved grader, printed chart label (Figure 48) | Cyber risk |
| Jul 9, 2026 | GPT-5.6 Terra system card | SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) | Cyber risk | |
| Jul 9, 2026 | GPT-5.6 Sol system card | SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) | Cyber risk | |
| Jul 9, 2026 | GPT-5.6 Luna system card | SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) | Cyber risk | |
| Jun 25, 2026 | GPT-5.6 Terra system card | SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) | Cyber risk | |
| Jun 25, 2026 | GPT-5.6 Sol system card | SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) | Cyber risk | |
| Jun 25, 2026 | GPT-5.6 Luna system card | SEC-bench Pro (May 2026 version), pass@1 by output tokens, chart without printed numbers (Figure 35, 9.1.2.4.3) | Cyber risk |
Versions
We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.
Sources: Official leaderboards and labs’ model cards
More in Cybersecurity
- ExploitGym In use 869 real-vulnerability exploit-development tasks (userspace, V8, Linux kernel). Success means capturing a flag through the intended vulnerability. GPT-6 Astra 42.4%
- ExploitBench In use 41 recent V8 JavaScript-engine N-day vulnerabilities. Progress measured by 16 grader-verified capability flags across five tiers. GPT-6 Astra 100%
- ExploitBench - Internal Port In use Agents develop working exploits for recently disclosed vulnerabilities to limit contamination. Scored as arbitrary code execution success rate. GPT-6.2 Astra 53.5%
- Atomic Challenges In use Agents solve 22 skill-level offensive security challenges in vulnerability research, network attack simulation and evasion. Scored as challenges solved. GPT-6 Astra 90.9%
- MiMo Cyber Bench In use Agents do in-house cybersecurity tasks that add to public vulnerability reproduction and exploitation benchmarks. The report gives no task count or grading details. MiMo-V2.6 Pro 81.7%
- PatchBench (OpenAI) In use Models patch security flaws. Automated tests and model-based review check that each patch fixes the flaw, keeps the intended behavior and is ready to merge. GPT-6.2 Astra 79.7%
Data audit
All 10 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Pass Independent runs Scores from the official leaderboard (6 entries), launch posts (7 scores) and system cards (15 scores).
- Pass Models scored 17 models we track, plus 3 other entries on its leaderboard.
- Pass Newest score The newest is from November 2026, for Gemini 4 Neon.
- Pass Frontier models 5 of the top 10 have a score, led by GPT-6.2 Astra (#1).
- Note Headroom Saturated. The best is 90.5%, past 85% since September 2026.
Leaderboard import 3 pass
- Pass Importer We read the board on sec-bench.github.io: 6 rows for models we track at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.