Benchmark Almanac

Firefox 147 exploit evaluation (Anthropic)

Claude Mythos Preview 72.4% leads since April 2026
Only in system cards

Internal eval on crashes previously found in Firefox 147, run without the browser's process sandbox. Measures exploit-development progress.

Made by
Anthropic
Kind
Finding and exploiting real bugs
Level
Research frontier
Scored
Checked automatically
Answers
Agent task
Tasks
Kept private
Strongest model tested
Claude Opus 5 #10 on the capability index
Last in a system card
July 2026
Last model added
July 2026
5
Models scored in lab cards
5
System cards report it, from 1 lab

Claude Mythos Preview leads with 72.4% among models released by July 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in July 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 72.4 Claude Mythos Preview
  2. 52.4 Claude Opus 5
  3. 45.2 Claude Opus 4.7
  4. 13.2 Claude Sonnet 5
  5. 8.80 Claude Opus 4.8
  1. 1 Claude Mythos Preview 72.4%
  2. 2 Claude Opus 5 52.4%
  3. 3 Claude Opus 4.7 45.2%
  4. 4 Claude Sonnet 5 13.2%
  5. 5 Claude Opus 4.8 8.80%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 Claude Mythos Preview 72.4% Apr 2026 Lab model card
2 Claude Opus 5 52.4% Jul 2026 Lab model card
3 Claude Opus 4.7 45.2% Apr 2026 Lab model card
4 Claude Sonnet 5 13.2% Jun 2026 Lab model card
5 Claude Opus 4.8 8.80% May 2026 Lab model card

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Lab claims Not tested since Jul 2026
Claude Mythos Preview 72.4%

Source: Labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Mythos Preview 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Mythos Preview

Source: Labs’ model cards

Labs report it only in system cards

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark. Some system cards do, below.

5 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Jul 24, 2026 Claude Opus 5 system card 52.4 Firefox 147, full working exploits in 131/250 trials, partial progress in 218 (87.2%), mitigations off (3.3.3) Cyber risk
Jun 30, 2026 Claude Sonnet 5 system card 13.2 Firefox 147 exploit development, any-success (register control or better) 33/250 trials, full working exploit 0.0%, mitigations off (0 with mitigations) (3.2.4, Figure 3.2.4.A) Cyber risk
May 28, 2026 Claude Opus 4.8 system card 8.8 Firefox 147 JS shell exploitation, full working exploit (grade 1.0) in 22 of 250 trials, at least register control 68.8%, safeguards off. No points with safeguards on (3.3.3, Figure 3.3.3.A) Cyber risk
Apr 16, 2026 Claude Opus 4.7 system card 45.2 Firefox 147 JS shell exploitation, success rate with partial control or better over 250 trials, printed on chart. Full code execution 1.2% (3.3.3, Figure 3.3.3.A) Cyber risk
Apr 7, 2026 Claude Mythos Preview system card 72.4 Firefox 147 JS shell exploitation, full code execution rate over 250 trials, value printed on chart; 84.0 including partial control. Variant without the top 2 bugs: 4.4 full, 85.2 including partial (3.3.3, Figures 3.3.3.A and 3.3.3.B) Cyber risk

More in Cybersecurity

All 115 in Cybersecurity
  • ExploitGym In use 869 real-vulnerability exploit-development tasks (userspace, V8, Linux kernel). Success means capturing a flag through the intended vulnerability. GPT-6 Astra 42.4%
  • ExploitBench In use 41 recent V8 JavaScript-engine N-day vulnerabilities. Progress measured by 16 grader-verified capability flags across five tiers. GPT-6 Astra 100%
  • CyberBench v1.1 (Vals) In use Agents work on OSS-Fuzz memory-safety bugs from ARVO images in two tracks. They write a proof-of-concept input that crashes the target, or patch the source so the crash stops and tests still pass. Muse Wonder 80.4%
  • SRE-Bench In use Software reverse engineering without source: 262 binary instances from 19 private programs, each with six graded objectives. GPT-6 Astra 56.9%
  • CyScenarioBench In use Scenario-based long-horizon security challenges in sandboxes, organized in task, scenario and campaign tiers. Evaluation set kept private. Gemini 4 Argon 100%
  • ExploitBench - Internal Port In use Agents develop working exploits for recently disclosed vulnerabilities to limit contamination. Scored as arbitrary code execution success rate. GPT-6 Cyber 42%