Multimodal Troubleshooting Virology
Models troubleshoot virology lab scenarios, sometimes with images, and must select every correct answer. Scored as refusal-adjusted pass@1.
Questions come from SecureBio. OpenAI Preparedness evaluation.
- Made by
- OpenAI with SecureBio
- Released
- December 2024
- Kind
- Lab protocols
- Level
- Expert
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- Kept private
- Measures
- Capability
- Risk
- CBRN
- Subject
- Biological sciences
- Language
- English
- Interaction
- Single turn
- Items from
- Written by experts
- Refresh
- One fixed set
- Run by
- A model developer
- Based on
- Virology Capabilities Test (VCT)
- Strongest model tested
- GPT-6.2 Astra #1 on the capability index
- Last in a system card
- November 2026
- Last model added
- November 2026
GPT-Rosalind-6 leads with 63.1%
Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.
- 63.1 GPT-Rosalind-6
- 61.8 GPT-6.2 Astra
- 59.0 o1
- 59.0 o3
- 58.0 o4-mini
- 56.0 GPT-4.5
- 55.8 GPT-6 Astra
- 55.5 GPT-5.6 Sol
- 55.4 GPT-6.2 Sol
- 55.3 GPT-5.5 Pro
- 55.3 GPT-6.1 Sol
- 55.0 OpenAI deep research
- 55.0 gpt-oss-120b
- 53.7 GPT-5.5
- 53.0 o3-mini
- 52.5 GPT-6.1 Terra
- 51.7 GPT-6 Sol
- 51.1 GPT-5.5 Instant
- 50.4 GPT-5.4
- 50.2 GPT-5.6 Luna
- 50.0 gpt-oss-20b
- 50.0 GPT-5.3 Codex
- 49.1 GPT-6 Luna
- 48.0 GPT-5.2 Codex
- 46.5 GPT-5.4 Mini
- 43.0 GPT-5.2
- 42.0 GPT-5
- 39.0 GPT-5 mini
- 38.0 GPT-5.1-Codex-Max
- 1 GPT-Rosalind-6 63.1%
- 2 GPT-6.2 Astra 61.8%
- 3 o1 59%
- 4 o3 59%
- 5 o4-mini 58%
- 6 GPT-4.5 56%
- 7 GPT-6 Astra 55.8%
- 8 GPT-5.6 Sol 55.5%
- 9 GPT-6.2 Sol 55.4%
- 10 GPT-5.5 Pro 55.3%
- 11 GPT-6.1 Sol 55.3%
- 12 OpenAI deep research 55%
- 13 gpt-oss-120b 55%
- 14 GPT-5.5 53.7%
- 15 o3-mini 53%
- 16 GPT-6.1 Terra 52.5%
- 17 GPT-6 Sol 51.7%
- 18 GPT-5.5 Instant 51.1%
- 19 GPT-5.4 50.4%
- 20 GPT-5.6 Luna 50.2%
- 21 gpt-oss-20b 50%
- 22 GPT-5.3 Codex 50%
- 23 GPT-6 Luna 49.1%
- 24 GPT-5.2 Codex 48%
- 25 GPT-5.4 Mini 46.5%
- 26 GPT-5.2 43%
- 27 GPT-5 42%
- 28 GPT-5 mini 39%
- 29 GPT-5.1-Codex-Max 38%
Source: Labs’ model cards
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-Rosalind-6 | 63.1% | Oct 2026 | Lab model card | |
| 2 | GPT-6.2 Astra | 61.8% | Nov 2026 | Lab model card | |
| 3 | o1 | 59% | Dec 2024 | Lab model card | |
| 4 | o3 | 59% | Apr 2025 | Lab model card | |
| 5 | o4-mini | 58% | Apr 2025 | Lab model card | |
| 6 | GPT-4.5 | 56% | Feb 2025 | Lab model card | |
| 7 | GPT-6 Astra | 55.8% | Sep 2026 | Lab model card | |
| 8 | GPT-5.6 Sol | 55.5% | Jun 2026 | Lab model card | |
| 9 | GPT-6.2 Sol | 55.4% | Nov 2026 | Lab model card | |
| 10 | GPT-5.5 Pro | 55.3% | Apr 2026 | Lab model card | |
| 11 | GPT-6.1 Sol | 55.3% | Sep 2026 | Lab model card | |
| 12 | OpenAI deep research | 55% | Feb 2025 | Lab model card | |
| 13 | gpt-oss-120b | 55% | Aug 2025 | Lab model card | |
| 14 | GPT-5.5 | 53.7% | Apr 2026 | Lab model card | |
| 15 | o3-mini | 53% | Jan 2025 | Lab model card | |
| 16 | GPT-6.1 Terra | 52.5% | Oct 2026 | Lab model card | |
| 17 | GPT-6 Sol | 51.7% | Oct 2026 | Lab model card | |
| 18 | GPT-5.5 Instant | 51.1% | May 2026 | Lab model card | |
| 19 | GPT-5.4 | 50.4% | Mar 2026 | Lab model card | |
| 20 | GPT-5.6 Luna | 50.2% | Aug 2026 | Lab model card | |
| 21 | gpt-oss-20b | 50% | Aug 2025 | Lab model card | |
| 22 | GPT-5.3 Codex | 50% | Feb 2026 | Lab model card | |
| 23 | GPT-6 Luna | 49.1% | Sep 2026 | Lab model card | |
| 24 | GPT-5.2 Codex | 48% | Dec 2025 | Lab model card | |
| 25 | GPT-5.4 Mini | 46.5% | Mar 2026 | Lab model card | |
| 26 | GPT-5.2 | 43% | Dec 2025 | Lab model card | |
| 27 | GPT-5 | 42% | Aug 2025 | Lab model card | |
| 28 | GPT-5 mini | 39% | Aug 2025 | Lab model card | |
| 29 | GPT-5.1-Codex-Max | 38% | Nov 2025 | Lab model card |
2 models have held number one in labs’ own numbers since its release in December 2024
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Labs’ model cards
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Labs’ model cards
Labs report it only in system cards
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark. Some system cards do, below.
39 system cards report it, from 1 lab
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests. OpenAI accounts for 30 of the 30 releases that named it since 2024.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Nov 17, 2026 | GPT-6.2 Sol system card | 55.4 | Multi-select Virology Troubleshooting, pass@1 observed (refusal-adjusted 55.71%) (Table 2, 2.1.1.1.2) | Bio and chem risk |
| Nov 17, 2026 | GPT-6.2 Astra system card | 61.8 | Multi-select Virology Troubleshooting, pass@1 observed (refusal-adjusted 61.93%) (Table 2, 2.1.1.1.2) | Bio and chem risk |
| Oct 30, 2026 | GPT-Rosalind-6 system card | 63.1 | SecureBio multimodal virology troubleshooting, refusal-adjusted pass@1 | Bio and chem risk |
| Oct 29, 2026 | GPT-6 Luna system card addendum | 45.9 | Multimodal Troubleshooting Virology, pass@1 (Table 8) | Bio and chem risk |
| Oct 26, 2026 | GPT-6.1 Terra system card addendum | 52.5 | Multimodal Troubleshooting Virology, refusal-adjusted pass@1 (46.52% excluding refusals, printed chart label) (table and Figure 21, 9.0.1.1) | Bio and chem risk |
| Oct 7, 2026 | GPT-6 Sol system card addendum | 51.7 | Multimodal Troubleshooting Virology, pass@1 (Table 8) | Bio and chem risk |
| Oct 7, 2026 | GPT-6 Luna system card addendum | 48.1 | Multimodal Troubleshooting Virology, pass@1 (Table 8) | Bio and chem risk |
| Sep 29, 2026 | GPT-6.1 Sol system card addendum | 55.3 | Multimodal Troubleshooting Virology, refusal-adjusted pass@1 (9.1.1.1) | Bio and chem risk |
| Sep 22, 2026 | GPT-6 Sol system card | 50.6 | Multimodal Troubleshooting Virology, pass@1 (Table 33, A.8.1.1.1) | Bio and chem risk |
| Sep 22, 2026 | GPT-6 Luna system card | 49.1 | Multimodal Troubleshooting Virology, pass@1 (Table 33, A.8.1.1.1) | Bio and chem risk |
| Sep 3, 2026 | GPT-6 Astra system card | 55.8 | Multimodal Troubleshooting Virology, pass@1 (refusal-adjusted 63.11%) (10.1.1.1.1) | Bio and chem risk |
| Aug 6, 2026 | GPT-5.6 Sol system card addendum | 49.4 | Multimodal Troubleshooting Virology, pass@1 with refusals or safe completions counted (49.03% without), printed chart label (Figure 3, 7.1.1.1) | Bio and chem risk |
| Aug 6, 2026 | GPT-5.6 Luna system card addendum | 50.2 | Multimodal Troubleshooting Virology, pass@1 with refusals or safe completions counted (49.72% without), printed chart label (Figure 3, 7.1.1.1) | Bio and chem risk |
| Jul 9, 2026 | GPT-5.6 Terra system card | Multimodal Troubleshooting Virology, pass@1 by cost and latency, chart without printed numbers (Figures 22, 23, 9.1.1.1) | Bio and chem risk | |
| Jul 9, 2026 | GPT-5.6 Sol system card | 55.5 | Multimodal Troubleshooting Virology, pass@1, top score stated in text (9.1.1.1) | Bio and chem risk |
| Jul 9, 2026 | GPT-5.6 Luna system card | Multimodal Troubleshooting Virology, pass@1 by cost and latency, chart without printed numbers (Figures 22, 23, 9.1.1.1) | Bio and chem risk | |
| Jun 25, 2026 | GPT-5.6 Terra system card | Multimodal Troubleshooting Virology, pass@1 by cost and latency, chart without printed numbers (Figures 22, 23, 9.1.1.1) | Bio and chem risk | |
| Jun 25, 2026 | GPT-5.6 Sol system card | 55.5 | Multimodal Troubleshooting Virology, pass@1, top score stated in text (9.1.1.1) | Bio and chem risk |
| Jun 25, 2026 | GPT-5.6 Luna system card | Multimodal Troubleshooting Virology, pass@1 by cost and latency, chart without printed numbers (Figures 22, 23, 9.1.1.1) | Bio and chem risk | |
| May 4, 2026 | GPT-5.5 Instant system card | 51.1 | Multi-select Virology Troubleshooting, pass@1 (52.91% counting refusals or safe completions), printed chart label (Figure 3, 8.1.1.1) | Bio and chem risk |
| Apr 23, 2026 | GPT-5.5 system card | 53.7 | Multi-select virology troubleshooting, pass@1, printed chart label (Figure 12, 9.1.1.1) | Bio and chem risk |
| Apr 23, 2026 | GPT-5.5 Pro system card | 55.3 | Multi-select virology troubleshooting, pass@1, GPT-5.5 Pro, printed chart label (Figure 12, 9.1.1.1) | Bio and chem risk |
| Mar 17, 2026 | GPT-5.4 Mini system card | 46.5 | Multi-select Multimodal Troubleshooting Virology, pass@1 (47.0% counting refusals) (Table 18) | Bio and chem risk |
| Mar 5, 2026 | GPT-5.4 system card | 50.4 | Multi-select Multimodal Troubleshooting Virology, pass@1, printed chart label (Figure 8, 5.1.1.1) | Bio and chem risk |
| Feb 5, 2026 | GPT-5.3 Codex system card | 50 | Multimodal troubleshooting virology, pass@1, printed chart label (Figure 3, 5.1.1.3) | Bio and chem risk |
| Dec 18, 2025 | GPT-5.2 Codex system card addendum | 48 | Multimodal Troubleshooting Virology (Multi-select), pass@1, printed chart label (Figure 1, 5.1.1.1) | Bio and chem risk |
| Dec 11, 2025 | GPT-5.2 system card addendum | 43 | Multimodal Troubleshooting Virology (Multi-select), pass@1, gpt-5.2 browsing, printed chart label (Figure 5) | Bio and chem risk |
| Nov 18, 2025 | GPT-5.1-Codex-Max system card | 38 | Multimodal Troubleshooting Virology (Multi-select), pass@1, printed chart label (45% with refusals counted) (Figure 2, 5.1.1.2) | Bio and chem risk |
| Aug 7, 2025 | GPT-5 system card | 42 | Multimodal Troubleshooting Virology (Multi-select), pass@1, gpt-5-thinking browsing launch candidate (Figure 10) | Bio and chem risk |
| Aug 7, 2025 | GPT-5 mini system card | 39 | Multimodal Troubleshooting Virology (Multi-select), pass@1, gpt-5-thinking-mini browsing launch candidate (Figure 10) | Bio and chem risk |
| Aug 5, 2025 | gpt-oss-20b model card | 50 | Multimodal Troubleshooting Virology, cons@32 axis, gpt-oss 20b browsing launch candidate (Figure 6) | Bio and chem risk |
| Aug 5, 2025 | gpt-oss-120b model card | 55 | Multimodal Troubleshooting Virology, cons@32 axis, gpt-oss 120b browsing launch candidate (Figure 6) | Bio and chem risk |
| Jul 17, 2025 | ChatGPT agent system card | Multimodal Troubleshooting Virology, cons@32, chart shows only ChatGPT Agent browsing helpful-only at 57%, no released-model value (Figure 2, 5.1.1.2) | Bio and chem risk | |
| Apr 16, 2025 | o4-mini system card | 58 | Multimodal Troubleshooting Virology, cons@32, o4-mini no browsing launch candidate, printed chart label (Figure 4) | Bio and chem risk |
| Apr 16, 2025 | o3 system card | 59 | Multimodal Troubleshooting Virology, cons@32, o3 no browsing launch candidate, printed chart label (Figure 4) | Bio and chem risk |
| Feb 27, 2025 | GPT-4.5 system card | 56 | Multimodal Troubleshooting Virology, single select, post-mitigation (4.3.2) | Bio and chem risk |
| Feb 25, 2025 | OpenAI deep research system card | 55 | Multimodal Troubleshooting Virology, single select, cons@32, post-mitigation with browsing, text and printed chart label (without browsing 56%) (3.4.4) | Bio and chem risk |
| Jan 31, 2025 | o3-mini system card | 53 | Multimodal Troubleshooting Virology, single select, post-mitigation (5.4.5) | Bio and chem risk |
| Dec 5, 2024 | o1 system card | 59 | Multimodal Troubleshooting Virology, single select, cons@32, o1 (post-mitigation) (5.5.5 chart and text) | Bio and chem risk |
More in Biology and chemistry
- GeneBench Pro In use Agents analyze messy simulated genomics and biology datasets to estimate a decision-relevant quantity across 129 problems. Scored automatically against known answers. GPT-5.6 Sol 31.5%
- MedChemBench (OpenAI) In use Models work through 99 medicinal chemistry problems drawn from recent papers and patents. Each task mixes prose with structures, assay tables and plots, then asks for the program’s next decision. GPT-6 Astra 83.8%
- SUPERChem In use Models solve 500 expert-written chemistry reasoning problems, offered in multimodal and text-only forms. Expert solution paths also allow a reasoning path score.
Tencent Hy4 preview 66.4%
- LifeSciBench In use Models complete 750 expert-written life science research tasks, often with attached figures, tables, or sequence files. Graded against expert rubrics as score and pass rate. GPT-Rosalind-6 61.3%
- ProtocolQA Open-Ended In use Models correct errors in published lab protocols using open-ended questions adapted from LAB-Bench ProtocolQA. Scored as pass@1. GPT-6 Cyber 47.2%
- MolecularIQ In use Models answer questions about molecular graphs, such as counting, indexing and constrained generation, and a symbolic checker verifies each answer. gpt-oss-120b 47.5%
Data audit
All 7 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access The tasks are private.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Note Independent runs Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (34 scores).
- Pass Models scored 29 models we track.
- Pass Newest score The newest is from November 2026, for GPT-6.2 Astra.
- Pass Frontier models 4 of the top 10 have a score, led by GPT-6.2 Astra (#1).
- Pass Headroom The best is 63.1%, under 85%, so it still separates models.
Leaderboard import 1 note
- Note Importer No public leaderboard to read.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.