Capabilities
- MedChemBench (OpenAI) 49.6
- GeneBench Pro 37.7
- LifeSciBench 61.3
GPT-Rosalind-6 ranks first on 5 of the 18 benchmarks with five or more models scored.
Closed weights
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Multimodal Troubleshooting Virology | 63.1% | #1 of 27 | Lab model card | |
| TroubleshootingBench | 59% | #1 of 20 | Lab model card | |
| GeneBench Pro | 37.7% | #1 of 5 | Lab model card | |
| LifeSciBench | 61.3% | #1 of 5 | Lab model card | |
| MedChemBench (OpenAI) | 49.6% | #1 of 5 | Lab model card | |
| ProtocolQA Open-Ended | 42% | #6 of 27 | Lab model card | |
| Broken Search Tool | 1.30% | #2 of 9 | Lab model card | |
| CoT-Control | 71.8% | #2 of 9 | Lab model card | |
| Misaligned Behavior in Realistic Work Environments | 1.40% | #1 of 4 | Lab model card | |
| Tacit Knowledge and Troubleshooting | 78.3% | #9 of 30 | Lab model card | |
| SHP2 Protein Function Prediction | 0.38 | #2 of 6 | Lab model card | |
| Unintended Engagement with External Agent Messages | 2.50% | #2 of 6 | Lab model card | |
| Blocker Deception | 9.50% | #2 of 5 | Lab model card | |
| Respecting Warnings | 21.7% | #3 of 7 | Lab model card | |
| AAV Capsid Packaging Prediction (OpenAI) | 0.53 | #4 of 7 | Lab model card | |
| Declining to Exploit a Honeypot During Difficult ExploitGym Problems | 0% | #5 of 7 | Lab model card | |
| Coronavirus-ACE2 Cell-Entry Screen | 0.42 | #5 of 6 | Lab model card | |
| Respecting Auto-Review | 0% | #6 of 7 | Lab model card | |
| Phage-plasmid Co-evolution | 13.4 | #6 of 6 | Lab model card | |
| Math Side-tasking Monitor Evasion | 0% | #2 of 2 | Lab model card | |
| Agent Replay Auto-Review | 27.1% | #1 of 1 | Lab model card | |
| Historical Jailbreak Regression | 14.3% | #1 of 1 | Lab model card | |
| OAI-repo Sabotage v2 | 0% | #1 of 1 | Lab model card |
Every result in the longer documents the lab published with it. 23 of them aren't in the launch post.