GPT-6.1 Sol
GPT-6.1 Sol ranks first on 21 of the 307 benchmarks with five or more models scored.
GPT-6.1 Sol
Closed weights Mid-size
- MultiChallenge 82% #1 /42
- Terminal-Bench 4.0 58.2% #5 /40
- GPQA Diamond 95.4% #6 /229
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MultiChallenge | 82% | #1 of 42 | Official leaderboard | |
| Extended NYT Connections | 97.8% | #4 of 103 | Official leaderboard | |
| FACTS Parametric | 82.1% | #2 of 45 | Official leaderboard | |
| LMArena (Elo) | 1,482 | #24 of 335 | Official leaderboard | |
| FACTS Parametric V2 | 83.8% | #2 of 27 | Official leaderboard | |
| SimpleQA Verified | 73.8% | #5 of 56 | Official leaderboard | |
| FACTS Benchmark Suite | 76.8% | #3 of 23 | Official leaderboard | |
| FACTS Search V2 | 88.2% | #3 of 21 | Official leaderboard | |
| FACTS Grounding v2 | 78% | #4 of 25 | Official leaderboard | |
| HLE-Diamond | 61.3% | #4 of 20 | Official leaderboard | |
| ComplexConstraints | 70.1% | #3 of 15 | Official leaderboard | |
| BullshitBench V2 | 65% | #30 of 129 | Official leaderboard | |
| EQ-Bench 4 | 1,272 | #8 of 32 | Official leaderboard | |
| FACTS Multimodal | 52.6% | #7 of 23 | Official leaderboard | |
| Joke Arena | 1,519 | #9 of 25 | Official leaderboard | |
| Corp Speak Arena | 1,548 | #8 of 22 | Official leaderboard | |
| StudyArena Text | 1,236 | #40 of 85 | Official leaderboard | |
| HANDBOOK.md | 28.6% | #11 of 16 | Official leaderboard | |
| Global MMLU Lite | 78.9% | #33 of 43 | Official leaderboard | |
| Opening Lines Arena | 1,416 | #13 of 15 | Official leaderboard | |
| ELI5 Arena | 1,344 | #18 of 18 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| FrontierMath Tier 4 v2 | 100% | #1 of 68 | Official leaderboard | |
| ARC-AGI-2 | 94.2% | #2 of 98 | Official leaderboard | |
| GPQA Diamond | 95.4% | #6 of 229 | Epoch AI run | |
| FrontierMath Tiers 1-3 v2 | 93.7% | #3 of 86 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 100% | #11 of 212 | Epoch AI run | |
| ARC-AGI | 98.5% | #5 of 95 | Official leaderboard | |
| FrontierMath Erdős | 4.41% | #1 of 15 | Official leaderboard | |
| SimpleBench | 82.9% | #8 of 106 | Official leaderboard | |
| scBench (LatchBio SingleCellBench) | 63.6% | #2 of 26 | Official leaderboard | |
| DTBench | 96.5% | #15 of 180 | Official leaderboard | |
| BioMysteryBench | 79.6% | #3 of 30 | Official leaderboard | |
| Science's Last Exam | 44.7 | #1 of 9 | Official leaderboard | |
| ProofBench v1.1 | 99% | #7 of 52 | Official leaderboard | |
| FrontierPhysics | 19.6% | #3 of 22 | Official leaderboard | |
| ArXivMath | 92.8% | #1 of 6 | Official leaderboard | |
| BrokenArXiv | 83.1% | #1 of 6 | Official leaderboard | |
| CritPt | 82% | #2 of 11 | Official leaderboard | |
| CritPt v2 | 82% | #2 of 11 | Official leaderboard | |
| MysteryMechanism | 46.4% | #6 of 31 | Official leaderboard | |
| LatchBio Capabilities v1.0 | 50.6% | #6 of 27 | Official leaderboard | |
| Prophet Arena | 86.0 | #6 of 24 | Official leaderboard | |
| Riemann-bench | 79.2% | #5 of 20 | Official leaderboard | |
| ArXivLean | 35.9% | #4 of 7 | Official leaderboard | |
| Terminal-Bench-Science | 57% | #7 of 12 | Lab model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ALE-Bench | 2,990 | #2 of 126 | Official leaderboard | |
| DesignArena Agentic Overall | 1,400 | #2 of 74 | Official leaderboard | |
| DesignArena Game Dev | 1,407 | #5 of 143 | Official leaderboard | |
| WebDev Arena | 1,759 | #4 of 105 | arena.ai, via Epoch AI | |
| DesignArena Code | 1,361 | #6 of 143 | Official leaderboard | |
| DesignArena UI Component | 1,358 | #6 of 139 | Official leaderboard | |
| Code Arena | 1,743 | #6 of 128 | Official leaderboard | |
| DesignArena 3D Design | 1,428 | #7 of 136 | Official leaderboard | |
| Code Migration | 65.1% | #5 of 78 | Official leaderboard | |
| IOI v2 (Vals) | 96.9% | #3 of 45 | Official leaderboard | |
| Chart Arena | 1,628 | #1 of 15 | Official leaderboard | |
| Landing Page Arena | 1,692 | #1 of 14 | Official leaderboard | |
| DesignArena Website | 1,315 | #12 of 151 | Official leaderboard | |
| Vibe Code Bench | 88.9% | #10 of 120 | Official leaderboard | |
| Image-to-WebDev Arena | 1,696 | #5 of 55 | Official leaderboard | |
| DAXBench v2 | 91.4% | #5 of 47 | Official leaderboard | |
| WeirdML v3 | 38.3% | #3 of 27 | Official leaderboard | |
| ViBench 2.0 | 64.7% | #1 of 9 | Official leaderboard | |
| GameDevBench | 70% | #2 of 17 | Official leaderboard | |
| DeepSWE | 75.2% | #7 of 56 | Lab model card | |
| OpenDesign Arena | 90.1 | #3 of 21 | Official leaderboard | |
| Code Migration v1.1 | 77.3% | #5 of 30 | Official leaderboard | |
| Android Bench 2.0 | 29.3% | #3 of 17 | Official leaderboard | |
| DesignArena React Native | 1,230 | #10 of 56 | Official leaderboard | |
| SlopCodeBench | 29.1% | #4 of 22 | Official leaderboard | |
| FrontierCode 2.0 | 39.6% | #6 of 32 | Official leaderboard | |
| Snorkel Agentic Coding 2.0 | 42.8% | #3 of 16 | Official leaderboard | |
| Brokk Power Ranking | 19.9 | #4 of 19 | Official leaderboard | |
| sudo L7 | 33.9% | #4 of 18 | Official leaderboard | |
| DesignArena Frontend Web App | 1,263 | #12 of 53 | Official leaderboard | |
| SWE Atlas Codebase QnA Extended | 31.3% | #9 of 38 | Official leaderboard | |
| MacroscopeBench | 78.5% | #4 of 16 | Official leaderboard | |
| RSI Bench | 37.0 | #2 of 8 | Official leaderboard | |
| KernelGen 1P | 60.4% | #1 of 4 | Lab model card | |
| DesignArena Image to HTML | 1,231 | #13 of 51 | Official leaderboard | |
| DesignArena Data Visualization | 1,273 | #31 of 118 | Official leaderboard | |
| Real-SWE 1.1 | 47.2% | #4 of 15 | Official leaderboard | |
| MirrorCode | 61.1% | #11 of 39 | Epoch AI run | |
| SWE Atlas v1.1 | 65.9% | #8 of 28 | Official leaderboard | |
| OWE-Bench | 41.2% | #4 of 14 | Official leaderboard | |
| DesignArena Android (Kotlin) | 1,200 | #15 of 46 | Official leaderboard | |
| Vals RSI Index | 0.27 | #9 of 27 | Official leaderboard | |
| DesignArena Blender | 1,367 | #4 of 12 | Official leaderboard | |
| FDE-Gym | 0.37 | #4 of 12 | Official leaderboard | |
| APEX-SWE | 46.9% | #22 of 65 | Official leaderboard | |
| FrontierCS 2 | 37.5% | #7 of 19 | Official leaderboard | |
| PostTrainBench 1.2 | 38.8% | #8 of 21 | Official leaderboard | |
| DAYJOB: Data Science | 18.7% | #6 of 15 | Official leaderboard | |
| DesignArena Image to Frontend | 1,216 | #13 of 31 | Official leaderboard | |
| SWE-bench CLI | 9.80% | #7 of 16 | Official leaderboard | |
| ScarfBench | 68.4% | #5 of 11 | Official leaderboard | |
| SWE-Bench Pro V2 | 96.1% | #9 of 18 | Official leaderboard | |
| Web Clone Bench | 12.6% | #5 of 10 | Official leaderboard | |
| Internal Research Debugging Evaluation | 75.5% | #3 of 6 | Lab model card | |
| SWE-bench Verified Extended | 45.3% | #28 of 44 | Official leaderboard | |
| Floorplan Arena | 1,473 | #7 of 11 | Official leaderboard | |
| FrontierMemory | 51.1% | #5 of 7 | Official leaderboard | |
| SWE-Marathon v1.1 | 42.5% | #15 of 20 | Official leaderboard | |
| SWE-Together | 58.3% | #17 of 22 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LLM Chess | 1,612 | #2 of 123 | Official leaderboard | |
| Mystery Game Puzzles | 80% | #3 of 84 | Epoch AI run | |
| RuneBench | 6.88 | #4 of 76 | Official leaderboard | |
| Chess Puzzles | 68% | #5 of 93 | Epoch AI run | |
| LMArena Search Arena | 1,248 | #2 of 37 | Official leaderboard | |
| BU Bench V2.1 | 81.3 | #1 of 16 | Official leaderboard | |
| BrowseComp Extended | 67.7% | #3 of 42 | Official leaderboard | |
| GAIA | 86.4% | #5 of 64 | Official leaderboard | |
| DuelLab GameBench 2 | 92.0 | #5 of 57 | Official leaderboard | |
| Terminal-Bench 2.1 Extended | 45.8% | #4 of 44 | Official leaderboard | |
| Terminal-Bench 4.1 | 60.9% | #4 of 43 | Official leaderboard | |
| Game Arena Dots and Boxes | 1,120 | #4 of 40 | Official leaderboard | |
| Game Arena Clobber | 653 | #4 of 39 | Official leaderboard | |
| CivBench | 1,602 | #2 of 19 | Official leaderboard | |
| Kaggle Game Arena | 444 | #5 of 42 | Official leaderboard | |
| LisanBench | 8736.0 | #16 of 133 | Official leaderboard | |
| Game Arena Checkers | 1,206 | #5 of 41 | Official leaderboard | |
| Agent Arena | 12.5% | #7 of 56 | Official leaderboard | |
| Terminal-Bench 4.0 | 58.2% | #5 of 40 | Official leaderboard | |
| Game Arena Lines of Action | 866 | #5 of 39 | Official leaderboard | |
| Game Arena Word Art | 2,076 | #5 of 39 | Official leaderboard | |
| Game Arena Word Association | 441 | #6 of 40 | Official leaderboard | |
| OSWorld 2.0 | 71.4% | #3 of 19 | Lab model card | |
| Collaborative Intelligence Benchmark | 78.7% | #3 of 18 | Official leaderboard | |
| Hermes Bench | 73.9% | #3 of 17 | Official leaderboard | |
| Game Arena Bargaining | 239 | #7 of 39 | Official leaderboard | |
| Game Arena Ultimate Tic-Tac-Toe | 495 | #7 of 39 | Official leaderboard | |
| o11y-bench | 87.5% | #4 of 22 | Official leaderboard | |
| Clocktower Bench | 1,734 | #2 of 11 | Official leaderboard | |
| ARC-AGI-3 | 96.4% | #5 of 27 | Official leaderboard | |
| Game Arena Go | 1,448 | #8 of 39 | Official leaderboard | |
| Game Arena Reversi | 796 | #8 of 39 | Official leaderboard | |
| Game Arena Four in a Row | 584 | #9 of 43 | Official leaderboard | |
| SpireClimb | 23.8% | #2 of 9 | Epoch AI run | |
| StarSkirmish Bench | 45.8% | #2 of 9 | Official leaderboard | |
| Game Arena Chess | 1,233 | #10 of 42 | Official leaderboard | |
| EBR-bench | 54.3% | #8 of 31 | Epoch AI run | |
| Game Arena Coin Game | 419 | #11 of 39 | Official leaderboard | |
| Game Arena Nine Men's Morris | 780 | #13 of 39 | Official leaderboard | |
| SpireClimb StS1 A20 Heart | 18.3% | #3 of 9 | Official leaderboard | |
| SpireClimb StS2 A10 | 29.3% | #3 of 9 | Official leaderboard | |
| Game Arena Dark Hex | 271 | #19 of 41 | Official leaderboard | |
| EnterpriseRAG-Bench | 91.5% | #1 of 2 | Official leaderboard | |
| AutomationBench | 36.1% | #27 of 47 | Lab model card |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| DesignArena HTML Slides | 1,489 | #2 of 28 | Official leaderboard | |
| Vending-Bench 2 | $12,861 | #5 of 67 | Official leaderboard | |
| DesignArena Python-PPTX Slides | 1,345 | #3 of 36 | Official leaderboard | |
| Clipping Bench | $5,707 | #2 of 19 | Official leaderboard | |
| LifeSci Diamond | 14.5% | #2 of 18 | Official leaderboard | |
| Short Form Bench | $2,949 | #1 of 9 | Official leaderboard | |
| Tuesday Work Index | 70.6 | #5 of 44 | Official leaderboard | |
| SnorkelManufacturing | 19.6% | #2 of 16 | Official leaderboard | |
| DAYJOB: Finance | 33.5% | #5 of 39 | Official leaderboard | |
| HW-Bench | 57% | #2 of 15 | Official leaderboard | |
| TaxCalcBench v2 | 74% | #5 of 32 | Official leaderboard | |
| DAYJOB: Healthcare | 25.2% | #6 of 37 | Official leaderboard | |
| MedXpertQA MM Extended | 64.7% | #7 of 41 | Official leaderboard | |
| Bedside Bench | 93.9% | #3 of 17 | Official leaderboard | |
| APEX-1 | 69.3% | #13 of 71 | Official leaderboard | |
| Vals Index v2.1 | 61.1% | #9 of 49 | Official leaderboard | |
| Excel Modeling Benchmark | 70.8% | #14 of 76 | Official leaderboard | |
| DAYJOB: Law | 31.8% | #3 of 16 | Official leaderboard | |
| PolicyBench | 90.2% | #10 of 50 | Official leaderboard | |
| Hebbia Financial Services Benchmark | 73.5% | #2 of 10 | Official leaderboard | |
| DeepFinance | 54.3% | #5 of 23 | Official leaderboard | |
| Holdings-IndividualTaxBench | 74.2% | #3 of 13 | Official leaderboard | |
| HealthBench Consensus | 96% | #4 of 17 | Lab model card | |
| MedScribe | 86.5% | #23 of 97 | Official leaderboard | |
| APEX-Accounting | 64.3% | #12 of 49 | Official leaderboard | |
| DesignArena Spreadsheet | 1,270 | #9 of 35 | Official leaderboard | |
| RWE-bench | 25% | #4 of 15 | Official leaderboard | |
| Vals Index v2.2 | 63.9% | #10 of 37 | Official leaderboard | |
| MedCode | 48.8% | #28 of 96 | Official leaderboard | |
| EEBench | 63.6% | #9 of 30 | Official leaderboard | |
| APEX-Agents 1.1 | 73.2% | #20 of 65 | Official leaderboard | |
| APEX-Agents | 60% | #19 of 61 | Collected by Epoch AI, source not named | |
| HealthBench Professional | 64.2% | #9 of 26 | Lab model card | |
| DAYJOB: Accounting | 25.2% | #6 of 17 | Official leaderboard | |
| SnorkelUnderwrite 2.0 | 25% | #6 of 16 | Official leaderboard | |
| SheetBench | 60.5% | #3 of 8 | Official leaderboard | |
| GDP.xlsx | 22% | #8 of 20 | Official leaderboard | |
| GDPx | 8.43% | #8 of 20 | Official leaderboard | |
| DAYJOB: Tax | 4.00% | #6 of 15 | Official leaderboard | |
| Excel Modeling Benchmark v1.1 | 71.1% | #17 of 42 | Official leaderboard | |
| PRBench | 47.9% | #20 of 48 | Official leaderboard | |
| HealthBench Hard | 36.2% | #9 of 21 | Lab model card | |
| WorkplaceAgents | 15% | #6 of 14 | Official leaderboard | |
| Digital Work Index | 21.2 | #7 of 16 | Official leaderboard | |
| Immigration Bench | 67.8% | #11 of 25 | Official leaderboard | |
| Harvey LAB (Legal Agent Benchmark) | 5.42% | #36 of 80 | Official leaderboard | |
| EnterpriseBench: CoreCraft | 50.3% | #23 of 51 | Official leaderboard | |
| Legal Research Bench | 38.5% | #42 of 93 | Official leaderboard | |
| Vals Finance Agent v2 | 52% | #37 of 80 | Official leaderboard | |
| Vals Finance Agent v2.1 | 61.8% | #19 of 40 | Official leaderboard | |
| HealthBench | 58.5% | #15 of 31 | Lab model card | |
| SnorkelRevOps | 12.1% | #9 of 16 | Official leaderboard | |
| Legora BAR | 102.0 | #10 of 17 | Official leaderboard | |
| Tax Agent Bench | 21.8% | #52 of 84 | Official leaderboard | |
| Vals Legal Index | 68.7% | #24 of 38 | Official leaderboard | |
| YC-Bench | $936,267 | #39 of 59 | Official leaderboard | |
| Company Agent Bench | 79.1% | #4 of 6 | Official leaderboard | |
| Public Benefits Bench | 59.3% | #35 of 50 | Official leaderboard | |
| Vals Finance Index | 55.8% | #28 of 40 | Official leaderboard | |
| RCM-Bench-1 | 87.4% | #14 of 20 | Official leaderboard | |
| DesignArena PDF | 1,139 | #28 of 37 | Official leaderboard | |
| Harvey LAB v1.1 (HLAB-Vals-1.1) | 6.25% | #29 of 38 | Official leaderboard | |
| MissionReport | 61% | #8 of 10 | Official leaderboard | |
| SnorkelFinance 2.0 | 6.60% | #14 of 16 | Official leaderboard | |
| SnorkelLegal | 20.7% | #16 of 16 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| vCX Benchmark | 93.3% | #1 of 42 | Official leaderboard | |
| MMMU-Pro Extended | 61.3% | #1 of 38 | Official leaderboard | |
| ExtractBench | 95.3% | #1 of 27 | Official leaderboard | |
| VoxelBench | 2,532 | #3 of 56 | Official leaderboard | |
| BVB | 75.4 | #3 of 45 | Official leaderboard | |
| Vision Arena | 1,293 | #10 of 136 | Official leaderboard | |
| VoxelBench Image | 2,264 | #1 of 13 | Official leaderboard | |
| Frontier3DBench | 66.0 | #1 of 11 | Official leaderboard | |
| ClockBench | 74.4% | #4 of 37 | Official leaderboard | |
| Furniture Assembly Benchmark | 80% | #4 of 35 | Official leaderboard | |
| CharXiv Extended | 85.1% | #5 of 38 | Official leaderboard | |
| Robo-bench | 28.3% | #1 of 7 | Official leaderboard | |
| Humanity's Sixth Sense | 46.6% | #5 of 34 | Official leaderboard | |
| Blueprint-Bench 2 | 43.2% | #6 of 38 | Official leaderboard | |
| GDP.pdf | 51.6% | #3 of 19 | Official leaderboard | |
| ScienceVision | 0.49 | #2 of 12 | Official leaderboard | |
| DataCenterBench Tier 1 | 38.3% | #2 of 9 | Official leaderboard | |
| SplatBench | 38.6 | #2 of 8 | Official leaderboard | |
| Code2Video-Compass | 4.77 | #4 of 15 | Official leaderboard | |
| ParseBench | 69.6% | #12 of 44 | Official leaderboard | |
| RoboColiseum | 70.9% | #3 of 9 | Official leaderboard | |
| RoboDojo-RC Tier 1 | 41.7% | #3 of 9 | Official leaderboard | |
| Dual-Arm Sim | 40.8 | #4 of 11 | Official leaderboard | |
| DesignArena Code to Video | 1,192 | #11 of 29 | Official leaderboard | |
| OmniParseBench | 91.5 | #4 of 9 | Official leaderboard | |
| Code2Video Arena | 1,511 | #7 of 13 | Official leaderboard | |
| ShadowBench | 44.7 | #4 of 6 | Official leaderboard | |
| ExtractBench Pro | 12% | #6 of 7 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| nocot-bench (NCRI) | 149.0 | #3 of 185 | Official leaderboard | |
| SRE-Bench | 50.8% | #2 of 35 | Official leaderboard | |
| SAFE-Teen Health advice | 9.00% | #1 of 17 | Official leaderboard | |
| SAFE-Teen Relationship boundaries | 19% | #1 of 17 | Official leaderboard | |
| SAFE-Teen Urgent safety | 15% | #1 of 17 | Official leaderboard | |
| Tacit Knowledge and Troubleshooting | 88.5% | #2 of 30 | Lab model card | |
| DeepsecBench | 39.5 | #2 of 29 | Official leaderboard | |
| WellnessBench | 66% | #1 of 12 | Official leaderboard | |
| SpatialBench-Long | 39.9% | #3 of 27 | Official leaderboard | |
| ExploitGym | 35.1% | #2 of 18 | Official leaderboard | |
| ExploitBench | 99.7% | #3 of 21 | Lab model card | |
| VIALS 2 | 49.1% | #2 of 14 | Official leaderboard | |
| Biopharma Bench | 62.1% | #2 of 13 | Official leaderboard | |
| TxBench-Oligo-Discovery | 52.5% | #5 of 30 | Official leaderboard | |
| Suicide Memory Bench | 1.21 | #2 of 12 | Official leaderboard | |
| BioSecBench-Function | 46% | #5 of 27 | Official leaderboard | |
| SecIT Bench | 79.9% | #5 of 26 | Official leaderboard | |
| PCT (Polarized Contrastive Pairs) | 37.3% | #10 of 46 | Official leaderboard | |
| BioSecBench-Surveillance | 50.8% | #6 of 27 | Official leaderboard | |
| VariantBench | 46% | #6 of 27 | Official leaderboard | |
| OncologyBench | 24% | #7 of 30 | Official leaderboard | |
| TroubleshootingBench | 48% | #5 of 20 | Lab model card | |
| MentalHealthBench | 57.9% | #1 of 4 | Lab model card | |
| EpiBench | 28.3% | #7 of 27 | Official leaderboard | |
| Instruction hierarchy attacks | 99.99% | #2 of 7 | Lab model card | |
| ProtocolQA Open-Ended | 40.7% | #8 of 27 | Lab model card | |
| scBench-Long | 26.7% | #8 of 27 | Official leaderboard | |
| MetagenomicsBench | 45.7% | #8 of 26 | Official leaderboard | |
| BioTIER Refuse | 77.8% | #23 of 69 | Official leaderboard | |
| DistressBench | 74.2% | #14 of 42 | Official leaderboard | |
| Multimodal Troubleshooting Virology | 55.3% | #9 of 27 | Lab model card | |
| AI Wellbeing Index percent positive experiences | 75.5% | #2 of 6 | Official leaderboard | |
| Coding Deception | 1.50% | #2 of 6 | Lab model card | |
| TxBench-Antibody-Discovery | 49.7% | #11 of 30 | Official leaderboard | |
| TxBench-Preclinical-Pharmacology | 59.7% | #11 of 30 | Official leaderboard | |
| BioSecBench-Refusal | 37.6% | #9 of 22 | Official leaderboard | |
| BioSimBench | 0.15 | #5 of 12 | Official leaderboard | |
| Prompt injection robustness | 99% | #3 of 7 | Lab model card | |
| Respecting Auto-Review | 0% | #3 of 7 | Lab model card | |
| Broken Search Tool | 2.08% | #4 of 9 | Lab model card | |
| CoT-Control | 44.8% | #4 of 9 | Lab model card | |
| SEC-bench Pro | 78.8% | #5 of 11 | Lab model card | |
| Coronavirus-ACE2 Cell-Entry Screen | 0.42 | #3 of 6 | Lab model card | |
| Misaligned Behavior in Realistic Work Environments | 1.94% | #2 of 4 | Lab model card | |
| Declining to Exploit a Honeypot During Difficult ExploitGym Problems | 0% | #4 of 7 | Lab model card | |
| Respecting Warnings | 23.5% | #4 of 7 | Lab model card | |
| CheatBench | 49.3% | #6 of 10 | Official leaderboard | |
| ExploitBench - Internal Port | 21.5% | #3 of 5 | Lab model card | |
| OpenAI Deployment Simulation | 0.06% | #3 of 5 | Lab model card | |
| IPI Bench | 18.4% | #4 of 6 | Official leaderboard | |
| Phage-plasmid Co-evolution | 12.9 | #4 of 6 | Lab model card | |
| SHP2 Protein Function Prediction | 0.33 | #4 of 6 | Lab model card | |
| BioTIER Permit | 98.8% | #49 of 69 | Official leaderboard | |
| Unintended Engagement with External Agent Messages | 38% | #5 of 6 | Lab model card | |
| AAV Capsid Packaging Prediction (OpenAI) | 0.53 | #6 of 7 | Lab model card | |
| CyberBench v1.1 (Vals) | 39.3% | #42 of 44 | Official leaderboard | |
| Perceptual Attacks on Embodied AI (Robocurve) | 46.7% | #5 of 5 | Official leaderboard | |
| Honeypot exploitation monitorability | 100% | #3 of 3 | Lab model card |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Conceptual Reasoning Index | 72.7 | #7 of 151 | Official leaderboard | |
| LMCA | 62.8% | #8 of 147 | Official leaderboard | |
| CAIS Vision Capabilities Index | 77.9 | #3 of 53 | Official leaderboard | |
| CAIS Text Capabilities Index | 61.4 | #4 of 68 | Official leaderboard | |
| ACCoRD | 80.9 | #10 of 152 | Official leaderboard | |
| SAGE v1.1 (Vals) | 90.5% | #2 of 29 | Official leaderboard | |
| Hermes Index | 57.1 | #2 of 17 | Official leaderboard | |
| Vals Science Index | 59.6% | #3 of 25 | Official leaderboard | |
| SuperCLUE | 71.8 | #2 of 16 | Official leaderboard | |
| LiveBench | 81.4% | #6 of 31 | Official leaderboard | |
| CAIS Risk Index | 39.9 | #15 of 67 | Official leaderboard | |
| Political Science LLM Benchmark | 68.3% | #10 of 42 | Official leaderboard | |
| SAGE (Vals) | 46.5% | #35 of 81 | Official leaderboard |
Its lab's numbers put it at 164.0, 4.5 points below its ECI
The Self-Reported Index fits one score to the 9 capability results OpenAI reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 159.9 to 168.0. How we compute it.
- HealthBench Consensus 96
- DeepSWE 75.2
- OSWorld 2.0 71.4
- HealthBench Professional 64.2
- HealthBench 58.5
- Terminal-Bench-Science 57
- HealthBench Hard 36.2
- AutomationBench 36.1
- GDP.pdf 32
Its lab's numbers next to 2 leaderboards
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 43 of them aren't in the launch post.
Health
Hallucination
Safety
Robustness
Alignment
- Respecting Auto-Review 0
- Respecting Warnings 23.5
- Declining to Exploit a Honeypot During Difficult ExploitGym Problems 0
- Coding Deception 1.5
- Broken Search Tool 2.08
- Misaligned Behavior in Realistic Work Environments 1.94
- Unintended Engagement with External Agent Messages 38
- OpenAI Deployment Simulation 0.056
Monitorability
Bio and chem risk
Cyber risk
- ExploitBench 99.7
- ExploitBench - Internal Port 21.5
- SEC-bench Pro 78.8
- ExploitGym 35.1