o1-mini ranks first on 7 of the 79 benchmarks with five or more models scored.
Closed weights
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Arena-Hard-Auto | 92% | #1 of 38 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 1.40% | #11 of 148 | Official leaderboard | |
| AgentIF | 59.8% | #1 of 12 | Official leaderboard | |
| MMLU-Pro | 80.3% | #56 of 212 | Official leaderboard | |
| JudgeBench | 65.7% | #3 of 11 | Official leaderboard | |
| AraGen | 70.5% | #12 of 42 | Official leaderboard | |
| AidanBench | 1577.0 | #4 of 13 | Official leaderboard | |
| Chinese SimpleQA | 44.1% | #11 of 27 | Official leaderboard | |
| SuperGPQA | 45.2% | #28 of 59 | Official leaderboard | |
| BuzzBench | 46.2 | #10 of 20 | Official leaderboard | |
| BirthdayFacts | 24% | #1 of 2 | Lab model card | |
| EQ-Bench v2 | 72.0 | #48 of 85 | Official leaderboard | |
| LMArena (Elo) | 1,337 | #191 of 335 | Official leaderboard | |
| LLM Confabulation Leaderboard | 18.6% | #35 of 58 | Official leaderboard | |
| LongBench v2 | 38.9% | #24 of 33 | Official leaderboard | |
| SimpleQA Hallucination Rate | 60% | #9 of 12 | Lab model card | |
| HUMAINE | 21 | #40 of 50 | Official leaderboard | |
| ECLeKTic | 8.69% | #28 of 35 | Official leaderboard | |
| FACTS Grounding | 62.1% | #30 of 34 | Official leaderboard | |
| SimpleQA | 7.00% | #53 of 60 | Lab model card | |
| SimpleVQA | 4.10% | #12 of 13 | Official leaderboard | |
| Open Ended Questions (hallucination) | 0.93 | #2 of 2 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| KOR-Bench | 70.2% | #2 of 33 | Official leaderboard | |
| PHYSICS | 43.6% | #2 of 20 | Official leaderboard | |
| ZebraLogic | 59.7% | #5 of 46 | Official leaderboard | |
| Omni-MATH | 60.5% | #1 of 9 | Official leaderboard | |
| U-MATH | 76.3% | #5 of 30 | Official leaderboard | |
| MATH Level 5 | 89.2% | #18 of 94 | Epoch AI run | |
| GSM8K Platinum Initial 268 | 99.3% | #5 of 19 | Official leaderboard | |
| PolyMath | 36.6% | #15 of 38 | Official leaderboard | |
| GPQA Diamond | 62.4% | #134 of 229 | Epoch AI run | |
| OTIS Mock AIME 2024-2025 | 46.9% | #137 of 212 | Epoch AI run | |
| QuantBench | 48% | #2 of 3 | Lab model card | |
| ARC-AGI-2 | 0.83% | #85 of 98 | Official leaderboard | |
| ARC-AGI | 14% | #84 of 95 | Official leaderboard | |
| FrontierMath | 1.72% | #49 of 55 | Official leaderboard | |
| SimpleBench | 18.1% | #102 of 106 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval | 96.3% | #1 of 46 | Official leaderboard | |
| HumanEval+ | 89% | #1 of 46 | Official leaderboard | |
| SEAL Coding | 1,237 | #1 of 26 | Official leaderboard | |
| MBPP | 93.1% | #2 of 35 | Official leaderboard | |
| MBPP+ | 78.8% | #2 of 35 | Official leaderboard | |
| MHPP | 66.2% | #2 of 23 | Official leaderboard | |
| FullStackBench | 64.7% | #2 of 10 | Official leaderboard | |
| BigCodeBench-Hard | 27.4% | #29 of 96 | Official leaderboard | |
| BigO(Bench) | 35.6% | #8 of 18 | Official leaderboard | |
| BIRD-CRITIC | 25% | #4 of 9 | Official leaderboard | |
| StructEval | 75.6% | #13 of 28 | Official leaderboard | |
| Aider-Edit | 61.1% | #21 of 34 | Official leaderboard | |
| Aider Polyglot | 32.9% | #29 of 46 | Official leaderboard | |
| OpenAI Research Engineer Interview: Multiple Choice | 77.3% | #7 of 9 | Lab model card | |
| WeirdML | 36.3% | #92 of 112 | Official leaderboard | |
| SciCode | 1.50% | #15 of 18 | Official leaderboard | |
| SWE-bench Verified | 35.8% | #149 of 155 | Lab model card | |
| cognition-golden | 34.6% | #2 of 2 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Step Race | 2 | #28 of 59 | Official leaderboard | |
| LLM Chess | -52 | #70 of 123 | Official leaderboard | |
| LisanBench | 295.3 | #81 of 133 | Official leaderboard | |
| lmgame-Bench | 12.5 | #20 of 23 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Situational Awareness Dataset (SAD) | 55.4% | #3 of 17 | Official leaderboard | |
| Jailbreak Augmented Examples | 100% | #1 of 5 | Lab model card | |
| Challenging Refusal Evaluation | 93.2% | #2 of 9 | Lab model card | |
| HELM Safety | 95.5 | #20 of 75 | Official leaderboard | |
| SORRY-Bench | 24.3% | #7 of 23 | Official leaderboard | |
| MakeMeSay | 38.9% | #5 of 10 | Lab model card | |
| Production Jailbreaks | 99% | #3 of 5 | Lab model card | |
| Standard Refusal Evaluation | 99% | #5 of 8 | Lab model card | |
| BrowserART | 24% | #4 of 6 | Official leaderboard | |
| Anthropic Red Team | 98.3% | #51 of 75 | Official leaderboard | |
| Cybench | 10% | #11 of 16 | Official leaderboard | |
| SimpleSafetyTests | 97% | #55 of 75 | Official leaderboard | |
| StrongREJECT | 83% | #14 of 19 | Lab model card | |
| Political Persuasion Parallel Generation | 46.8% | #3 of 4 | Lab model card | |
| Tacit Knowledge and Troubleshooting | 66.7% | #23 of 30 | Lab model card | |
| XSTest | 94.8% | #13 of 16 | Lab model card | |
| AIR-Bench 2024 | 45.3% | #69 of 82 | Official leaderboard | |
| Capture the Flag (Professional) | 3.90% | #18 of 21 | Lab model card | |
| Human Sourced Jailbreaks | 95% | #7 of 8 | Lab model card | |
| Capture the Flag (High School) | 28.7% | #8 of 9 | Lab model card | |
| LAB-Bench CloningScenarios | 39.4% | #11 of 12 | Lab model card | |
| Capture the Flag (Collegiate) | 0% | #12 of 13 | Lab model card | |
| LAB-Bench ProtocolQA | 67.6% | #15 of 16 | Lab model card | |
| MakeMePay | 0.70% | #6 of 6 | Lab model card | |
| ChangeMyView | 69.5% | #5 of 5 | Lab model card | |
| WildChat (toxic conversations) | 95.7% | #4 of 4 | Lab model card | |
| External red team jailbreaks | 15% | #2 of 2 | Lab model card |
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)
Every result in the longer documents the lab published with it. 37 of them aren't in the launch post.