GPT-4o Aug 2024
GPT-4o ranks first on 8 of the 114 benchmarks with five or more models scored.
117
Benchmarks with a score
8
Ranked first, out of five or more models
23
In the top three
OpenAI Aug 2024
GPT-4o
Aug 2024 snapshot Closed weights
127.0
#166 of 256
Epoch Capabilities Index
- GPQA Diamond 49.2% #156 /229
- Vals Finance Agent 8.06% #46 /49
- FrontierMath 0.34% #53 /55
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| CulturalBench | 62% | #1 of 15 | Official leaderboard | |
| Chinese SimpleQA | 59.7% | #2 of 27 | Official leaderboard | |
| MM-IFEval | 64.6% | #1 of 9 | Official leaderboard | |
| QuALITY | 83.1% | #1 of 4 | Official leaderboard | |
| RewardBench | 86.7% | #14 of 48 | Official leaderboard | |
| MMBench-CN v1.1 | 79.2% | #55 of 178 | Official leaderboard | |
| Arena-Hard-Auto | 77.9% | #12 of 38 | Official leaderboard | |
| LongBench v2 | 51.2% | #11 of 33 | Official leaderboard | |
| MMLU-Pro | 74.7% | #74 of 212 | Official leaderboard | |
| SimpleQA | 40.1% | #6 of 16 | Official leaderboard | |
| LLM Confabulation Leaderboard | 15.3% | #22 of 58 | Official leaderboard | |
| SEAL Multilingual | 1,067 | #8 of 19 | Official leaderboard | |
| Vectara Hallucination Leaderboard (2025 dataset) | 9.60% | #55 of 104 | Official leaderboard | |
| AidanBench | 1333.0 | #7 of 13 | Official leaderboard | |
| LMArena (Elo) | 1,336 | #193 of 335 | Official leaderboard | |
| SuperGPQA | 41.6% | #34 of 59 | Official leaderboard | |
| RewardBench 2 | 64.9% | #25 of 42 | Official leaderboard | |
| BullshitBench | 16.4% | #76 of 107 | Official leaderboard | |
| SEAL Precise Instruction Following | 80.2 | #14 of 18 | Official leaderboard | |
| SimpleQA Verified | 26% | #75 of 94 | Epoch AI run | |
| BullshitBench V2 | 12% | #109 of 129 | Official leaderboard | |
| MultiNRC | 12.4% | #35 of 38 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM1k | 95.7% | #2 of 15 | Official leaderboard | |
| ZebraLogic | 31.7% | #10 of 46 | Official leaderboard | |
| GSM8K Platinum Initial 268 | 98.9% | #9 of 19 | Official leaderboard | |
| MATH Level 5 | 53.3% | #47 of 94 | Epoch AI run | |
| AutoLogi | 69.7% | #1 of 2 | Official leaderboard | |
| U-MATH | 50.2% | #16 of 30 | Official leaderboard | |
| GPQA Diamond | 49.2% | #156 of 229 | Epoch AI run | |
| TSP-Opt | 10.2% | #9 of 11 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 6.39% | #176 of 212 | Epoch AI run | |
| 3SAT-Search | 12.7% | #10 of 11 | Official leaderboard | |
| FrontierMath | 0.34% | #53 of 55 | Official leaderboard | |
| SimpleBench | 17.8% | #103 of 106 | Official leaderboard | |
| FrontierMath Tiers 1-3 v2 | 0.35% | #86 of 86 | Official leaderboard | |
| AstaBench | 16.2% | #13 of 13 | Official leaderboard | |
| UGMathBench | 62.8% | #7 of 7 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval | 92.7% | #3 of 46 | Official leaderboard | |
| HumanEval+ | 87.2% | #3 of 46 | Official leaderboard | |
| HumanEval-V | 27.7% | #4 of 24 | Official leaderboard | |
| BigCodeBench-Hard | 30.8% | #17 of 96 | Official leaderboard | |
| MBPP | 87.6% | #8 of 35 | Official leaderboard | |
| Aider-Edit | 71.4% | #10 of 34 | Official leaderboard | |
| FullStackBench | 61.8% | #3 of 10 | Official leaderboard | |
| MBPP+ | 72.2% | #12 of 35 | Official leaderboard | |
| SEAL Coding | 1,045 | #10 of 26 | Official leaderboard | |
| GitTaskBench | 37% | #4 of 10 | Official leaderboard | |
| SWE-bench Lite | 32% | #8 of 15 | Official leaderboard | |
| Aider Polyglot | 23.1% | #33 of 46 | Official leaderboard | |
| SWE-bench Verified | 38.8% | #44 of 60 | Official leaderboard | |
| LiveCodeBench v6 | 29.5% | #17 of 21 | Official leaderboard | |
| MLE-bench | 8.63% | #10 of 12 | Official leaderboard | |
| LiveCodeBench | 42.6% | #18 of 21 | Official leaderboard | |
| LiveCodeBench v5 | 29.5% | #18 of 21 | Official leaderboard | |
| Spider 2.0 | 10.8% | #21 of 23 | Official leaderboard | |
| SWE-bench Multimodal | 12.2% | #9 of 9 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ComplexFuncBench | 60.5% | #2 of 11 | Official leaderboard | |
| VideoGameBench Lite | 1.60% | #1 of 5 | Official leaderboard | |
| Orak | 4.30 | #2 of 6 | Official leaderboard | |
| VideoGameBench | 0.09% | #2 of 5 | Official leaderboard | |
| ToolComp | 59.9% | #16 of 32 | Official leaderboard | |
| Factorio Learning Environment | 87599.0 | #3 of 6 | Official leaderboard | |
| GAIA | 32.9% | #42 of 64 | Official leaderboard | |
| AppWorld Test-Challenge | 38.9% | #8 of 12 | Official leaderboard | |
| LLM Chess | -134 | #84 of 123 | Official leaderboard | |
| AppWorld Test-Normal | 68.5% | #9 of 12 | Official leaderboard | |
| Online-Mind2Web | 30.7% | #4 of 5 | Official leaderboard | |
| Step Race | 1 | #50 of 59 | Official leaderboard | |
| Chess Puzzles | 9.00% | #86 of 93 | Epoch AI run | |
| MCP-Universe | 15.6% | #29 of 30 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedQA | 86.3% | #1 of 71 | Official leaderboard | |
| MedXpertQA-MM | 42.8% | #2 of 9 | Official leaderboard | |
| TaxEval v2 | 74.5% | #35 of 129 | Official leaderboard | |
| CaseLaw v2 | 59.7% | #25 of 53 | Official leaderboard | |
| LegalBench | 82.2% | #72 of 138 | Official leaderboard | |
| MortgageTax | 61% | #60 of 87 | Official leaderboard | |
| Vals Finance Agent v1 | 19.3% | #34 of 47 | Official leaderboard | |
| CadEval | 26% | #11 of 13 | Official leaderboard | |
| CorpFin v2 | 45.9% | #103 of 119 | Official leaderboard | |
| HealthBench | 32% | #7 of 8 | Official leaderboard | |
| HealthBench Hard | 0% | #7 of 8 | Official leaderboard | |
| Vals Finance Agent | 8.06% | #46 of 49 | Official leaderboard | |
| PLawBench | 35.8% | #11 of 11 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench-Video | 2.15 | #1 of 28 | Official leaderboard | |
| Image2Struct | 94.7 | #1 of 18 | Official leaderboard | |
| A-Bench | 79.1% | #8 of 116 | Official leaderboard | |
| SGP-Bench | 64.8% | #2 of 22 | Official leaderboard | |
| TOMATO | 37.7% | #2 of 22 | Official leaderboard | |
| DynaMath | 64.9% | #1 of 11 | Official leaderboard | |
| MMVU | 67.4% | #3 of 28 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 66.5% | #28 of 152 | Official leaderboard | |
| MMStar | 64.7% | #35 of 173 | Official leaderboard | |
| MMBench-EN v1.1 | 81.8% | #39 of 178 | Official leaderboard | |
| MMBench | 82.1% | #38 of 155 | Official leaderboard | |
| OmniBench | 51.1% | #4 of 16 | Official leaderboard | |
| Video-MMMU | 61.2% | #7 of 24 | Official leaderboard | |
| VHELM | 59.8 | #15 of 41 | Official leaderboard | |
| VisualPuzzles | 41.3% | #13 of 29 | Official leaderboard | |
| VideoDR (with search) | 43% | #3 of 6 | Official leaderboard | |
| CC-OCR | 66.7% | #2 of 3 | Official leaderboard | |
| VISTA | 34.9% | #40 of 54 | Official leaderboard | |
| Vision Arena | 1,118 | #113 of 136 | Official leaderboard | |
| MADQA | 48.6% | #15 of 18 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| IHEval | 70% | #1 of 10 | Official leaderboard | |
| BioTIER Permit | 99.7% | #18 of 69 | Official leaderboard | |
| nocot-bench (NCRI) | 74.2 | #79 of 185 | Official leaderboard | |
| AIR-Bench 2024 | 62.4% | #48 of 82 | Official leaderboard | |
| FORTRESS | 47.2% | #55 of 73 | Official leaderboard | |
| Molecular Biology Capabilities Test | 39.6% | #77 of 90 | Official leaderboard | |
| Human Pathogen Capabilities Test v2 (HPCT v2) | 30% | #65 of 71 | Official leaderboard | |
| Human Pathogen Capabilities Test | 33.4% | #79 of 86 | Official leaderboard | |
| BioTIER Refuse | 14.5% | #65 of 69 | Official leaderboard | |
| World-Class Bio | 15% | #71 of 74 | Official leaderboard | |
| BrowserART | 74% | #6 of 6 | Official leaderboard | |
| UK AISI and CAISI simulated enterprise network range | 1.70 | #6 of 6 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Lite | 92.8 | #2 of 71 | Official leaderboard | |
| OpenVLM Leaderboard | 71.6 | #27 of 173 | Official leaderboard |
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)