GPT-4o Nov 2024
GPT-4o ranks first on 3 of the 84 benchmarks with five or more models scored.
87
Benchmarks with a score
3
Ranked first, out of five or more models
11
In the top three
OpenAI Nov 2024
GPT-4o
Nov 2024 snapshot Closed weights
128.8
#160 of 256
Epoch Capabilities Index
- Humanity's Last Exam 2.72% #47 /47
- SWE-bench Verified 31% #32 /32
- GPQA Diamond 47.9% #163 /229
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMLU | 88.1% | #1 of 85 | Claim in Phi-4 Technical Report, via Epoch AI | |
| MMLU-CF | 73.4% | #1 of 37 | Official leaderboard | |
| Creation-MMBench | 0.00 | #2 of 19 | Official leaderboard | |
| MMBench-CN v1.1 | 83.9% | #21 of 178 | Official leaderboard | |
| ToRR | 49.8 | #2 of 14 | Official leaderboard | |
| AgentIF | 58.5% | #2 of 12 | Official leaderboard | |
| LongBench v2 | 51.4% | #10 of 33 | Official leaderboard | |
| MMLU-Pro | 77.9% | #67 of 212 | Official leaderboard | |
| FaithJudge | 15.8% | #13 of 37 | Official leaderboard | |
| SimpleQA | 38.8% | #8 of 16 | Official leaderboard | |
| SuperGPQA | 44.4% | #31 of 59 | Official leaderboard | |
| LLM Confabulation Leaderboard | 17.2% | #31 of 58 | Official leaderboard | |
| SEAL Precise Instruction Following | 82.5 | #10 of 18 | Official leaderboard | |
| AidanBench | 1269.0 | #8 of 13 | Official leaderboard | |
| IFEval | 81.7% | #37 of 60 | Official leaderboard | |
| MARS-Bench | 35.8% | #10 of 15 | Official leaderboard | |
| Internal API instruction following (OpenAI) | 29.2% | #7 of 8 | Official leaderboard | |
| Humanity's Last Exam | 2.72% | #47 of 47 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| miniCTX-v2 | 27.8% | #2 of 4 | Official leaderboard | |
| GPQA Main | 52% | #33 of 60 | Official leaderboard | |
| MATH Level 5 | 49.8% | #53 of 94 | Epoch AI run | |
| Misguided Attention | 33.7% | #32 of 45 | Official leaderboard | |
| GPQA Diamond | 47.9% | #163 of 229 | Epoch AI run | |
| GSM8K Platinum Initial 268 | 98.1% | #15 of 19 | Official leaderboard | |
| ForecastBench | 0.15 | #68 of 82 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 6.25% | #178 of 212 | Epoch AI run | |
| GSM8K-Platinum | 98.6% | #7 of 8 | Official leaderboard | |
| ARC-AGI-2 | 0% | #91 of 98 | Official leaderboard | |
| ARC-AGI | 4.50% | #91 of 95 | Official leaderboard | |
| PolyMath | 13.7% | #37 of 38 | Official leaderboard | |
| FrontierMath | 0.34% | #55 of 55 | Official leaderboard | |
| AMO-Bench | 1.50% | #21 of 21 | Official leaderboard | |
| LiveMathematicianBench | 19.5% | #17 of 17 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench | 53.5% | #5 of 69 | Official leaderboard | |
| SEAL Coding | 1,132 | #3 of 26 | Official leaderboard | |
| BigCodeBench-Hard | 31.1% | #15 of 96 | Official leaderboard | |
| Aider-Edit | 71.4% | #11 of 34 | Official leaderboard | |
| BIRD-CRITIC | 27.5% | #3 of 9 | Official leaderboard | |
| MMTU | 50.7% | #5 of 9 | Official leaderboard | |
| AutoCodeBench | 41.1% | #21 of 34 | Official leaderboard | |
| Aider Polyglot | 18.2% | #36 of 46 | Official leaderboard | |
| Multi-SWE-bench | 3.80% | #9 of 11 | Official leaderboard | |
| Spider 2.0 | 13% | #20 of 23 | Official leaderboard | |
| WeirdML | 25.1% | #100 of 112 | Official leaderboard | |
| SWE-bench Verified | 31% | #32 of 32 | Epoch AI run |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Berkeley Function Calling Leaderboard (BFCL) | 71.7% | #4 of 67 | Official leaderboard | |
| AceBench | 85.4% | #1 of 16 | Official leaderboard | |
| CRMArena | 54.4% | #2 of 9 | Official leaderboard | |
| Galileo Agent Leaderboard | 0.90 | #9 of 29 | Official leaderboard | |
| LLM Chess | -108 | #77 of 123 | Official leaderboard | |
| GAIA | 33.2% | #41 of 64 | Official leaderboard | |
| AgentBench FC | 39.6% | #9 of 12 | Official leaderboard | |
| lmgame-Bench | 16.9 | #18 of 23 | Official leaderboard | |
| WARC-Bench | 9.17% | #7 of 8 | Official leaderboard | |
| AndroidLab | 31.2% | #1 of 1 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MortgageTax | 57.4% | #70 of 87 | Official leaderboard | |
| Cohere enterprise human evaluation | 33% | #2 of 2 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench-EN v1.1 | 84.8% | #19 of 178 | Official leaderboard | |
| KRETA | 84.6% | #2 of 17 | Official leaderboard | |
| CharXiv | 50.5% | #10 of 77 | Official leaderboard | |
| MMStar | 65.1% | #32 of 173 | Official leaderboard | |
| SpaCE-10 | 49% | #7 of 36 | Official leaderboard | |
| MMMU | 70.7% | #30 of 136 | Official leaderboard | |
| LVBench | 48.9% | #2 of 9 | Official leaderboard | |
| VHELM | 62.7 | #13 of 41 | Official leaderboard | |
| MME-CC | 26.9% | #5 of 12 | Official leaderboard | |
| VPCT | 40% | #15 of 27 | VPCT leaderboard, via Epoch AI | |
| MMLongBench-Doc | 46.3% | #10 of 17 | Official leaderboard | |
| VISTA | 38% | #38 of 54 | Official leaderboard | |
| VisFactor | 21.4% | #19 of 26 | Official leaderboard | |
| Drone-Bench | 20.4% | #18 of 19 | Official leaderboard | |
| Math-VR | 4.30% | #25 of 26 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BioTIER Permit | 99.7% | #19 of 69 | Official leaderboard | |
| nocot-bench (NCRI) | 70.3 | #99 of 185 | Official leaderboard | |
| Lakera B3 (Backbone Breaker Benchmark) | 51.1% | #24 of 35 | Official leaderboard | |
| BioTIER Refuse | 22.1% | #53 of 69 | Official leaderboard | |
| Molecular Biology Capabilities Test | 39.9% | #75 of 90 | Official leaderboard | |
| BABE | 23.9% | #5 of 6 | Official leaderboard | |
| Human Pathogen Capabilities Test | 36.2% | #74 of 86 | Official leaderboard | |
| Human Pathogen Capabilities Test v2 (HPCT v2) | 31.6% | #62 of 71 | Official leaderboard | |
| World-Class Bio | 21.2% | #69 of 74 | Official leaderboard | |
| PCT (Polarized Contrastive Pairs) | 59.6% | #43 of 46 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 72.0 | #25 of 173 | Official leaderboard | |
| HELM Capabilities | 63.4 | #36 of 60 | Official leaderboard | |
| CAIS Vision Capabilities Index | 36.0 | #50 of 53 | Official leaderboard | |
| CAIS Risk Index | 74.4 | #66 of 67 | Official leaderboard | |
| CAIS Text Capabilities Index | 7.90 | #67 of 68 | Official leaderboard |
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)