GPT-4o ranks first on 11 of the 113 benchmarks with five or more models scored.
May 2024 snapshot Closed weights Flagship
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AlpacaEval 2.0 | 57.5% | #1 of 47 | Official leaderboard | |
| WildBench | 58.8 | #2 of 47 | Official leaderboard | |
| RepoQA | 90.6% | #2 of 22 | Official leaderboard | |
| LLM-AggreFact | 75.9% | #1 of 10 | Official leaderboard | |
| MixEval | 87.9% | #4 of 34 | Official leaderboard | |
| MMBench-CN v1.1 | 82.5% | #28 of 178 | Official leaderboard | |
| MMLU | 88.7% | #22 of 131 | Lab model card | |
| MGSM | 90.5% | #11 of 57 | Lab model card | |
| Arena-Hard-Auto | 79.2% | #9 of 38 | Official leaderboard | |
| ThaiExam | 63.3% | #6 of 24 | Official leaderboard | |
| SEAL Multilingual | 1,084 | #5 of 19 | Official leaderboard | |
| SEAL Precise Instruction Following | 85.3 | #6 of 18 | Official leaderboard | |
| MMLU-Pro | 72.5% | #80 of 212 | Official leaderboard | |
| Multilingual human evaluation | 4.09 | #2 of 5 | Official leaderboard | |
| SimpleQA | 39% | #7 of 16 | Official leaderboard | |
| RewardBench | 83.3% | #23 of 48 | Official leaderboard | |
| DROP | 83.4% | #20 of 41 | Lab model card | |
| LMArena (Elo) | 1,346 | #182 of 335 | Official leaderboard | |
| SuperGPQA | 39.8% | #37 of 59 | Official leaderboard | |
| AidanBench | 1052.0 | #9 of 13 | Official leaderboard | |
| JudgeBench | 56.6% | #9 of 11 | Official leaderboard | |
| MMLU College Biology | 95% | #1 of 1 | Lab model card | |
| SuperCLUE-RAG | 66.7 | #1 of 1 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OlympicArena | 40.5% | #1 of 25 | Official leaderboard | |
| MathBench | 70.9% | #1 of 21 | Official leaderboard | |
| KOR-Bench | 58% | #4 of 33 | Official leaderboard | |
| ZebraLogic | 28.2% | #13 of 46 | Official leaderboard | |
| MATH | 76.6% | #34 of 99 | Lab model card | |
| GSM1k | 94.8% | #6 of 15 | Official leaderboard | |
| Omni-MATH | 30.5% | #4 of 9 | Official leaderboard | |
| MATH Level 5 | 51% | #50 of 94 | Epoch AI run | |
| CNMO 2024 | 10.8% | #4 of 7 | Official leaderboard | |
| GPQA Diamond | 48.9% | #161 of 229 | Epoch AI run | |
| DTBench | 64.5% | #127 of 180 | Official leaderboard | |
| ForecastBench | 0.15 | #65 of 82 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 6.25% | #177 of 212 | Epoch AI run |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench | 56.1% | #1 of 69 | Official leaderboard | |
| HumanEval | 90.2% | #12 of 101 | Lab model card | |
| MHPP | 51.4% | #3 of 23 | Official leaderboard | |
| Aider-Edit | 72.9% | #6 of 34 | Official leaderboard | |
| McEval | 65.2% | #1 of 4 | Official leaderboard | |
| BigCodeBench-Hard | 27.1% | #31 of 96 | Official leaderboard | |
| SWE-bench Lite | 38% | #5 of 15 | Official leaderboard | |
| SEAL Coding | 1,036 | #11 of 26 | Official leaderboard | |
| StructEval | 76% | #12 of 28 | Official leaderboard | |
| SWE-bench | 18.8% | #4 of 9 | Official leaderboard | |
| OpenAI Research Engineer Interview: Coding | 95% | #4 of 7 | Lab model card | |
| SciCode | 1.50% | #11 of 18 | Official leaderboard | |
| METR Time Horizon | 10 min | #21 of 31 | Official leaderboard | |
| METR Time Horizon 1.1 | 7.0 min | #19 of 26 | Official leaderboard | |
| SWE-bench Verified | 38.4% | #45 of 60 | Official leaderboard | |
| TasteVal | 5.96× | #19 of 20 | Official leaderboard | |
| OpenAI Research Engineer Interview: Multiple Choice | 61% | #9 of 9 | Lab model card | |
| Agentic tasks (autonomous replication and adaptation) | 0% | #4 of 4 | Lab model card | |
| Select machine learning engineering tasks from METR | 0% | #1 of 1 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GTA | 41.5% | #2 of 15 | Official leaderboard | |
| SuperCLUE-Agent | 78.4 | #1 of 7 | Official leaderboard | |
| VisualAgentBench | 36.2% | #1 of 5 | Official leaderboard | |
| ToolComp | 64.6% | #8 of 32 | Official leaderboard | |
| CRAB | 38% | #1 of 3 | Official leaderboard | |
| WebCanvas | 28.8% | #1 of 3 | Official leaderboard | |
| GroundUI | 13.4% | #4 of 11 | Official leaderboard | |
| BALROG | 32.3% | #13 of 33 | Official leaderboard | |
| Spider2-V | 13.8% | #2 of 5 | Official leaderboard | |
| GAIA | 34.5% | #40 of 64 | Official leaderboard | |
| LLM Chess | -158 | #90 of 123 | Official leaderboard | |
| WebArena | 13.1% | #3 of 4 | Official leaderboard | |
| AppWorld Test-Challenge | 30.2% | #10 of 12 | Official leaderboard | |
| AppWorld Test-Normal | 48.8% | #11 of 12 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedQA | 85.7% | #3 of 71 | Official leaderboard | |
| MedHELM | 49.1 | #9 of 13 | Official leaderboard | |
| MedMCQA | 77% | #1 of 1 | Lab model card | |
| MedQA Mainland China | 84% | #1 of 1 | Lab model card | |
| MedQA Taiwan | 91% | #1 of 1 | Lab model card | |
| MedQA USMLE (5 options) | 86% | #1 of 1 | Lab model card | |
| MMLU Anatomy | 89% | #1 of 1 | Lab model card | |
| MMLU Clinical Knowledge | 92% | #1 of 1 | Lab model card | |
| MMLU College Medicine | 84% | #1 of 1 | Lab model card | |
| MMLU Medical Genetics | 96% | #1 of 1 | Lab model card | |
| MMLU Professional Medicine | 94% | #1 of 1 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LongVideoBench | 66.7% | #1 of 23 | Official leaderboard | |
| A-Bench | 79.2% | #7 of 116 | Official leaderboard | |
| VCR (Visual Caption Restoration) | 51.8% | #3 of 42 | Official leaderboard | |
| MEGA-Bench | 54.2% | #2 of 22 | Official leaderboard | |
| Image2Struct | 80.7 | #2 of 18 | Official leaderboard | |
| ScienceQA | 88.5% | #2 of 17 | Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI | |
| AI2D | 94.2% | #7 of 52 | Lab model card | |
| SGP-Bench | 63.3% | #3 of 22 | Official leaderboard | |
| CharXiv (DQ) | 84.5% | #10 of 67 | Official leaderboard | |
| Video-MME | 77.2% | #4 of 26 | Official leaderboard | |
| CharXiv | 47.1% | #12 of 77 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 71.2% | #25 of 152 | Official leaderboard | |
| MTVQA-Test | 27.8% | #3 of 18 | Official leaderboard | |
| MMBench | 83.4% | #26 of 155 | Official leaderboard | |
| MMBench-EN v1.1 | 83.1% | #31 of 178 | Official leaderboard | |
| LongDocURL | 64.5% | #1 of 5 | Official leaderboard | |
| MMStar | 63.9% | #38 of 173 | Official leaderboard | |
| MMMU | 69.1% | #36 of 136 | Official leaderboard | |
| VHELM | 65.3 | #12 of 41 | Official leaderboard | |
| ActivityNet-QA | 61.9% | #3 of 8 | Lab model card | |
| EgoSchema | 72.2% | #5 of 12 | Lab model card | |
| MMMU-Pro | 51.9% | #24 of 57 | Official leaderboard | |
| ChartQA | 85.7% | #21 of 37 | Lab model card | |
| HR-Bench | 66% | #5 of 8 | Official leaderboard | |
| DocVQA | 92.8% | #23 of 36 | Lab model card | |
| Vision Arena | 1,162 | #93 of 136 | Official leaderboard | |
| MATH-Vision (MathVision) | 30.4% | #73 of 105 | Official leaderboard | |
| MathVista | 63.8% | #57 of 70 | Lab model card | |
| Drone-Bench | 14% | #19 of 19 | Official leaderboard | |
| LVBench | 30.8% | #9 of 9 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OR-Bench | 6.80% | #1 of 28 | Official leaderboard | |
| AgentDojo | 3.66% | #4 of 20 | Official leaderboard | |
| HELM Safety | 94.6 | #25 of 75 | Official leaderboard | |
| Situational Awareness Dataset (SAD) | 50.7% | #6 of 17 | Official leaderboard | |
| Anthropic Red Team | 99.1% | #32 of 75 | Official leaderboard | |
| nocot-bench (NCRI) | 72.2 | #90 of 185 | Official leaderboard | |
| SimpleSafetyTests | 98.5% | #42 of 75 | Official leaderboard | |
| Cybench | 12.5% | #9 of 16 | Official leaderboard | |
| SORRY-Bench | 29.8% | #13 of 23 | Official leaderboard | |
| Tacit Knowledge and Troubleshooting | 69% | #19 of 30 | Lab model card | |
| AIR-Bench 2024 | 52.8% | #62 of 82 | Official leaderboard | |
| Capture the Flag (Collegiate) | 0% | #11 of 13 | Lab model card | |
| Capture the Flag (Professional) | 1.00% | #21 of 21 | Lab model card | |
| Capture the Flag (High School) | 19% | #9 of 9 | Lab model card | |
| SEAL Adversarial Robustness | 67.0 | #8 of 8 | Official leaderboard | |
| Standard Refusal Evaluation | 93% | #8 of 8 | Lab model card | |
| Unauthorized voice generation | 100% | #1 of 1 | Lab model card | |
| Ungrounded inference and sensitive trait attribution | 84% | #1 of 1 | Lab model card | |
| Voice persuasion study on party preferences | 5.74% | #1 of 1 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Lite | 93.8 | #1 of 71 | Official leaderboard | |
| OpenVLM Leaderboard | 70.9 | #31 of 173 | Official leaderboard | |
| Conceptual Reasoning Index | 27.1 | #121 of 151 | Official leaderboard | |
| ACCoRD | 44.8 | #124 of 152 | Official leaderboard |
The Self-Reported Index fits one score to the 14 capability results OpenAI reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 123.4 to 128.8. How we compute it.
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
| Benchmark | Lab says | Board says | Gap | What differs |
|---|---|---|---|---|
| SWE-bench Verified | 19% SWE-Bench Verified (n=477), pass@1, best available post-training and public scaffolds (3.8) | 38.4% Leaderboard, AutoCodeRover | −19 pts | The lab reports pass@1 on 477 tasks, while the AutoCodeRover board entry uses multiple attempts on the full Verified set. |
| GPQA Diamond | 53.6% GPQA (0-shot) | 48.9% Epoch AI run | +4.7 pts | |
| MedQA | 89.4% MedQA USMLE 4 Options (0-shot), 89.4% in text (Table 7 shows 0.89, 5-shot 0.89) | 85.7% Leaderboard | +3.7 pts | |
| MathVista | 63.8% MathVista (%) (testmini) | 63.8% Leaderboard | Same | |
| MMMU | 69.1% MMMU (%) (val) | 69.1% Leaderboard | Same |
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)
The benchmarks the lab chose to headline, with the scores it gave.
Every result in the longer documents the lab published with it. 34 of them aren't in the launch post.