Introducing GPT-4.5 Feb 27, 2025
GPT-4.5
GPT-4.5 ranks first on 3 of the 85 benchmarks with five or more models scored.
94
Benchmarks with a score
3
Ranked first, out of five or more models
10
In the top three
OpenAI Feb 2025
GPT-4.5
Closed weights Flagship
136.4
#136 of 256
Epoch Capabilities Index
Self-Reported Index 132.3 #258 of 404
- Humanity's Last Exam 5.44% #40 /47
- ZeroBench 6.00% #45 /76
- GPQA Diamond 68.7% #121 /229
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (HHEM) | 1.20% | #7 of 148 | Official leaderboard | |
| Video SimpleQA | 54.1% | #2 of 27 | Official leaderboard | |
| MMLU-Pro | 86.1% | #24 of 212 | Official leaderboard | |
| Internal API instruction following (OpenAI) | 54% | #1 of 8 | Official leaderboard | |
| MMBench-CN v1.1 | 83.3% | #25 of 178 | Official leaderboard | |
| SimpleQA | 62.5% | #9 of 60 | Lab model card | |
| FaithJudge | 11.9% | #7 of 37 | Official leaderboard | |
| MARS-Bench | 53.3% | #3 of 15 | Official leaderboard | |
| LMArena (Elo) | 1,445 | #70 of 335 | Official leaderboard | |
| Creation-MMBench | -5.88 | #5 of 19 | Official leaderboard | |
| LLM Confabulation Leaderboard | 13.6% | #16 of 58 | Official leaderboard | |
| SimpleQA Hallucination Rate | 37.1% | #4 of 12 | Lab model card | |
| PersonQA | 78% | #3 of 9 | Lab model card | |
| Arena-Hard v2.0 | 50% | #10 of 23 | Official leaderboard | |
| Fiction.LiveBench | 63.9% | #28 of 57 | Fiction.live leaderboard, via Epoch AI | |
| MMMLU | 85.1% | #35 of 71 | Lab model card | |
| WritingBench | 67.6 | #28 of 45 | Official leaderboard | |
| Creative Writing v3 | 60.6 | #81 of 120 | Official leaderboard | |
| EQ-Bench 3 | 1,070 | #59 of 78 | Official leaderboard | |
| Humanity's Last Exam | 5.44% | #40 of 47 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K Platinum Initial 268 | 99.6% | #4 of 19 | Official leaderboard | |
| ForecastBench | 0.13 | #21 of 82 | Official leaderboard | |
| MATH Level 5 | 78.6% | #29 of 94 | Epoch AI run | |
| GSM8K-Platinum | 99.3% | #4 of 8 | Official leaderboard | |
| GPQA Diamond | 68.7% | #121 of 229 | Epoch AI run | |
| PolyMath | 26.9% | #25 of 38 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 37.8% | #141 of 212 | Epoch AI run | |
| SimpleBench | 34.5% | #81 of 106 | Official leaderboard | |
| AIME 2024 | 36.7% | #78 of 88 | Lab model card | |
| ARC-AGI-2 | 0.80% | #87 of 98 | Official leaderboard | |
| ARC-AGI | 10.3% | #86 of 95 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenAI Research Engineer Interview: Multiple Choice | 80% | #3 of 9 | Lab model card | |
| Agentic tasks (autonomous replication and adaptation) | 40% | #2 of 4 | Lab model card | |
| SWE-Lancer Diamond (SWE Manager) | 44% | #2 of 4 | Lab model card | |
| Aider Polyglot | 44.9% | #25 of 46 | Official leaderboard | |
| WeirdML | 39.4% | #80 of 112 | Official leaderboard | |
| OpenAI PRs | 7.00% | #11 of 12 | Lab model card | |
| SWE-bench Verified | 38% | #148 of 155 | Lab model card | |
| MLE-bench | 11% | #10 of 10 | Lab model card | |
| SWE-Lancer | 32.6% | #10 of 10 | Lab model card | |
| OpenAI Research Engineer Interview: Coding | 79% | #7 of 7 | Lab model card |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Berkeley Function Calling Leaderboard (BFCL) | 70.3% | #5 of 67 | Official leaderboard | |
| Elimination Game | 5 | #13 of 56 | Official leaderboard | |
| ToolComp | 63.8% | #10 of 32 | Official leaderboard | |
| Galileo Agent Leaderboard | 0.90 | #10 of 29 | Official leaderboard | |
| LLM Chess | -111 | #79 of 123 | Official leaderboard | |
| Step Race | 1 | #53 of 59 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| CharXiv (DQ) | 90% | #3 of 67 | Official leaderboard | |
| MM-Vet | 75.3% | #11 of 164 | Official leaderboard | |
| MMStar | 69.3% | #15 of 173 | Official leaderboard | |
| CharXiv | 55.4% | #7 of 77 | Official leaderboard | |
| VHELM | 72.8 | #5 of 41 | Official leaderboard | |
| MMSI-Bench | 40.3% | #4 of 28 | Official leaderboard | |
| MMBench-EN v1.1 | 83.5% | #27 of 178 | Official leaderboard | |
| MMMU | 74.4% | #23 of 136 | Official leaderboard | |
| VPCT | 45% | #11 of 27 | VPCT leaderboard, via Epoch AI | |
| Vision Arena | 1,225 | #65 of 136 | Official leaderboard | |
| MATH-Vision (MathVision) | 47.3% | #55 of 105 | Official leaderboard | |
| VISTA | 42.1% | #30 of 54 | Official leaderboard | |
| ZeroBench | 6.00% | #45 of 76 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MakeMeSay | 72% | #1 of 10 | Lab model card | |
| SimpleSafetyTests | 100% | #9 of 75 | Official leaderboard | |
| MakeMePay | 57% | #1 of 6 | Lab model card | |
| Multimodal Troubleshooting Virology | 56% | #5 of 27 | Lab model card | |
| HELM Safety | 96.5 | #15 of 75 | Official leaderboard | |
| AIRTBench | 36.9% | #2 of 9 | Official leaderboard | |
| WildChat (toxic conversations) | 98% | #1 of 4 | Lab model card | |
| Anthropic Red Team | 99.4% | #24 of 75 | Official leaderboard | |
| AIR-Bench 2024 | 74.2% | #29 of 82 | Official leaderboard | |
| Human Sourced Jailbreaks | 99% | #3 of 8 | Lab model card | |
| Tacit Knowledge and Troubleshooting | 72% | #15 of 30 | Lab model card | |
| Cybench | 17.5% | #8 of 16 | Official leaderboard | |
| XSTest | 85% | #8 of 16 | Lab model card | |
| Challenging Red Teaming Evaluation 1 (created for o3-mini) | 51% | #1 of 2 | Lab model card | |
| Image input safety evaluations | 99% | #1 of 2 | Lab model card | |
| Capture the Flag (High School) | 53% | #5 of 9 | Lab model card | |
| Molecular Biology Capabilities Test | 46.5% | #53 of 90 | Official leaderboard | |
| Gray Swan Agent Red Teaming (ART) benchmark | 2.58% | #5 of 8 | Official leaderboard | |
| WMDP Biology | 85% | #5 of 8 | Lab model card | |
| Long-form Biological Risk Questions | 0% | #2 of 3 | Lab model card | |
| Capture the Flag (Collegiate) | 16% | #9 of 13 | Lab model card | |
| Human Pathogen Capabilities Test | 43.3% | #62 of 86 | Official leaderboard | |
| Standard Refusal Evaluation | 99% | #6 of 8 | Lab model card | |
| Radiological and Nuclear Expert Knowledge | 68% | #3 of 4 | Lab model card | |
| BioLP-bench | 29% | #7 of 9 | Lab model card | |
| Virology Capabilities Test (VCT) | 28.3% | #6 of 7 | Official leaderboard | |
| Capture the Flag (Professional) | 2.00% | #19 of 21 | Lab model card | |
| ProtocolQA Open-Ended | 18% | #26 of 27 | Lab model card | |
| World-Class Bio | 4.20% | #74 of 74 | Official leaderboard | |
| StrongREJECT | 34% | #19 of 19 | Lab model card | |
| Challenging Refusal Evaluation | 85% | #9 of 9 | Lab model card | |
| Contextual Nuclear Knowledge | 71% | #4 of 4 | Lab model card | |
| Challenging Red Teaming Evaluation 2 (created for deep research) | 46% | #2 of 2 | Lab model card |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 75.3 | #11 of 173 | Official leaderboard |
Its lab's numbers put it at 132.3, 4.1 points below its ECI
The Self-Reported Index fits one score to the 8 capability results OpenAI reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 125.9 to 138.7. How we compute it.
- MMMLU 85.1
- PersonQA 78
- MMMU 74.4
- GPQA Diamond 71.4
- SimpleQA 62.5
- SWE-bench Verified 38
- AIME 2024 36.7
- SWE-Lancer 32.6
Its lab's numbers next to 3 leaderboards
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 33 of them aren't in the launch post.
OpenAI GPT-4.5 System Card Feb 27, 2025
Capabilities
Hallucination
- PersonQA 78