- MMLU (5-shot) 86.4
- HellaSwag (10-shot) 95.3
- AI2 Reasoning Challenge (ARC) (25-shot) 96.3
- WinoGrande (5-shot) 87.5
- HumanEval (0-shot) 67
- DROP (F1 score) (3-shot) 80.9
- GSM-8K (5-shot chain-of-thought) 92
- VQAv2 VQA score (test-dev) 0-shot 77.2
- TextVQA VQA score (val) 0-shot 78
- ChartQA Relaxed accuracy (test) 78.5
- AI2 Diagram (AI2D) Accuracy (test) 0-shot 78.2
- DocVQA ANLS score (test) 0-shot (pixel-only) 88.4
- Infographic VQA ANLS score (test) 0-shot (pixel-only) 75.1
- TVQA Accuracy (val) 0-shot 87.3
- LSMDC Fill-in-the-blank accuracy (test) 0-shot 45.7
GPT-4 Mar 2023
GPT-4 ranks first on 2 of the 29 benchmarks with five or more models scored.
67
Benchmarks with a score
2
Ranked first, out of five or more models
5
In the top three
OpenAI Mar 2023
GPT-4
Mar 2023 snapshot Closed weights Flagship
121.8
#180 of 256
Epoch Capabilities Index
Self-Reported Index 123.8 #322 of 404
- GPQA Diamond 35.7% #195 /229
- TextVQA 78% #10 /17
- Molecular Biology Capabilities Test 27.5% #86 /90
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| EQ-Bench v2 | 85.7 | #3 of 85 | Official leaderboard | |
| HellaSwag | 95.3% | #3 of 53 | Lab model card | |
| ARC-Challenge | 96.3% | #4 of 49 | Lab model card | |
| WinoGrande | 87.5% | #4 of 47 | Lab model card | |
| AlpacaEval | 94.8% | #3 of 19 | Official leaderboard | |
| GRE Verbal | 169.0 | #1 of 6 | Lab model card | |
| AlpacaEval 2.0 | 35.3% | #11 of 47 | Official leaderboard | |
| LSAT | 163.0 | #1 of 4 | Lab model card | |
| MMLU | 86.4% | #46 of 131 | Lab model card | |
| QuALITY | 82.6% | #2 of 4 | Official leaderboard | |
| TruthfulQA | 60% | #16 of 30 | Lab model card | |
| DROP | 80.9% | #24 of 41 | Lab model card | |
| LMArena (Elo) | 1,288 | #240 of 335 | Official leaderboard | |
| Creative Writing v3 | 50.8 | #106 of 120 | Official leaderboard | |
| EQ-Bench 3 | 653 | #75 of 78 | Official leaderboard | |
| GRE Analytical Writing | 4.00 | #3 of 3 | Lab model card | |
| AP Art History | 5.00 | #1 of 1 | Lab model card | |
| AP Biology | 5.00 | #1 of 1 | Lab model card | |
| AP Calculus BC | 4.00 | #1 of 1 | Lab model card | |
| AP Chemistry | 4.00 | #1 of 1 | Lab model card | |
| AP English Language and Composition | 2.00 | #1 of 1 | Lab model card | |
| AP English Literature and Composition | 2.00 | #1 of 1 | Lab model card | |
| AP Environmental Science | 5.00 | #1 of 1 | Lab model card | |
| AP Macroeconomics | 5.00 | #1 of 1 | Lab model card | |
| AP Microeconomics | 5.00 | #1 of 1 | Lab model card | |
| AP Physics 2 | 4.00 | #1 of 1 | Lab model card | |
| AP Psychology | 5.00 | #1 of 1 | Lab model card | |
| AP Statistics | 5.00 | #1 of 1 | Lab model card | |
| AP US Government | 5.00 | #1 of 1 | Lab model card | |
| AP US History | 5.00 | #1 of 1 | Lab model card | |
| AP World History | 4.00 | #1 of 1 | Lab model card | |
| MMLU calibration (expected calibration error) | 0.07 | #1 of 1 | Lab model card | |
| Preference on ChatGPT and API prompts | 70.2% | #1 of 1 | Lab model card | |
| SAT Evidence-Based Reading & Writing | 710.0 | #1 of 1 | Lab model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GRE Quantitative | 163.0 | #1 of 6 | Lab model card | |
| ZebraLogic | 27.1% | #14 of 46 | Official leaderboard | |
| GSM8K | 92% | #32 of 92 | Lab model card | |
| GPQA Diamond | 35.7% | #195 of 229 | Epoch AI run | |
| OTIS Mock AIME 2024-2025 | 0.56% | #207 of 212 | Epoch AI run | |
| AMC 10 2022 | 30.0 | #1 of 1 | Lab model card | |
| AMC 12 2022 | 60.0 | #1 of 1 | Lab model card | |
| SAT Math | 700.0 | #1 of 1 | Lab model card | |
| USABO Semifinal Exam 2020 | 87.0 | #1 of 1 | Lab model card | |
| USNCO Local Section Exam 2022 | 36.0 | #1 of 1 | Lab model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval+ | 79.3% | #12 of 46 | Official leaderboard | |
| HumanEval | 67% | #76 of 101 | Lab model card | |
| METR Time Horizon 1.1 | 4.0 min | #21 of 26 | Official leaderboard | |
| METR Time Horizon | 6.0 min | #26 of 31 | Official leaderboard | |
| Codeforces rating | 392 | #47 of 47 | Lab model card | |
| Leetcode (easy) | 75.6% | #1 of 1 | Lab model card | |
| Leetcode (hard) | 6.70% | #1 of 1 | Lab model card | |
| Leetcode (medium) | 26.3% | #1 of 1 | Lab model card |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| PubMedQA | 75.2% | #2 of 3 | Official leaderboard | |
| Advanced Sommelier (theory knowledge) | 77% | #1 of 1 | Lab model card | |
| Certified Sommelier (theory knowledge) | 86% | #1 of 1 | Lab model card | |
| Introductory Sommelier (theory knowledge) | 92% | #1 of 1 | Lab model card | |
| Medical Knowledge Self-Assessment Program (MKSAP) | 75% | #1 of 1 | Lab model card | |
| Uniform Bar Exam (MBE+MEE+MPT) | 298.0 | #1 of 1 | Lab model card |
Vision, audio and robotics
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| DecodingTrust | 69.2 | #4 of 5 | Official leaderboard | |
| Molecular Biology Capabilities Test | 27.5% | #86 of 90 | Official leaderboard | |
| RealToxicityPrompts | 0.73% | #2 of 2 | Lab model card |
Its lab's numbers put it at 123.8, 2.0 points above its ECI
The Self-Reported Index fits one score to the 11 capability results OpenAI reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 119.2 to 128.5. How we compute it.
- ARC-Challenge 96.3
- HellaSwag 95.3
- GSM8K 92
- WinoGrande 87.5
- MMLU 86.4
- DROP 80.9
- ChartQA 78.5
- AI2D 78.2
- TextVQA 78
- HumanEval 67
- TruthfulQA 60
Its lab's number next to a leaderboard
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 43 of them aren't in the launch post.
Capabilities
- Uniform Bar Exam (MBE+MEE+MPT) 298
- LSAT 163
- SAT Evidence-Based Reading & Writing 710
- SAT Math 700
- GRE Quantitative 163
- GRE Verbal 169
- GRE Analytical Writing 4
- USABO Semifinal Exam 2020 87
- USNCO Local Section Exam 2022 36
- Medical Knowledge Self-Assessment Program (MKSAP) 75
- Codeforces rating 392
- AP Art History 5
- AP Biology 5
- AP Calculus BC 4
- AP Chemistry 4
- AP English Language and Composition 2
- AP English Literature and Composition 2
- AP Environmental Science 5
- AP Macroeconomics 5
- AP Microeconomics 5
- AP Physics 2 4
- AP Psychology 5
- AP Statistics 5
- AP US Government 5
- AP US History 5
- AP World History 4
- AMC 10 2022 30
- AMC 12 2022 60
- Introductory Sommelier (theory knowledge) 92
- Certified Sommelier (theory knowledge) 86
- Advanced Sommelier (theory knowledge) 77
- Leetcode (easy) 75.6
- Leetcode (medium) 26.3
- Leetcode (hard) 6.7
- MMLU 86.4
- HellaSwag 95.3
- ARC-Challenge 96.3
- WinoGrande 87.5
- HumanEval 67
- DROP 80.9
- GSM8K 92
- MMLU translated with Azure Translate
- Hindsight Neglect
- Preference on ChatGPT and API prompts 70.2