davinci-002
davinci-002 ranks first on GSM8K.
9
Benchmarks with a score
1
Ranked first, out of five or more models
2
In the top three
OpenAI Mar 2022
davinci-002
Closed weights
No Epoch Capabilities Index score yet
- Situational Awareness Dataset (SAD) 30.1% #17 /17
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ARC-Challenge | 85.2% | #11 of 75 | Claim in GPT-4 Technical Report, via Epoch AI | |
| WinoGrande | 81.6% | #12 of 76 | Claim in GPT-4 technical report, via Epoch AI | |
| HellaSwag | 81.5% | #2 of 12 | Stanford HELM, via Epoch AI | |
| MMLU | 70% | #25 of 85 | Claim in GPT-4 Technical Report, via Epoch AI |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 56.8% | #1 of 24 | Stanford HELM, via Epoch AI |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| METR Time Horizon 1.1 | 0.1 min | #25 of 26 | Official leaderboard | |
| METR Time Horizon | 0.1 min | #30 of 31 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| nocot-bench (NCRI) | 32.6 | #181 of 185 | Official leaderboard | |
| Situational Awareness Dataset (SAD) | 30.1% | #17 of 17 | Official leaderboard |