GPT-3.5 Turbo Jan 2024
GPT-3.5 Turbo ranks first on TOMG-Bench.
41
Benchmarks with a score
1
Ranked first, out of five or more models
3
In the top three
OpenAI Jan 2024
GPT-3.5 Turbo
Jan 2024 snapshot Closed weights
113.1
#199 of 256
Epoch Capabilities Index
- GPQA Diamond 27.2% #215 /229
- IHEval 33.9% #4 /10
- TOMG-Bench 18.6% #1 /5
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (HHEM) | 1.90% | #21 of 148 | Official leaderboard | |
| CriticEval | 5.89 | #2 of 12 | Official leaderboard | |
| FaithJudge | 14.9% | #10 of 37 | Official leaderboard | |
| MixEval | 79.7% | #21 of 34 | Official leaderboard | |
| MMLU-CF | 58.2% | #23 of 37 | Official leaderboard | |
| Chinese SimpleQA | 30.1% | #18 of 27 | Official leaderboard | |
| RepoQA | 60.4% | #15 of 22 | Official leaderboard | |
| EQ-Bench v2 | 65.0 | #59 of 85 | Official leaderboard | |
| ThaiExam | 39.2% | #17 of 24 | Official leaderboard | |
| ECLeKTic | 10.3% | #25 of 35 | Official leaderboard | |
| WildBench | 32.3 | #38 of 47 | Official leaderboard | |
| LMArena (Elo) | 1,226 | #273 of 335 | Official leaderboard | |
| RewardBench | 65.3% | #47 of 48 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ZebraLogic | 10.1% | #36 of 46 | Official leaderboard | |
| MATH Level 5 | 11.6% | #86 of 94 | Epoch AI run | |
| OTIS Mock AIME 2024-2025 | 2.22% | #195 of 212 | Epoch AI run | |
| GPQA Diamond | 27.2% | #215 of 229 | Epoch AI run | |
| FrontierMath Tiers 1-3 v2 | 0% | #92 of 92 | Epoch AI run |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| DevBench Acceptance Testing | 11.7% | #1 of 2 | Official leaderboard | |
| DevBench Environment Setup | 33.3% | #1 of 2 | Official leaderboard | |
| DevBench Unit Testing | 28.7% | #1 of 2 | Official leaderboard | |
| BigCodeBench | 44.9% | #35 of 69 | Official leaderboard | |
| MHPP | 28.1% | #13 of 23 | Official leaderboard | |
| BigCodeBench-Hard | 19.9% | #61 of 96 | Official leaderboard | |
| DS-1000 | 39.4% | #3 of 4 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GTA | 23.6% | #3 of 15 | Official leaderboard | |
| Pencil Puzzle Bench | 0% | #30 of 53 | Official leaderboard | |
| SuperCLUE-Agent | 65.4 | #4 of 7 | Official leaderboard | |
| WebCanvas | 21.2% | #2 of 3 | Official leaderboard | |
| LisanBench | 189.7 | #92 of 133 | Official leaderboard | |
| Mystery Game Puzzles | 3.00% | #83 of 84 | Epoch AI run |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OR-Bench | 12.7% | #5 of 28 | Official leaderboard | |
| TOMG-Bench | 18.6% | #1 of 5 | Official leaderboard | |
| AgentDojo | 10.3% | #8 of 20 | Official leaderboard | |
| IHEval | 33.9% | #4 of 10 | Official leaderboard | |
| SALAD-Bench | 88.6% | #9 of 13 | Official leaderboard | |
| Anthropic Red Team | 96.7% | #62 of 75 | Official leaderboard | |
| AIR-Bench 2024 | 44% | #73 of 82 | Official leaderboard | |
| SimpleSafetyTests | 92% | #70 of 75 | Official leaderboard | |
| HELM Safety | 81.4 | #71 of 75 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LMCA | 9.65% | #133 of 147 | Official leaderboard |
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)