Qwen2.5-Coder-32B
Qwen2.5-Coder-32B places in the top three on MultiPL-E.
13
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Qwen Sep 2024 Qwen2.5-Coder-32B
Open weights
118.9
#187 of 256
Epoch Capabilities Index
- LiveSQLBench 12.5% #32 /36
- OIBench 1.90% #7 /9
- AutoCodeBench 35.8% #25 /34
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MuSR | 45.3% | #13 of 98 | Official leaderboard | |
| MMLU | 79.1% | #12 of 85 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| WinoGrande | 80.8% | #16 of 76 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| HellaSwag | 83% | #16 of 63 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| ARC-Challenge | 70.5% | #22 of 75 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI |
Math and science
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Aider-Edit | 72.9% | #7 of 34 | Official leaderboard | |
| MultiPL-E | 55.1% | #2 of 9 | Official leaderboard | |
| AutoCodeBench | 35.8% | #25 of 34 | Official leaderboard | |
| OIBench | 1.90% | #7 of 9 | Official leaderboard | |
| LiveSQLBench | 12.5% | #32 of 36 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open LLM Leaderboard v2 | 33.3 | #16 of 99 | Official leaderboard |
Qwen's models over time
Qwen's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)