Qwen2.5-Coder-32B-Instruct
Qwen2.5-Coder-32B-Instruct ranks first on MultiPL-E.
24
Benchmarks with a score
1
Ranked first, out of five or more models
4
In the top three
Qwen Nov 2024 Qwen2.5-Coder-32B-Instruct
Open weights
No Epoch Capabilities Index score yet
- BigCodeBench-Hard 30.8% #18 /96
- ArtifactsBench (Gemini 2.5 Pro 0506 judge) 35.3% #14 /18
- BigO(Bench) 34.9% #9 /18
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MuSR | 43.9% | #28 of 98 | Official leaderboard | |
| TableBench | 45.5% | #7 of 10 | Official leaderboard | |
| LMArena (Elo) | 1,271 | #253 of 335 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 93% | #3 of 70 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MBPP | 90.5% | #3 of 35 | Official leaderboard | |
| MBPP+ | 77% | #3 of 35 | Official leaderboard | |
| BigCodeBench | 53.5% | #6 of 69 | Official leaderboard | |
| HumanEval | 92.1% | #4 of 46 | Official leaderboard | |
| HumanEval+ | 87.2% | #4 of 46 | Official leaderboard | |
| MultiPL-E | 63.4% | #1 of 9 | Official leaderboard | |
| BigCodeBench-Hard | 30.8% | #18 of 96 | Official leaderboard | |
| Aider-Edit | 71.4% | #12 of 34 | Official leaderboard | |
| BigO(Bench) | 34.9% | #9 of 18 | Official leaderboard | |
| FullStackBench | 56.9% | #6 of 10 | Official leaderboard | |
| SWE-bench Verified | 38% | #46 of 60 | Official leaderboard | |
| ArtifactsBench (Gemini 2.5 Pro 0506 judge) | 35.3% | #14 of 18 | Official leaderboard | |
| Aider Polyglot | 16.4% | #38 of 46 | Official leaderboard | |
| Spider 2.0 | 5.48% | #23 of 23 | Official leaderboard | |
| BIRD-CRITIC | 20% | #9 of 9 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AceBench | 79.6% | #4 of 16 | Official leaderboard | |
| SnakeBench | 20 | #135 of 188 | Official leaderboard | |
| GAIA | 0.33% | #63 of 64 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| nocot-bench (NCRI) | 60.0 | #138 of 185 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open LLM Leaderboard v2 | 39.9 | #5 of 99 | Official leaderboard |