Qwen2.5-Coder-14B
Qwen2.5-Coder-14B.
Released September 18th, 2024 Compare with others
11
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Unknown lab Sep 2024
Qwen2.5-Coder-14B
113.9
#196 of 256
Epoch Capabilities Index
- Open Arabic LLM Leaderboard v1 46% #24 /55
- Ko-IFEval (Upstage) 33% #15 /21
- GAIA 9.30% #58 /64
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMLU | 75.2% | #19 of 85 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| ARC-Challenge | 66% | #28 of 75 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| WinoGrande | 76.8% | #30 of 76 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| HellaSwag | 80.2% | #26 of 63 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI | |
| Open Arabic LLM Leaderboard v1 | 46% | #24 of 55 | Official leaderboard | |
| MuSR | 38.7% | #69 of 98 | Official leaderboard | |
| Ko-IFEval (Upstage) | 33% | #15 of 21 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 88.7% | #9 of 70 | Claim in Qwen2.5-Coder Technical Report, via Epoch AI |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Aider-Edit | 61.7% | #20 of 34 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GAIA | 9.30% | #58 of 64 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open LLM Leaderboard v2 | 24.8 | #42 of 99 | Official leaderboard |
The lab's models over time
's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)