Qwen2.5-Coder-3B-Instruct
Qwen2.5-Coder-3B-Instruct places in the top three on Delulu.
Released November 6th, 2024 Compare with others
5
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Unknown lab Nov 2024
Qwen2.5-Coder-3B-Instruct
No Epoch Capabilities Index score yet
- Delulu 76.7% #3 /5
- JSONSchemaBench 69% #4 /5
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| JSONSchemaBench | 69% | #4 of 5 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 80.7% | Not ranked | Claim in Qwen2.5-Coder Technical Report, via Epoch AI |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Delulu | 76.7% | #3 of 5 | Official leaderboard | |
| CodeElo | 160 | #8 of 9 | Official leaderboard | |
| Aider-Edit | 39.1% | #41 of 41 | Official leaderboard |
Who measured it?
Independent runs cover 4 of its 5 benchmarks. For the other 1, we have only its lab's own number or a claim Epoch AI collected.
- An independent run only 4
- A claim only 1
Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards