Benchmark Almanac

Qwen2.5-Coder-3B-Instruct

Qwen2.5-Coder-3B-Instruct places in the top three on Delulu.

Released November 6th, 2024 Compare with others
5
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Unknown lab Nov 2024

Qwen2.5-Coder-3B-Instruct

No Epoch Capabilities Index score yet
  • Delulu 76.7% #3 /5
  • JSONSchemaBench 69% #4 /5

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
JSONSchemaBench 69% #4 of 5 Official leaderboard

Math and science

BenchmarkScoreRankSource
GSM8K 80.7% Not ranked Claim in Qwen2.5-Coder Technical Report, via Epoch AI

Coding

BenchmarkScoreRankSource
Delulu 76.7% #3 of 5 Official leaderboard
CodeElo 160 #8 of 9 Official leaderboard
Aider-Edit 39.1% #41 of 41 Official leaderboard

Who measured it?

Independent runs cover 4 of its 5 benchmarks. For the other 1, we have only its lab's own number or a claim Epoch AI collected.

  • An independent run only 4
  • A claim only 1
Official leaderboards scored it on 4.

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards