Benchmark Almanac

SmolLM2-1.7B

SmolLM2-1.7B.

Hugging Face Released October 31st, 2024 Family SmolLM2 The lab's model card Compare with others
16
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Hugging Face Oct 2024

SmolLM2-1.7B

Open weights Small

No Epoch Capabilities Index score yet
  • Angry Tweets 8.53% #142 /149
  • FoQA 25.7% #72 /91
  • GermanQuAD 39% #68 /88

How does it do against its rivals?

SmolLM2-1.7B beats DeepSeek-R1-Distill-Qwen-1.5B on 6 of the 9 benchmarks they share. Llama 3.2 1B does best against it, ahead on 10 of 14.

  1. DeepSeek-R1-Distill-Qwen-1.5B 6 ahead, 3 behind Compare
  2. Gemma 3 1B PT 5 ahead, 5 behind Compare
  3. DeepSeek-R1-Distill-Qwen-7B 5 ahead, 6 behind Compare
  4. DeepSeek-R1-Distill-Qwen-14B 3 ahead, 5 behind Compare
  5. Qwen3-0.6B 3 ahead, 5 behind Compare
  6. Llama 3.2 1B 4 ahead, 10 behind Compare

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
EusReading 27.6% #15 of 24 Official leaderboard
EuroEval Faroese Grammatical Error Correction 0.36% #47 of 71 Official leaderboard
EusExams 29.1% #16 of 24 Official leaderboard
GermanQuAD 39% #68 of 88 Official leaderboard
FoQA 25.7% #72 of 91 Official leaderboard
EusTrivia 29.9% #19 of 24 Official leaderboard
Uncheatable Eval 8.71 #27 of 33 Official leaderboard
Dutch Book Reviews Dataset 74.3% #88 of 105 Official leaderboard
Open Hebrew LLM Leaderboard 29.1 #80 of 88 Official leaderboard
MMLU-Pro 18.3% #198 of 212 Official leaderboard
HeQ 25.5% #83 of 88 Official leaderboard
Angry Tweets 8.53% #142 of 149 Official leaderboard
FoSent 0.77% #144 of 147 Official leaderboard
SEND Pedagogy Benchmark 28% #226 of 230 Official leaderboard
Pedagogy Benchmark 27.6% #213 of 216 Official leaderboard

Agents and tools

BenchmarkScoreRankSource
WordleArenas 0% #28 of 32 Official leaderboard

Who measured it?

Independent runs cover all 16 of its benchmarks.

  • An independent run only 16
Official leaderboards scored it on 16.

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards