Benchmark Almanac

SmolLM2-135M

SmolLM2-135M.

Hugging Face Released October 31st, 2024 Family SmolLM2 The lab's model card Compare with others
16
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Hugging Face Oct 2024

SmolLM2-135M

Open weights Small

No Epoch Capabilities Index score yet
  • Angry Tweets 3.55% #149 /149
  • FoQA 2.52% #89 /91
  • GermanQuAD 4.89% #88 /88

How does it do against its rivals?

SmolLM2-135M beats Gemma 2 27B on 1 of the 8 benchmarks they share. Gemma 3 12B does best against it, ahead on 13 of 13.

  1. Gemma 2 27B 1 ahead, 7 behind Compare
  2. Qwen3-8B 1 ahead, 9 behind Compare
  3. Qwen3-0.6B 1 ahead, 12 behind Compare
  4. Llama 3.1-8B 0 ahead, 14 behind Compare
  5. Llama 3.2 1B 0 ahead, 14 behind Compare
  6. Gemma 3 12B 0 ahead, 13 behind Compare

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
EuroEval Faroese Grammatical Error Correction 0.21% #53 of 71 Official leaderboard
BLiMP-NL 0.70% #20 of 21 Official leaderboard
Dutch CoLA 0.70% #20 of 21 Official leaderboard
OrangeSum 22.3% #33 of 34 Official leaderboard
FoSent 1.38% #143 of 147 Official leaderboard
EuroEval ERRNews 13.6 #74 of 76 Official leaderboard
FoQA 2.52% #89 of 91 Official leaderboard
SEND Pedagogy Benchmark 24.8% #230 of 230 Official leaderboard
Pedagogy Benchmark 21.1% #216 of 216 Official leaderboard
MMLU-Pro 10.8% #212 of 212 Official leaderboard
Angry Tweets 3.55% #149 of 149 Official leaderboard
Dutch Book Reviews Dataset 1.78% #105 of 105 Official leaderboard
GermanQuAD 4.89% #88 of 88 Official leaderboard
GuiaCat 8.87% #35 of 35 Official leaderboard
Uncheatable Eval 11.5 #33 of 33 Official leaderboard

Agents and tools

BenchmarkScoreRankSource
WordleArenas 0% #29 of 32 Official leaderboard

Who measured it?

Independent runs cover all 16 of its benchmarks.

  • An independent run only 16
Official leaderboards scored it on 16.

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards