Benchmark Almanac

Gemma 3 1B PT

Gemma 3 1B PT.

Google DeepMind Released March 12th, 2025 Family Gemma 3 The lab's model card Compare with others
26
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Google DeepMind Mar 2025

Gemma 3 1B PT

Open weights Small

No Epoch Capabilities Index score yet
  • Angry Tweets 36.7% #127 /149
  • FoQA 33.7% #62 /91
  • GermanQuAD 43.2% #65 /88

How does it do against its rivals?

Gemma 3 1B PT beats DeepSeek-R1-Distill-Qwen-7B on 6 of the 12 benchmarks they share. Llama 3.2 1B does best against it, ahead on 12 of 21.

  1. DeepSeek-R1-Distill-Qwen-7B 6 ahead, 6 behind Compare
  2. SmolLM2-1.7B 5 ahead, 5 behind Compare
  3. Qwen3-0.6B 8 ahead, 10 behind Compare
  4. DeepSeek-R1-Distill-Qwen-14B 4 ahead, 5 behind Compare
  5. DeepSeek-R1-Distill-Qwen-32B 4 ahead, 5 behind Compare
  6. Llama 3.2 1B 9 ahead, 12 behind Compare

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
GuiaCat 59.9% #9 of 35 Official leaderboard
EuroEval Faroese Grammatical Error Correction 0.51% #44 of 71 Official leaderboard
FoQA 33.7% #62 of 91 Official leaderboard
BLiMP-NL 5.90% #15 of 21 Official leaderboard
Dutch CoLA 5.90% #15 of 21 Official leaderboard
GermanQuAD 43.2% #65 of 88 Official leaderboard
EusExams 26% #19 of 24 Official leaderboard
FoSent 22.2% #118 of 147 Official leaderboard
Global PIQA 54.7% #48 of 59 Official leaderboard
EusReading 25.6% #20 of 24 Official leaderboard
Angry Tweets 36.7% #127 of 149 Official leaderboard
Open Portuguese LLM Leaderboard 35.1 #153 of 175 Official leaderboard
Uncheatable Eval 9.32 #29 of 33 Official leaderboard
Dutch Book Reviews Dataset 56.4% #94 of 105 Official leaderboard
EuroEval ERRNews 20.3 #69 of 76 Official leaderboard
EuroEval 3.81 #69 of 75 Official leaderboard
BLUEX 22.3% #164 of 177 Official leaderboard
OrangeSum 24% #32 of 34 Official leaderboard
EusTrivia 27.1% #23 of 24 Official leaderboard
DaLA 0.68% #24 of 25 Official leaderboard
QIMMA 27.4% #43 of 44 Official leaderboard
NOVA-63 22.1% #52 of 52 Official leaderboard
GreekMMLU 31.9% #34 of 34 Official leaderboard
MultiBLiMP 55.3% #25 of 25 Official leaderboard

Safety and security

BenchmarkScoreRankSource
CaBBQ 41% #11 of 11 Official leaderboard
EsBBQ 35.5% #11 of 11 Official leaderboard

Who measured it?

Independent runs cover all 26 of its benchmarks.

  • An independent run only 26
Official leaderboards scored it on 26.

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards