Benchmark Almanac

Gemma 3 12B PT

Gemma 3 12B PT places in the top three on 6 benchmarks.

27
Benchmarks with a score
0
Ranked first, out of five or more models
6
In the top three
Google DeepMind Mar 2025

Gemma 3 12B PT

Open weights Mid-size

No Epoch Capabilities Index score yet
  • Angry Tweets 53% #76 /149
  • FoQA 77.3% #5 /91
  • GermanQuAD 68.4% #4 /88

How does it do against its rivals?

Gemma 3 12B PT beats Aya Expanse 32B on 7 of the 13 benchmarks they share. Apertus 70B does best against it, ahead on 5 of 8.

  1. Aya Expanse 32B 7 ahead, 6 behind Compare
  2. Qwen3-14B 9 ahead, 8 behind, 1 tied Compare
  3. Voxtral Small 4 ahead, 4 behind Compare
  4. Phi-4 6 ahead, 7 behind Compare
  5. Qwen3-32B 5 ahead, 8 behind Compare
  6. Apertus 70B 3 ahead, 5 behind Compare

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
EuroEval Faroese Grammatical Error Correction 12.9% #3 of 71 Official leaderboard
GermanQuAD 68.4% #4 of 88 Official leaderboard
FoQA 77.3% #5 of 91 Official leaderboard
GuiaCat 67.5% #3 of 35 Official leaderboard
BLiMP-NL 48.5% #2 of 21 Official leaderboard
Dutch CoLA 48.5% #2 of 21 Official leaderboard
DaLA 52.3% #3 of 25 Official leaderboard
MultiBLiMP 88.8% #3 of 25 Official leaderboard
Open Hebrew LLM Leaderboard 64.0 #15 of 88 Official leaderboard
BLUEX 68.2% #38 of 177 Official leaderboard
GreekMMLU 77.3% #8 of 34 Official leaderboard
Dutch Book Reviews Dataset 90.7% #28 of 105 Official leaderboard
Uncheatable Eval 6.86 #10 of 33 Official leaderboard
FoSent 55.4% #50 of 147 Official leaderboard
HeQ 71.5% #30 of 88 Official leaderboard
QIMMA 53.2% #18 of 44 Official leaderboard
Open Portuguese LLM Leaderboard 68.6 #74 of 175 Official leaderboard
Angry Tweets 53% #76 of 149 Official leaderboard
3LM 73.3% #13 of 25 Official leaderboard
Global PIQA 64.2% #31 of 59 Official leaderboard
EuroEval 2.37 #46 of 75 Official leaderboard
NOVA-63 47.9% #34 of 52 Official leaderboard
EuroEval ERRNews 24.0 #61 of 76 Official leaderboard
OrangeSum 29.3% #28 of 34 Official leaderboard

Safety and security

BenchmarkScoreRankSource
CaBBQ 50.5% #6 of 11 Official leaderboard
EsBBQ 47.5% #8 of 11 Official leaderboard
nocot-bench (NCRI) 49.9 #176 of 201 Official leaderboard

Who measured it?

Independent runs cover all 27 of its benchmarks.

  • An independent run only 27
Official leaderboards scored it on 27.

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards