Benchmark Almanac

Gemma 3 27B PT

Gemma 3 27B PT ranks first on 3 of the 24 benchmarks with five or more models scored.

Google DeepMind Released March 12th, 2025 Family Gemma 3 The lab's model card Compare with others
24
Benchmarks with a score
3
Ranked first, out of five or more models
7
In the top three
Google DeepMind Mar 2025

Gemma 3 27B PT

Open weights

No Epoch Capabilities Index score yet
  • Alyah 74.7% #2 /16
  • Angry Tweets 54.3% #70 /149
  • FoQA 77.2% #6 /91

Where has it been number one?

Gemma 3 27B PT is number one on 1 benchmark today. It still tops 2 that nobody has tested since.

Source: Official leaderboards

How does it do against its rivals?

Gemma 3 27B PT beats Qwen3-14B on 12 of the 17 benchmarks they share. Qwen3-32B does best against it, ahead on 6 of 11.

  1. Qwen3-14B 12 ahead, 5 behind Compare
  2. Aya Expanse 32B 8 ahead, 4 behind Compare
  3. QwQ-32B 5 ahead, 3 behind Compare
  4. Apertus 70B 4 ahead, 4 behind Compare
  5. Voxtral Small 4 ahead, 4 behind Compare
  6. Qwen3-32B 5 ahead, 6 behind Compare

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
EuroEval Faroese Grammatical Error Correction 16.9% #1 of 71 Official leaderboard
DaLA 61.7% #1 of 25 Official leaderboard
MultiBLiMP 90.2% #1 of 25 Official leaderboard
FoQA 77.2% #6 of 91 Official leaderboard
GreekMMLU 82.6% #3 of 34 Official leaderboard
GermanQuAD 68% #8 of 88 Official leaderboard
Open Hebrew LLM Leaderboard 69.5 #9 of 88 Official leaderboard
Uncheatable Eval 6.58 #4 of 33 Official leaderboard
Alyah 74.7% #2 of 16 Official leaderboard
BLiMP-NL 48.2% #3 of 21 Official leaderboard
Dutch CoLA 48.2% #3 of 21 Official leaderboard
GuiaCat 62.3% #7 of 35 Official leaderboard
HeQ 78.3% #21 of 88 Official leaderboard
QIMMA 57.4% #12 of 44 Official leaderboard
Dutch Book Reviews Dataset 90.1% #37 of 105 Official leaderboard
3LM 78.8% #9 of 25 Official leaderboard
FoSent 52.2% #60 of 147 Official leaderboard
NOVA-63 51.2% #22 of 52 Official leaderboard
Angry Tweets 54.3% #70 of 149 Official leaderboard
Global PIQA 67.3% #28 of 59 Official leaderboard
EuroEval 2.16 #38 of 75 Official leaderboard
OrangeSum 31.8% #24 of 34 Official leaderboard
EuroEval ERRNews 24.4 #59 of 76 Official leaderboard

Safety and security

BenchmarkScoreRankSource
nocot-bench (NCRI) 57.5 #157 of 201 Official leaderboard

Who measured it?

Independent runs cover all 24 of its benchmarks.

  • An independent run only 24
Official leaderboards scored it on 24.

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards