NorEval
Models complete language understanding and generation tasks in both Bokmål and Nynorsk. The authors compare model performance with human baselines.
The authors release the datasets and more than 100 human-written prompts for the LM Evaluation Harness.
- Made by
- Vladislav Mikhailov and colleagues
- Released
- April 2025
- Kind
- Knowledge in many languages
- Level
- School
- Scored
- Checked automatically
- Answers
- Open-ended
- Tasks
- All public
- Measures
- Capability
- Subject
- Languages and literature
- Language
- Norwegian Bokmål, Norwegian Nynorsk
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
-
Qwen2.5-72B #166 on the capability index
- Last model added
- April 2026
- Scores refreshed
- November 20th, 2026
DeepSeek-V4-Flash-Base leads with 68.2 among models released by April 2026 DeepSeek-V4-Flash-Base leads with 68.2 among models released by April 2026
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. The dashboard averages chance-adjusted task scores within categories, then weights each category equally. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
The newest model with a score here came out in April 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 68.2
DeepSeek-V4-Flash-Base
- 67.2 Gemma4 31B (pretrained)
- 64.8 Nemotron 4 340B
- 64.0 Gemma3 27B (pretrained)
- 63.9 Gemma4 26B (4B active) (pretrained)
- 62.6
Qwen3.5-35B-A3B-Base
- 61.4 Llama4 Scout 109B (17B active)
- 61.4 GLM-4.5 Air (12B active)
- 61.1
Llama 3.1-70B
- 61.1 TildeOpen 30B 64k
- 60.9 Gemma4 12B (pretrained)
- 60.9 Gemma3 12B (pretrained)
- 59.6 NorOLMo 13B (stage 2, new mix)
- 59.4 NorwAI Mixtral 8x7B (47B)
- 59.2 Mistral 24B
- 58.5 GLM-4 32B
- 58.3
Apertus 70B
- 58.2 Gemma4 E4B (8B) (pretrained)
- 58.0
Qwen3.5-9B-Base
- 57.9 Mistral 24B (v2501)
- 57.8 NorMistral 11B Long
- 57.7 Prelude 9B
- 57.4 NorMistral 11B
- 57.3 Nemotron 3 Super 120B (12B active)
- 56.9
Qwen2.5-72B
- 56.3
Ministral 3 14B
- 56.1 Mixtral 8x22B (141B)
- 55.7 NorOLMo 13B
- 55.5 EuroLLM 1.5 9B
- 55.4 NorOLMo 13B (stage 2)
- 55.4 Bielik 11B v3
- 55.2
Qwen3-14B-Base
- 54.7
Apertus 8B Instruct
- 54.7 EuroLLM 22B
- 54.6
Qwen2.5-32B
- 54.4 Apertus 8B (stage 5)
- 53.6 AI Sweden Llama3 8B
- 53.2 Ministral3 8B
- 52.5 Mistral Nemo 12B
- 52.4
Qwen3-8B-Base
- 51.7 NorOLMo 13B (stage 1)
- 51.5 Apertus 8B (stage 4)
- 50.8 EuroLLM 9B
- 50.6 Mixtral 8x7B (47B)
- 50.6 Gemma3 4B (pretrained)
- 50.5 Nemotron 3 Nano 30B (3B active)
- 50.2 Viking 33B
- 49.5 Apertus 8B (stage 3)
- 49.5
Llama 3.1-8B
- 49.4 Tiny Aya 3B
- 49.3 Apertus 8B (stage 2)
- 49.0 DeepSeek V3 671B (37B active)
- 48.6 Gemma4 E2B (5B) (pretrained)
- 48.6 NorwAI Mistral 7B
- 48.4 Apertus 8B (stage 1)
- 48.4 OLMo3 32B
- 48.3 GPT-SW3 40B
- 48.0
Qwen3-4B-Base
- 47.6 Trinity Mini 26B (3B active)
- 47.0 Ministral3 3B
- 46.6 Marin 32B
- 46.5 NorMistral 7B
- 45.3 GPT-SW3 20B
- 45.1 Viking 13B
- 44.0 GPT-SW3 6.7B
- 42.6
Qwen3.5-2B-Base
- 42.1 Viking 7B
- 42.1
Llama 3.2 3B
- 42.1 Nemotron Nano 12B v2
- 41.7
Mistral 7B v0.1
- 41.2 NorMistral 7B (scratch)
- 40.1 NB-Llama 3.1 8B
- 39.7 Bielik 4.5B v3
- 39.6 Qwen3-1.7B-Base
- 39.4 OLMo2 13B
- 39.2 Marin 8B
- 38.1 NB-Llama 3.2 3B
- 37.5 GPT-SW3 1.3B
- 36.6 OLMo3 7B
- 36.4 Poro 34B
- 36.4 OLMo3 7B (stage 1)
- 35.6 Trinity Nano 6B (1B active)
- 35.3 OLMo2 13B (stage 1)
- 35.3 NB-GPT-J 6B v2
- 33.9
Qwen3.5-0.8B-Base
- 32.1 NB-GPT-J 6B
- 32.0 NorBERT4 1B
- 31.1 Gemma3 1B (pretrained)
- 30.3 EuroLLM 1.7B
- 29.4 DFM 7B
- 29.4 TildeOpen 30B
- 28.9 NB-Llama 3.2 1B
- 28.7 Bielik 1.5B v3
- 27.6
Llama 3.2 1B
- 20.5
OLMo 2 1B Instruct
- 20.4 NorwAI NorGPT 3B
- 8.93 NorwAI NorLlama 3B
- 1
DeepSeek-V4-Flash-Base 5-shot 68.2
- 2 Gemma4 31B (pretrained) 5-shot 67.2
- 3 Nemotron 4 340B 5-shot 64.8
- 4 Gemma3 27B (pretrained) 5-shot 64.0
- 5 Gemma4 26B (4B active) (pretrained) 5-shot 63.9
- 6
Qwen3.5-35B-A3B-Base 5-shot 62.6
- 7 Llama4 Scout 109B (17B active) 5-shot 61.4
- 8 GLM-4.5 Air (12B active) 5-shot 61.4
- 9
Llama 3.1-70B 5-shot 61.1
- 10 TildeOpen 30B 64k 5-shot 61.1
- 11 Gemma4 12B (pretrained) 5-shot 60.9
- 12 Gemma3 12B (pretrained) 5-shot 60.9
- 13 NorOLMo 13B (stage 2, new mix) 5-shot 59.6
- 14 NorwAI Mixtral 8x7B (47B) 5-shot 59.4
- 15 Mistral 24B 5-shot 59.2
- 16 GLM-4 32B 5-shot 58.5
- 17
Apertus 70B 5-shot 58.3
- 18 Gemma4 E4B (8B) (pretrained) 5-shot 58.2
- 19
Qwen3.5-9B-Base 5-shot 58.0
- 20 Mistral 24B (v2501) 5-shot 57.9
- 21 NorMistral 11B Long 5-shot 57.8
- 22 Prelude 9B 5-shot 57.7
- 23 NorMistral 11B 5-shot 57.4
- 24 Nemotron 3 Super 120B (12B active) 5-shot 57.3
- 25
Qwen2.5-72B 5-shot 56.9
- 26
Ministral 3 14B 5-shot 56.3
- 27 Mixtral 8x22B (141B) 5-shot 56.1
- 28 NorOLMo 13B 5-shot 55.7
- 29 EuroLLM 1.5 9B 5-shot 55.5
- 30 NorOLMo 13B (stage 2) 5-shot 55.4
- 31 Bielik 11B v3 5-shot 55.4
- 32
Qwen3-14B-Base 5-shot 55.2
- 33
Apertus 8B Instruct 5-shot 54.7
- 34 EuroLLM 22B 5-shot 54.7
- 35
Qwen2.5-32B 5-shot 54.6
- 36 Apertus 8B (stage 5) 5-shot 54.4
- 37 AI Sweden Llama3 8B 5-shot 53.6
- 38 Ministral3 8B 5-shot 53.2
- 39 Mistral Nemo 12B 5-shot 52.5
- 40
Qwen3-8B-Base 5-shot 52.4
- 41 NorOLMo 13B (stage 1) 5-shot 51.7
- 42 Apertus 8B (stage 4) 5-shot 51.5
- 43 EuroLLM 9B 5-shot 50.8
- 44 Mixtral 8x7B (47B) 5-shot 50.6
- 45 Gemma3 4B (pretrained) 5-shot 50.6
- 46 Nemotron 3 Nano 30B (3B active) 5-shot 50.5
- 47 Viking 33B 5-shot 50.2
- 48 Apertus 8B (stage 3) 5-shot 49.5
- 49
Llama 3.1-8B 5-shot 49.5
- 50 Tiny Aya 3B 5-shot 49.4
- 51 Apertus 8B (stage 2) 5-shot 49.3
- 52 DeepSeek V3 671B (37B active) 5-shot 49.0
- 53 Gemma4 E2B (5B) (pretrained) 5-shot 48.6
- 54 NorwAI Mistral 7B 5-shot 48.6
- 55 Apertus 8B (stage 1) 5-shot 48.4
- 56 OLMo3 32B 5-shot 48.4
- 57 GPT-SW3 40B 5-shot 48.3
- 58
Qwen3-4B-Base 5-shot 48.0
- 59 Trinity Mini 26B (3B active) 5-shot 47.6
- 60 Ministral3 3B 5-shot 47.0
- 61 Marin 32B 5-shot 46.6
- 62 NorMistral 7B 5-shot 46.5
- 63 GPT-SW3 20B 5-shot 45.3
- 64 Viking 13B 5-shot 45.1
- 65 GPT-SW3 6.7B 5-shot 44.0
- 66
Qwen3.5-2B-Base 5-shot 42.6
- 67 Viking 7B 5-shot 42.1
- 68
Llama 3.2 3B 5-shot 42.1
- 69 Nemotron Nano 12B v2 5-shot 42.1
- 70
Mistral 7B v0.1 5-shot 41.7
- 71 NorMistral 7B (scratch) 5-shot 41.2
- 72 NB-Llama 3.1 8B 5-shot 40.1
- 73 Bielik 4.5B v3 5-shot 39.7
- 74 Qwen3-1.7B-Base 5-shot 39.6
- 75 OLMo2 13B 5-shot 39.4
- 76 Marin 8B 5-shot 39.2
- 77 NB-Llama 3.2 3B 5-shot 38.1
- 78 GPT-SW3 1.3B 5-shot 37.5
- 79 OLMo3 7B 5-shot 36.6
- 80 Poro 34B 5-shot 36.4
- 81 OLMo3 7B (stage 1) 5-shot 36.4
- 82 Trinity Nano 6B (1B active) 5-shot 35.6
- 83 OLMo2 13B (stage 1) 5-shot 35.3
- 84 NB-GPT-J 6B v2 5-shot 35.3
- 85
Qwen3.5-0.8B-Base 5-shot 33.9
- 86 NB-GPT-J 6B 5-shot 32.1
- 87 NorBERT4 1B 5-shot 32.0
- 88 Gemma3 1B (pretrained) 5-shot 31.1
- 89 EuroLLM 1.7B 5-shot 30.3
- 90 DFM 7B 1-shot 29.4
- 91 TildeOpen 30B 1-shot 29.4
- 92 NB-Llama 3.2 1B 5-shot 28.9
- 93 Bielik 1.5B v3 5-shot 28.7
- 94
Llama 3.2 1B 5-shot 27.6
- 95
OLMo 2 1B Instruct 5-shot 20.5
- 96 NorwAI NorGPT 3B 0-shot 20.4
- 97 NorwAI NorLlama 3B 0-shot 8.93
Source: Official NorEval leaderboard
- 68.2
DeepSeek-V4-Flash-Base
- 62.6
Qwen3.5-35B-A3B-Base
- 61.1
Llama 3.1-70B
- 58.3
Apertus 70B
- 58.0
Qwen3.5-9B-Base
- 56.9
Qwen2.5-72B
- 56.3
Ministral 3 14B
- 55.2
Qwen3-14B-Base
- 54.7
Apertus 8B Instruct
- 54.6
Qwen2.5-32B
- 52.4
Qwen3-8B-Base
- 49.5
Llama 3.1-8B
- 48.0
Qwen3-4B-Base
- 42.6
Qwen3.5-2B-Base
- 42.1
Llama 3.2 3B
- 41.7
Mistral 7B v0.1
- 33.9
Qwen3.5-0.8B-Base
- 27.6
Llama 3.2 1B
- 20.5
OLMo 2 1B Instruct
- 1
DeepSeek-V4-Flash-Base 5-shot 68.2
- 2
Qwen3.5-35B-A3B-Base 5-shot 62.6
- 3
Llama 3.1-70B 5-shot 61.1
- 4
Apertus 70B 5-shot 58.3
- 5
Qwen3.5-9B-Base 5-shot 58.0
- 6
Qwen2.5-72B 5-shot 56.9
- 7
Ministral 3 14B 5-shot 56.3
- 8
Qwen3-14B-Base 5-shot 55.2
- 9
Apertus 8B Instruct 5-shot 54.7
- 10
Qwen2.5-32B 5-shot 54.6
- 11
Qwen3-8B-Base 5-shot 52.4
- 12
Llama 3.1-8B 5-shot 49.5
- 13
Qwen3-4B-Base 5-shot 48.0
- 14
Qwen3.5-2B-Base 5-shot 42.6
- 15
Llama 3.2 3B 5-shot 42.1
- 16
Mistral 7B v0.1 5-shot 41.7
- 17
Qwen3.5-0.8B-Base 5-shot 33.9
- 18
Llama 3.2 1B 5-shot 27.6
- 19
OLMo 2 1B Instruct 5-shot 20.5
Source: Official NorEval leaderboard
| # | Entry | Score ± | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 68.2 66.9 to 69.1 | 5-shot | Apr 2026 | Official leaderboard |
| 2 | Gemma4 31B (pretrained) 31B | 67.2 65.9 to 68.2 | 5-shot | Official leaderboard | |
| 3 | Nemotron 4 340B 340B | 64.8 63.4 to 65.8 | 5-shot | Official leaderboard | |
| 4 | Gemma3 27B (pretrained) 27.4B | 64.0 62.5 to 64.9 | 5-shot | Official leaderboard | |
| 5 | Gemma4 26B (4B active) (pretrained) 26.5B | 63.9 62.5 to 64.9 | 5-shot | Official leaderboard | |
| 6 | | 62.6 61.2 to 63.6 | 5-shot | Feb 2026 | Official leaderboard |
| 7 | Llama4 Scout 109B (17B active) 108.7B | 61.4 60 to 62.4 | 5-shot | Official leaderboard | |
| 8 | GLM-4.5 Air (12B active) 110.5B | 61.4 59.9 to 62.4 | 5-shot | Official leaderboard | |
| 9 | | 61.1 59.7 to 62.1 | 5-shot | Jul 2024 | Official leaderboard |
| 10 | TildeOpen 30B 64k 30.7B | 61.1 59.6 to 62.1 | 5-shot | Official leaderboard | |
| 11 | Gemma4 12B (pretrained) 12B | 60.9 59.4 to 61.9 | 5-shot | Official leaderboard | |
| 12 | Gemma3 12B (pretrained) 12.2B | 60.9 59.4 to 61.9 | 5-shot | Official leaderboard | |
| 13 | NorOLMo 13B (stage 2, new mix) 13.7B | 59.6 58.1 to 60.6 | 5-shot | Official leaderboard | |
| 14 | NorwAI Mixtral 8x7B (47B) 47B | 59.4 57.9 to 60.5 | 5-shot | Official leaderboard | |
| 15 | Mistral 24B 23.6B | 59.2 57.7 to 60.2 | 5-shot | Official leaderboard | |
| 16 | GLM-4 32B 32.6B | 58.5 57 to 59.5 | 5-shot | Official leaderboard | |
| 17 | | 58.3 56.8 to 59.4 | 5-shot | Sep 2025 | Official leaderboard |
| 18 | Gemma4 E4B (8B) (pretrained) 8B | 58.2 56.7 to 59.3 | 5-shot | Official leaderboard | |
| 19 | | 58.0 56.5 to 59 | 5-shot | Feb 2026 | Official leaderboard |
| 20 | Mistral 24B (v2501) 23.6B | 57.9 56.4 to 58.9 | 5-shot | Official leaderboard | |
| 21 | NorMistral 11B Long 11.4B | 57.8 56.2 to 58.8 | 5-shot | Official leaderboard | |
| 22 | Prelude 9B 9.1B | 57.7 56.2 to 58.7 | 5-shot | Official leaderboard | |
| 23 | NorMistral 11B 11.4B | 57.4 55.9 to 58.4 | 5-shot | Official leaderboard | |
| 24 | Nemotron 3 Super 120B (12B active) 123.6B | 57.3 56 to 58.1 | 5-shot | Official leaderboard | |
| 25 | | 56.9 55.5 to 58 | 5-shot | Sep 2024 | Official leaderboard |
| 26 | | 56.3 54.8 to 57.3 | 5-shot | Dec 2025 | Official leaderboard |
| 27 | Mixtral 8x22B (141B) 140.6B | 56.1 54.6 to 57.1 | 5-shot | Official leaderboard | |
| 28 | NorOLMo 13B 13.7B | 55.7 54.2 to 56.9 | 5-shot | Official leaderboard | |
| 29 | EuroLLM 1.5 9B 9.2B | 55.5 54 to 56.6 | 5-shot | Official leaderboard | |
| 30 | NorOLMo 13B (stage 2) 13.7B | 55.4 53.9 to 56.6 | 5-shot | Official leaderboard | |
| 31 | Bielik 11B v3 11.2B | 55.4 53.8 to 56.5 | 5-shot | Official leaderboard | |
| 32 | | 55.2 53.8 to 56.3 | 5-shot | Apr 2025 | Official leaderboard |
| 33 | | 54.7 53.2 to 55.9 | 5-shot | Sep 2025 | Official leaderboard |
| 34 | EuroLLM 22B 22.6B | 54.7 53.2 to 55.8 | 5-shot | Official leaderboard | |
| 35 | | 54.6 53.2 to 55.7 | 5-shot | Sep 2024 | Official leaderboard |
| 36 | Apertus 8B (stage 5) 8.1B | 54.4 52.9 to 55.5 | 5-shot | Official leaderboard | |
| 37 | AI Sweden Llama3 8B 8B | 53.6 52.1 to 54.7 | 5-shot | Official leaderboard | |
| 38 | Ministral3 8B 8.4B | 53.2 51.8 to 54.2 | 5-shot | Official leaderboard | |
| 39 | Mistral Nemo 12B 12.2B | 52.5 51 to 53.6 | 5-shot | Official leaderboard | |
| 40 | | 52.4 50.9 to 53.5 | 5-shot | Apr 2025 | Official leaderboard |
| 41 | NorOLMo 13B (stage 1) 13.7B | 51.7 50.1 to 52.8 | 5-shot | Official leaderboard | |
| 42 | Apertus 8B (stage 4) 8.1B | 51.5 50 to 52.6 | 5-shot | Official leaderboard | |
| 43 | EuroLLM 9B 9.2B | 50.8 49.3 to 52 | 5-shot | Official leaderboard | |
| 44 | Mixtral 8x7B (47B) 46.7B | 50.6 49.1 to 51.7 | 5-shot | Official leaderboard | |
| 45 | Gemma3 4B (pretrained) 4.3B | 50.6 49 to 51.7 | 5-shot | Official leaderboard | |
| 46 | Nemotron 3 Nano 30B (3B active) 31.6B | 50.5 49.1 to 51.6 | 5-shot | Official leaderboard | |
| 47 | Viking 33B 33.1B | 50.2 48.7 to 51.3 | 5-shot | Official leaderboard | |
| 48 | Apertus 8B (stage 3) 8.1B | 49.5 48 to 50.7 | 5-shot | Official leaderboard | |
| 49 | | 49.5 48 to 50.7 | 5-shot | Jul 2024 | Official leaderboard |
| 50 | Tiny Aya 3B 3.3B | 49.4 47.9 to 50.4 | 5-shot | Official leaderboard | |
| 51 | Apertus 8B (stage 2) 8.1B | 49.3 47.7 to 50.4 | 5-shot | Official leaderboard | |
| 52 | DeepSeek V3 671B (37B active) 671B | 49.0 47.5 to 50 | 5-shot | Official leaderboard | |
| 53 | Gemma4 E2B (5B) (pretrained) 5.1B | 48.6 47.1 to 49.8 | 5-shot | Official leaderboard | |
| 54 | NorwAI Mistral 7B 7.5B | 48.6 47.3 to 49.6 | 5-shot | Official leaderboard | |
| 55 | Apertus 8B (stage 1) 8.1B | 48.4 46.9 to 49.6 | 5-shot | Official leaderboard | |
| 56 | OLMo3 32B 32.3B | 48.4 46.9 to 49.5 | 5-shot | Official leaderboard | |
| 57 | GPT-SW3 40B 40B | 48.3 46.8 to 49.4 | 5-shot | Official leaderboard | |
| 58 | | 48.0 46.5 to 49.2 | 5-shot | Apr 2025 | Official leaderboard |
| 59 | Trinity Mini 26B (3B active) 26.1B | 47.6 46 to 48.7 | 5-shot | Official leaderboard | |
| 60 | Ministral3 3B 3.4B | 47.0 45.5 to 48.1 | 5-shot | Official leaderboard | |
| 61 | Marin 32B 32.5B | 46.6 45.1 to 47.7 | 5-shot | Official leaderboard | |
| 62 | NorMistral 7B 7.2B | 46.5 44.9 to 47.7 | 5-shot | Official leaderboard | |
| 63 | GPT-SW3 20B 20.8B | 45.3 43.9 to 46.4 | 5-shot | Official leaderboard | |
| 64 | Viking 13B 14B | 45.1 43.6 to 46.3 | 5-shot | Official leaderboard | |
| 65 | GPT-SW3 6.7B 6.7B | 44.0 42.5 to 45.2 | 5-shot | Official leaderboard | |
| 66 | | 42.6 41 to 43.8 | 5-shot | Feb 2026 | Official leaderboard |
| 67 | Viking 7B 7.6B | 42.1 40.6 to 43.3 | 5-shot | Official leaderboard | |
| 68 | | 42.1 40.5 to 43.3 | 5-shot | Sep 2024 | Official leaderboard |
| 69 | Nemotron Nano 12B v2 12.3B | 42.1 40.5 to 43.3 | 5-shot | Official leaderboard | |
| 70 | | 41.7 40.2 to 42.8 | 5-shot | Sep 2023 | Official leaderboard |
| 71 | NorMistral 7B (scratch) 7B | 41.2 39.8 to 42.3 | 5-shot | Official leaderboard | |
| 72 | NB-Llama 3.1 8B 8B | 40.1 38.7 to 41.1 | 5-shot | Official leaderboard | |
| 73 | Bielik 4.5B v3 4.8B | 39.7 38.2 to 40.8 | 5-shot | Official leaderboard | |
| 74 | Qwen3-1.7B-Base 1.7B | 39.6 38 to 40.7 | 5-shot | Official leaderboard | |
| 75 | OLMo2 13B 13.7B | 39.4 37.9 to 40.6 | 5-shot | Official leaderboard | |
| 76 | Marin 8B 8B | 39.2 37.7 to 40.4 | 5-shot | Official leaderboard | |
| 77 | NB-Llama 3.2 3B 3.2B | 38.1 36.7 to 39.3 | 5-shot | Official leaderboard | |
| 78 | GPT-SW3 1.3B 1.4B | 37.5 36.1 to 38.6 | 5-shot | Official leaderboard | |
| 79 | OLMo3 7B 7.3B | 36.6 35.1 to 37.8 | 5-shot | Official leaderboard | |
| 80 | Poro 34B 34.2B | 36.4 34.9 to 37.6 | 5-shot | Official leaderboard | |
| 81 | OLMo3 7B (stage 1) 7.3B | 36.4 34.8 to 37.6 | 5-shot | Official leaderboard | |
| 82 | Trinity Nano 6B (1B active) 6.1B | 35.6 34.1 to 36.7 | 5-shot | Official leaderboard | |
| 83 | OLMo2 13B (stage 1) 13.7B | 35.3 33.8 to 36.4 | 5-shot | Official leaderboard | |
| 84 | NB-GPT-J 6B v2 6.1B | 35.3 33.9 to 36.3 | 5-shot | Official leaderboard | |
| 85 | | 33.9 32.3 to 35.1 | 5-shot | Feb 2026 | Official leaderboard |
| 86 | NB-GPT-J 6B 6.1B | 32.1 30.8 to 33.1 | 5-shot | Official leaderboard | |
| 87 | NorBERT4 1B 1B | 32.0 30.6 to 33.1 | 5-shot | Official leaderboard | |
| 88 | Gemma3 1B (pretrained) 1B | 31.1 29.6 to 32.2 | 5-shot | Official leaderboard | |
| 89 | EuroLLM 1.7B 1.7B | 30.3 28.8 to 31.4 | 5-shot | Official leaderboard | |
| 90 | DFM 7B 7B | 29.4 28 to 30.5 | 1-shot | Official leaderboard | |
| 91 | TildeOpen 30B 30.7B | 29.4 28.1 to 30.4 | 1-shot | Official leaderboard | |
| 92 | NB-Llama 3.2 1B 1.2B | 28.9 27.7 to 29.8 | 5-shot | Official leaderboard | |
| 93 | Bielik 1.5B v3 1.6B | 28.7 27.2 to 29.9 | 5-shot | Official leaderboard | |
| 94 | | 27.6 26.1 to 28.8 | 5-shot | Sep 2024 | Official leaderboard |
| 95 | | 20.5 19.1 to 21.6 | 5-shot | Apr 2025 | Official leaderboard |
| 96 | NorwAI NorGPT 3B 3.1B | 20.4 19.3 to 21.4 | 0-shot | Official leaderboard | |
| 97 | NorwAI NorLlama 3B 2.8B | 8.93 7.9 to 9.8 | 0-shot | Official leaderboard |
| # | Model | Score ± | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 68.2 66.9 to 69.1 | 5-shot | Apr 2026 | Official leaderboard |
| 2 | | 62.6 61.2 to 63.6 | 5-shot | Feb 2026 | Official leaderboard |
| 3 | | 61.1 59.7 to 62.1 | 5-shot | Jul 2024 | Official leaderboard |
| 4 | | 58.3 56.8 to 59.4 | 5-shot | Sep 2025 | Official leaderboard |
| 5 | | 58.0 56.5 to 59 | 5-shot | Feb 2026 | Official leaderboard |
| 6 | | 56.9 55.5 to 58 | 5-shot | Sep 2024 | Official leaderboard |
| 7 | | 56.3 54.8 to 57.3 | 5-shot | Dec 2025 | Official leaderboard |
| 8 | | 55.2 53.8 to 56.3 | 5-shot | Apr 2025 | Official leaderboard |
| 9 | | 54.7 53.2 to 55.9 | 5-shot | Sep 2025 | Official leaderboard |
| 10 | | 54.6 53.2 to 55.7 | 5-shot | Sep 2024 | Official leaderboard |
| 11 | | 52.4 50.9 to 53.5 | 5-shot | Apr 2025 | Official leaderboard |
| 12 | | 49.5 48 to 50.7 | 5-shot | Jul 2024 | Official leaderboard |
| 13 | | 48.0 46.5 to 49.2 | 5-shot | Apr 2025 | Official leaderboard |
| 14 | | 42.6 41 to 43.8 | 5-shot | Feb 2026 | Official leaderboard |
| 15 | | 42.1 40.5 to 43.3 | 5-shot | Sep 2024 | Official leaderboard |
| 16 | | 41.7 40.2 to 42.8 | 5-shot | Sep 2023 | Official leaderboard |
| 17 | | 33.9 32.3 to 35.1 | 5-shot | Feb 2026 | Official leaderboard |
| 18 | | 27.6 26.1 to 28.8 | 5-shot | Sep 2024 | Official leaderboard |
| 19 | | 20.5 19.1 to 21.6 | 5-shot | Apr 2025 | Official leaderboard |
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: Official NorEval leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| 5-shot | 68.2 | ||||
| 1-shot | 63.3 | ||||
| 5-shot | 62.6 | ||||
| 5-shot | 61.1 | ||||
| 5-shot | 58.3 | ||||
| 5-shot | 58.0 | ||||
| 5-shot | 56.9 | ||||
| 1-shot | 56.8 | ||||
| 5-shot | 56.3 | ||||
| 1-shot | 55.9 | ||||
| 5-shot | 55.2 | ||||
| 5-shot | 54.7 | ||||
| 5-shot | 54.6 | ||||
| 1-shot | 53.0 | ||||
| 0-shot | 52.8 | ||||
| 5-shot | 52.4 | ||||
| 1-shot | 51.5 | ||||
| 1-shot | 51.3 | ||||
| 1-shot | 50.3 | ||||
| 1-shot | 50.2 | ||||
| 5-shot | 49.5 | ||||
| 1-shot | 49.4 | ||||
| 0-shot | 49.0 | ||||
| 0-shot | 48.9 | ||||
| 1-shot | 48.6 | ||||
| 0-shot | 48.1 | ||||
| 5-shot | 48.0 | ||||
| 1-shot | 47.1 | ||||
| 0-shot | 45.2 | ||||
| 0-shot | 44.7 | ||||
| 1-shot | 43.8 | ||||
| 5-shot | 42.6 | ||||
| 1-shot | 42.6 | ||||
| 5-shot | 42.1 | ||||
| 0-shot | 41.9 | ||||
| 5-shot | 41.7 | ||||
| 0-shot | 41.4 | ||||
| 0-shot | 41.0 | ||||
| 0-shot | 40.9 | ||||
| 0-shot | 38.9 | ||||
| 0-shot | 38.2 | ||||
| 1-shot | 36.3 | ||||
| 1-shot | 36.0 | ||||
| 1-shot | 34.9 | ||||
| 5-shot | 33.9 | ||||
| 0-shot | 33.3 | ||||
| 0-shot | 27.8 | ||||
| 5-shot | 27.6 | ||||
| 1-shot | 26.5 | ||||
| 0-shot | 25.8 | ||||
| 0-shot | 25.5 | ||||
| 1-shot | 23.4 | ||||
| 5-shot | 20.5 | ||||
| 0-shot | 19.5 | ||||
| 0-shot | 17.2 | ||||
| 1-shot | 15.8 | ||||
| 0-shot | 12.3 |
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: Official NorEval leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| 5-shot | 68.2 | ||||
| 1-shot | 63.3 | ||||
| 5-shot | 62.6 | ||||
| 5-shot | 61.1 | ||||
| 5-shot | 58.3 | ||||
| 5-shot | 58.0 | ||||
| 5-shot | 56.9 | ||||
| 1-shot | 56.8 | ||||
| 5-shot | 56.3 | ||||
| 1-shot | 55.9 | ||||
| 5-shot | 55.2 | ||||
| 5-shot | 54.7 | ||||
| 5-shot | 54.6 | ||||
| 1-shot | 53.0 | ||||
| 0-shot | 52.8 | ||||
| 5-shot | 52.4 | ||||
| 1-shot | 51.5 | ||||
| 1-shot | 51.3 | ||||
| 1-shot | 50.3 | ||||
| 1-shot | 50.2 | ||||
| 5-shot | 49.5 | ||||
| 1-shot | 49.4 | ||||
| 0-shot | 49.0 | ||||
| 0-shot | 48.9 | ||||
| 1-shot | 48.6 | ||||
| 0-shot | 48.1 | ||||
| 5-shot | 48.0 | ||||
| 1-shot | 47.1 | ||||
| 0-shot | 45.2 | ||||
| 0-shot | 44.7 | ||||
| 1-shot | 43.8 | ||||
| 5-shot | 42.6 | ||||
| 1-shot | 42.6 | ||||
| 5-shot | 42.1 | ||||
| 0-shot | 41.9 | ||||
| 5-shot | 41.7 | ||||
| 0-shot | 41.4 | ||||
| 0-shot | 41.0 | ||||
| 0-shot | 40.9 | ||||
| 0-shot | 38.9 | ||||
| 0-shot | 38.2 | ||||
| 1-shot | 36.3 | ||||
| 1-shot | 36.0 | ||||
| 1-shot | 34.9 | ||||
| 5-shot | 33.9 | ||||
| 0-shot | 33.3 | ||||
| 0-shot | 27.8 | ||||
| 5-shot | 27.6 | ||||
| 1-shot | 26.5 | ||||
| 0-shot | 25.8 | ||||
| 0-shot | 25.5 | ||||
| 1-shot | 23.4 | ||||
| 5-shot | 20.5 | ||||
| 0-shot | 19.5 | ||||
| 0-shot | 17.2 | ||||
| 1-shot | 15.8 | ||||
| 0-shot | 12.3 |
3 models have held number one on the official leaderboard since its release in April 2025
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official NorEval leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official NorEval leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5%
- MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87%
- CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7%
- Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
- MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%
Data audit
8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash-Base. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (97 entries).
- Pass Models scored 19 models we track, plus 78 other entries on its leaderboard.
- Warning Newest score The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash-Base.
- Note Frontier models None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166).
- Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on ltgoslo.github.io: 57 rows for models we track and 78 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.