BLiMP-NL
Models compare Dutch sentence pairs that differ in grammatical acceptability.
EuroEval compares the labels with human or constructed reference judgments using the Matthews correlation coefficient.
- Made by
- Suijkerbuijk and colleagues
- Released
- July 2025
- Kind
- Reasoning in many languages
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- All public
- Measures
- Capability
- Subject
- Languages and literature
- Language
- Dutch
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
- Based on
- BLiMP
- Strongest model tested
- gpt-oss-20b #143 on the capability index
- Last model added
- December 2025
- Scores refreshed
- November 21st, 2026
gemini/gemini-3.7-flash (zero-shot, val) tops the leaderboard with 64.8% Gemma 3 27B leads with 49.4% among models released by December 2025
Best score per entry from EuroEval’s leaderboard, read on November 21st, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Gemma 3 27B, at 49.4%. This view shows only models we track, each at its best.
The newest model with a score here came out in December 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 64.8 gemini/gemini-3.7-flash (zero-shot, val)
- 61.4 gemini/gemini-3.6-flash (zero-shot, val)
- 60.5 gpt-6.2-astra (zero-shot, val)
- 60.3 gpt-6.2-sol (zero-shot, val)
- 59.7 gemini/gemini-3.5-flash-lite (zero-shot, val)
- 58.6 mistralai/Mistral-Large-3-675B-Instruct-2512 (val)
- 57.8 openai/gpt-5.6-luna (zero-shot, val)
- 57.4 gemini/gemini-3-flash-preview#thinking (zero-shot, val)
- 57.3 NbAiLab/borealis2-31b-preview (val)
- 57.3 claude-sonnet-4-5-20250929#no-thinking (zero-shot, val)
- 56.4 gpt-4.1-2025-04-14 (zero-shot, val)
- 56.2 deepseek/deepseek-flash#no-thinking (val)
- 56.1 gpt-6.1-sol (zero-shot, val)
- 54.7 utter-project/EuroLLM-22B-Instruct-2512
- 54.6 zai-org/GLM-5.3-Flash (val)
- 54.3 gpt-6-astra (zero-shot, val)
- 54.2 gpt-6-sol (zero-shot, val)
- 54.1 gpt-5-mini-2025-08-07 (zero-shot, val)
- 54.0 gemini/gemini-3.1-flash-lite-preview (zero-shot, val)
- 53.7 mistralai/Mistral-Medium-3.5-128B (val)
- 53.0 google/gemma-4-31B-it (val)
- 52.9 claude-sonnet-4-5-20250929#thinking (zero-shot, val)
- 52.6 xai/grok-4-1-fast-reasoning (zero-shot, val)
- 52.2 mistralai/Mistral-Small-4-119B-2603 (val)
- 51.6 claude-sonnet-4-6 (zero-shot, val)
- 51.6 openai/gpt-5.4-mini-2026-03-17#none (zero-shot, val)
- 51.3 openai/gpt-5.4-mini-2026-03-17#low (zero-shot, val)
- 51.0 Qwen/Qwen3.6-27B-FP8 (val)
- 50.8 deepcogito/cogito-v1-preview-qwen-14B
- 50.6 mistralai/Mistral-Medium-3.5-128B (zero-shot, val)
- 50.6 openai/gpt-5.4-2026-03-05#none (zero-shot, val)
- 49.8 NbAiLab/borealis2-26b-a4b-preview (val)
- 49.7 speakleash/Bielik-11B-v3.1-Instruct (val)
- 49.6 mistralai/Ministral-3-14B-Base-2512 (val)
- 49.6 deepcogito/cogito-v2-preview-llama-70B
- 49.5 mistralai/Ministral-3-14B-Reasoning-2512 (val)
- 49.4
Gemma 3 27B
- 49.1 google/gemma-4-26B-A4B-it (val)
- 49.0 swiss-ai/Apertus-v1.5-70B (val)
- 48.9 Qwen/Qwen3.5-35B-A3B-FP8 (val)
- 48.8 gpt-6-luna (zero-shot, val)
- 48.7 NbAiLab/borealis2-26b-a4b-preview (zero-shot, val)
- 48.7 zai-org/GLM-5.3-Flash (zero-shot, val)
- 48.5
Gemma 3 12B PT
- 48.2
Gemma 3 27B PT
- 48.0 Qwen/Qwen3-14B#no-thinking
- 48.0 Qwen/Qwen3-32B#no-thinking
- 47.9 NbAiLab/borealis2-31b-preview (zero-shot, val)
- 47.7 gpt-5.6-sol (zero-shot, val)
- 47.5 deepcogito/cogito-v1-preview-qwen-32B
- 47.3 gemini/gemini-2.5-flash-lite#no-thinking (zero-shot, val)
- 46.6 amalia-llm/AMALIA-9B-0626-DPO (val)
- 46.4 deepseek/deepseek-flash#low (zero-shot, val)
- 46.3 gpt-5.2-2025-12-11 (zero-shot, val)
- 45.8
Gemma 3 12B
- 45.2 claude-haiku-4-5-20251001 (zero-shot, val)
- 45.1 gemini/gemini-2.5-flash-lite#thinking (zero-shot, val)
- 43.9 mistralai/Mistral-Small-3.1-24B-Base-2503
- 43.9 swiss-ai/Apertus-v1.5-8B (val)
- 43.7 openai/gpt-oss-120b (val)
- 43.1 speakleash/Bielik-11B-v3.0-Instruct (val)
- 43.1 openeurollm/prelude-long (val)
- 42.6 deepcogito/cogito-v1-preview-llama-8B
- 42.5 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (val)
- 42.5 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 (val)
- 42.2 latam-gpt/Llama-3.1-70B-LatamGPT-SFT-1.0 (val)
- 42.1 Qwen/Qwen3-8B#no-thinking
- 42.0 Qwen/Qwen3.5-9B (val)
- 41.9 gpt-5-nano-2025-08-07 (zero-shot, val)
- 41.6 openai/gpt-5.4-mini-2026-03-17#medium (zero-shot, val)
- 41.6 openai/gpt-5.4-nano-2026-03-17#medium (zero-shot, val)
- 41.3 allenai/Llama-3.1-Tulu-3-8B-SFT
- 41.3 google/gemma-4-12B-it (val)
- 41.2 mistralai/Mistral-Small-3.2-24B-Instruct-2506
- 40.9 openeurollm/prelude-base (val)
- 40.7 Qwen/Qwen3.8-27B (val)
- 40.2 gpt-5.6-terra (zero-shot, val)
- 39.9 speakleash/Bielik-Minitron-7B-v3.0-Instruct (val)
- 39.8 openai/gpt-5.4-nano-2026-03-17#low (zero-shot, val)
- 39.8 Qwen/Qwen3-4B#no-thinking
- 39.5 google/gemma-4-E4B-it (val)
- 39.3 mistralai/Mistral-Large-3-675B-Instruct-2512 (zero-shot, val)
- 39.1 speakleash/Bielik-One-11B-v3.1-r1@75a2d83890e3c68bf7fac2fe5bbe2bff6372036c (zero-shot)
- 38.9 nvidia/Nemotron-Cascade-2-30B-A3B (val)
- 38.7 zai-org/GLM-4.7-Flash (val)
- 38.6 swiss-ai/Apertus-v1.5-70B (zero-shot, val)
- 38.1 domyn/Domyn-Small-v1.0 (val)
- 38.0 mistralai/Ministral-3-8B-Reasoning-2512
- 38.0
Llama 3.1-70B
- 37.8
Gemma 3 4B
- 37.6 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 (val)
- 37.5 microsoft/mdeberta-v3-base
- 37.1 meta-llama/Llama-3.1-8B-Instruct
- 37.1 amalia-llm/AMALIA-9B-0626-SFT (val)
- 37.0 openai/gpt-5.4-nano-2026-03-17#high (zero-shot, val)
- 37.0 mistralai/Mistral-Small-3.1-24B-Instruct-2503
- 36.6 mistralai/Mistral-Small-4-119B-2603 (zero-shot, val)
- 36.3 gpt-oss-20b
- 36.3 Qwen/Qwen3.5-9B-Base (val)
- 36.3 mistralai/Ministral-3-14B-Instruct-2512 (val)
- 36.1 openai/gpt-5.4-mini-2026-03-17#high (zero-shot, val)
- 35.9 OpenLLM-France/Luciole-23B-Instruct-1.1 (val)
- 35.9 Qwen/Qwen3.5-35B-A3B-Base (val)
- 35.2 speakleash/Bielik-11B-v2.3-Instruct
- 35.2 NbAiLab/borealis2-e4b-preview (val)
- 34.7 CohereLabs/aya-expanse-8b (val)
- 33.9 utter-project/EuroLLM-9B-Instruct
- 33.6
Qwen3-4B-Thinking-2507
- 33.1 utter-project/EuroLLM-9B-Instruct-2512 (val)
- 32.4 OpenLLM-France/Luciole-8B-Instruct-1.1 (val)
- 32.2 google/gemma-4-31B (val)
- 31.6 Qwen/Qwen3.5-2B (val)
- 31.5 granite-4.0-micro
- 31.1 NbAiLab/borealis2-e2b-preview (val)
- 31.1
Gemma 3 4B PT
- 31.0 mii-llm/nesso-4B
- 30.7 allenai/Olmo-3-1125-32B
- 30.4 danish-foundation-models/DFM-Mimir-v1.8 (val)
- 29.8 Aleph-Alpha/Kolibri-1 (val)
- 29.8 google/gemma-4-E2B-it (val)
- 29.8 typesafe/jev-1.13.0 (zero-shot)
- 29.4 mistralai/Ministral-3-3B-Instruct-2512 (val)
- 29.0 deepcogito/cogito-v1-preview-llama-3B
- 28.8 allenai/Olmo-3.1-32B-Instruct-DPO
- 28.8 allenai/Olmo-3.1-32B-Instruct-SFT
- 28.5 danish-foundation-models/DFM-Mimir-v1.8 (zero-shot, val)
- 27.5
Olmo 3.1 32B Instruct
- 27.2 ibm-granite/granite-4.0-h-micro
- 27.1 Qwen/Qwen3.5-4B (val)
- 26.9
Ministral 3 3B Reasoning
- 26.8
Llama 3.1-8B
- 26.6 danish-foundation-models/DFM-Mimir-v1.5 (val)
- 25.9 danish-foundation-models/DFM-Mimir-v1.5 (zero-shot, val)
- 25.8 BSC-LT/salamandra-7b-instruct
- 25.5 amalia-llm/AMALIA-9B-0626-DPO (zero-shot, val)
- 24.8 amalia-llm/AMALIA-9B-0626-SFT (zero-shot, val)
- 24.6 swiss-ai/Apertus-v1.1-4B (val)
- 23.8 google/gemma-4-26B-A4B (val)
- 22.8 Qwen/Qwen3.5-2B-Base (val)
- 22.6 meta-llama/Llama-3.2-3B-Instruct
- 22.5 inclusionAI/Ling-mini-2.0 (val)
- 22.3 OpenLLM-France/Luciole-23B-Base (val)
- 22.0 google/gemma-4-E4B (val)
- 21.4 google/gemma-4-12B (val)
- 21.3 mistralai/Ministral-3-8B-Instruct-2512 (val)
- 21.2 Qwen/Qwen3.5-4B-Base (val)
- 20.5 LGAI-EXAONE/EXAONE-4.0-32B (val)
- 20.3 Qwen/Qwen3-1.7B#no-thinking
- 20.1 utter-project/EuroLLM-22B-2512
- 19.9 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
- 19.9 HuggingFaceTB/SmolLM3-3B-Base
- 19.8 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 (val)
- 19.6 danish-foundation-models/DFM-Mimir (val)
- 19.3 Qwen/Qwen3.5-0.8B-Base (val)
- 19.0 openai/gpt-5.4-nano-2026-03-17#none (zero-shot, val)
- 18.5 jhu-clsp/mmBERT-small
- 18.4 jhu-clsp/mmBERT-base
- 18.1 swiss-ai/Apertus-v1.5-8B (zero-shot, val)
- 17.6 allenai/Olmo-3-1025-7B
- 17.3 speakleash/Bielik-One-4B-r1@567d29b0f15df28794acf0f6187f9216860d4551 (zero-shot)
- 17.1 danish-foundation-models/DFM-Mimir-v1.8-Mini (val)
- 16.7 JetBrains/Mellum2-12B-A2.5B-Instruct (val)
- 16.2 openGPT-X/Teuken-7B-instruct-v0.6 (val)
- 15.7 HuggingFaceTB/SmolLM3-3B#no-thinking
- 15.7 mistralai/Ministral-3-8B-Base-2512 (val)
- 15.5 occiglot/occiglot-7b-eu5-instruct
- 15.2 JetBrains/Mellum2-12B-A2.5B-Base (val)
- 14.9 Qwen/Qwen3.5-0.8B (val)
- 14.9 swiss-ai/Apertus-v1.1-4B-Instruct (val)
- 14.8 danish-foundation-models/DFM-Mimir-v1.8-Mini (zero-shot, val)
- 14.3 allenai/Olmo-3-7B-Instruct
- 13.9 openeurollm/oellm-9b-128k-theta32m-v3 (val)
- 13.6 allenai/Olmo-3-7B-Instruct-DPO
- 13.6 danish-foundation-models/DFM-Mimir (zero-shot, val)
- 12.6 LiquidAI/LFM2.5-8B-A1B (val)
- 12.4 intfloat/multilingual-e5-base
- 11.8 gemini/gemini-3-flash-preview#no-thinking (zero-shot, val)
- 11.4 intfloat/multilingual-e5-large
- 11.3 allenai/Olmo-3-7B-Instruct-SFT
- 11.3 EuroBERT/EuroBERT-210m
- 11.3 LiquidAI/LFM2.5-8B-A1B-Base (val)
- 11.2 answerdotai/ModernBERT-large
- 11.1 EuroBERT/EuroBERT-610m
- 9.84 CohereLabs/aya-23-8B (val)
- 9.65 Aleph-Alpha/Kolibri-1 (zero-shot, val)
- 9.52 speakleash/Bielik-1.5B-v3.0-Instruct
- 9.41 utter-project/EuroLLM-9B
- 8.80 TrustLLMeu/trustllm-v2-8b-midtrain@3c7af30bc43bb3c8607e0fab6e7f2a23dd64af49
- 7.68 distilbert/distilbert-base-multilingual-cased
- 7.56 Geotrend/distilbert-base-25lang-cased
- 7.55 google/gemma-4-E2B (val)
- 7.00
Qwen3-0.6B
- 6.66 swiss-ai/Apertus-v1.1-0.5B (val)
- 6.56 answerdotai/ModernBERT-base
- 6.47 AI-Sweden-Models/bert-large-nordic-pile-1M-steps
- 6.40 swiss-ai/Apertus-v1.1-1.5B (val)
- 6.02 AI-Sweden-Models/Llama-3-8B-instruct (val)
- 5.99 OmniJev/OneJev1.6-0.8B (zero-shot)
- 5.92 fastino/GLiNER2.5-Decide (zero-shot)
- 5.90
Gemma 3 1B PT
- 5.74 EuropeanParliament/EUBERT
- 5.50 OpenLLM-France/Luciole-8B-Base (val)
- 5.00 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f (zero-shot)
- 4.90 TrustLLMeu/trustllm-v2-8b@d8cc924fa550fbf9474f5251e8e2f92d1c1bf894
- 4.70 fresh-electra-small
- 4.62 fresh-xlm-roberta-base
- 4.61 flwrlabs/Lizzy-7B (val)
- 4.37 NbAiLab/borealis2-e4b-preview (zero-shot, val)
- 4.30 syvai/claes-v1 (zero-shot)
- 4.00 convaiinnovations/laya (zero-shot)
- 3.89 swiss-ai/Apertus-v1.1-0.5B-Instruct (val)
- 3.88 occiglot/occiglot-7b-eu5
- 3.72 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f
- 3.60 Qwen/Qwen3-0.6B#no-thinking
- 3.56 OpenLLM-France/Luciole-1B-Instruct-1.1 (val)
- 3.52 LFM2-1.2B
- 3.37 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base
- 3.34 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 (zero-shot)
- 3.09 JetBrains/Mellum2-12B-A2.5B-Thinking-SFT (val)
- 2.98 PleIAs/Pleias-350m-Preview
- 2.90 yulan-team/YuLan-Mini-Instruct
- 2.78 meta-llama/Llama-3.2-1B-Instruct
- 2.62 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6
- 2.52
Gemma 3 1B
- 2.36
Gemma 3 270M
- 2.35
Llama 3.2 1B
- 2.26 OpenLLM-France/Luciole-1B-Base (val)
- 1.95 BSC-LT/salamandra-7b
- 1.91 PleIAs/Pleias-1.2b-Preview
- 1.82 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 (zero-shot)
- 1.76 syvai/claes-preview (zero-shot)
- 1.62 JetBrains/Mellum2-12B-A2.5B-Thinking (val)
- 1.34 AIDC-AI/Marco-Mini-Global-Base (val)
- 1.23 PleIAs/Monad
- 1.08 LiquidAI/LFM2-350M
- 0.92 JetBrains/Mellum2-12B-A2.5B-Instruct-SFT (val)
- 0.70
SmolLM2-135M
- 0.56 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660
- 0.46
SmolLM2-360M
- 0.40 HuggingFaceTB/SmolLM2-135M-Instruct
- 0.32 convaiinnovations/laya-multilingual (zero-shot)
- 0.23 google-bert/bert-base-multilingual-cased (val)
- 0.23 NbAiLab/borealis2-e2b-preview (zero-shot, val)
- 0.19 LiquidAI/LFM2-700M
- 0.15 swiss-ai/Apertus-v1.1-1.5B (zero-shot)
- 0.14 jaredpalmer/kev-0.5b (zero-shot)
- 0.06 HuggingFaceTB/SmolLM2-360M-Instruct
- 0.06 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct
- 0.0 AI-Sweden-Models/Llama-3-8B-instruct (zero-shot, val)
- 0.0 JetBrains/Mellum2-12B-A2.5B-Base-Pretrain (val)
- 0.0 mistralai/Ministral-3-14B-Instruct-2512 (zero-shot, val)
- 0.0 mistralai/Ministral-3-3B-Instruct-2512 (zero-shot, val)
- 0.0 Muse-research/Muse-3B (val)
- 0.0 openGPT-X/Teuken-7B-base-v0.6 (val)
- 0.0 PleIAs/Baguettotron@refs/pr/6#no-thinking
- 0.0 swiss-ai/Apertus-v1.1-1.5B-Instruct (val)
- 0.0 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview
- 1 gemini/gemini-3.7-flash (zero-shot, val) Dutch CoLA test split 64.8%
- 2 gemini/gemini-3.6-flash (zero-shot, val) Dutch CoLA test split 61.4%
- 3 gpt-6.2-astra (zero-shot, val) Dutch CoLA test split 60.5%
- 4 gpt-6.2-sol (zero-shot, val) Dutch CoLA test split 60.3%
- 5 gemini/gemini-3.5-flash-lite (zero-shot, val) Dutch CoLA test split 59.7%
- 6 mistralai/Mistral-Large-3-675B-Instruct-2512 (val) Dutch CoLA test split 58.6%
- 7 openai/gpt-5.6-luna (zero-shot, val) Dutch CoLA test split 57.8%
- 8 gemini/gemini-3-flash-preview#thinking (zero-shot, val) Dutch CoLA test split 57.4%
- 9 NbAiLab/borealis2-31b-preview (val) Dutch CoLA test split 57.3%
- 10 claude-sonnet-4-5-20250929#no-thinking (zero-shot, val) Dutch CoLA test split 57.3%
- 11 gpt-4.1-2025-04-14 (zero-shot, val) Dutch CoLA test split 56.4%
- 12 deepseek/deepseek-flash#no-thinking (val) Dutch CoLA test split 56.2%
- 13 gpt-6.1-sol (zero-shot, val) Dutch CoLA test split 56.1%
- 14 utter-project/EuroLLM-22B-Instruct-2512 Dutch CoLA test split 54.7%
- 15 zai-org/GLM-5.3-Flash (val) Dutch CoLA test split 54.6%
- 16 gpt-6-astra (zero-shot, val) Dutch CoLA test split 54.3%
- 17 gpt-6-sol (zero-shot, val) Dutch CoLA test split 54.2%
- 18 gpt-5-mini-2025-08-07 (zero-shot, val) Dutch CoLA test split 54.1%
- 19 gemini/gemini-3.1-flash-lite-preview (zero-shot, val) Dutch CoLA test split 54%
- 20 mistralai/Mistral-Medium-3.5-128B (val) Dutch CoLA test split 53.7%
- 21 google/gemma-4-31B-it (val) Dutch CoLA test split 53%
- 22 claude-sonnet-4-5-20250929#thinking (zero-shot, val) Dutch CoLA test split 52.9%
- 23 xai/grok-4-1-fast-reasoning (zero-shot, val) Dutch CoLA test split 52.6%
- 24 mistralai/Mistral-Small-4-119B-2603 (val) Dutch CoLA test split 52.2%
- 25 claude-sonnet-4-6 (zero-shot, val) Dutch CoLA test split 51.6%
- 26 openai/gpt-5.4-mini-2026-03-17#none (zero-shot, val) Dutch CoLA test split 51.6%
- 27 openai/gpt-5.4-mini-2026-03-17#low (zero-shot, val) Dutch CoLA test split 51.3%
- 28 Qwen/Qwen3.6-27B-FP8 (val) Dutch CoLA test split 51%
- 29 deepcogito/cogito-v1-preview-qwen-14B Dutch CoLA test split 50.8%
- 30 mistralai/Mistral-Medium-3.5-128B (zero-shot, val) Dutch CoLA test split 50.6%
- 31 openai/gpt-5.4-2026-03-05#none (zero-shot, val) Dutch CoLA test split 50.6%
- 32 NbAiLab/borealis2-26b-a4b-preview (val) Dutch CoLA test split 49.8%
- 33 speakleash/Bielik-11B-v3.1-Instruct (val) Dutch CoLA test split 49.7%
- 34 mistralai/Ministral-3-14B-Base-2512 (val) Dutch CoLA test split 49.6%
- 35 deepcogito/cogito-v2-preview-llama-70B Dutch CoLA test split 49.6%
- 36 mistralai/Ministral-3-14B-Reasoning-2512 (val) Dutch CoLA test split 49.5%
- 37
Gemma 3 27B Dutch CoLA test split 49.4%
- 38 google/gemma-4-26B-A4B-it (val) Dutch CoLA test split 49.1%
- 39 swiss-ai/Apertus-v1.5-70B (val) Dutch CoLA test split 49%
- 40 Qwen/Qwen3.5-35B-A3B-FP8 (val) Dutch CoLA test split 48.9%
- 41 gpt-6-luna (zero-shot, val) Dutch CoLA test split 48.8%
- 42 NbAiLab/borealis2-26b-a4b-preview (zero-shot, val) Dutch CoLA test split 48.7%
- 43 zai-org/GLM-5.3-Flash (zero-shot, val) Dutch CoLA test split 48.7%
- 44
Gemma 3 12B PT Dutch CoLA test split 48.5%
- 45
Gemma 3 27B PT Dutch CoLA test split 48.2%
- 46 Qwen/Qwen3-14B#no-thinking Dutch CoLA test split 48%
- 47 Qwen/Qwen3-32B#no-thinking Dutch CoLA test split 48%
- 48 NbAiLab/borealis2-31b-preview (zero-shot, val) Dutch CoLA test split 47.9%
- 49 gpt-5.6-sol (zero-shot, val) Dutch CoLA test split 47.7%
- 50 deepcogito/cogito-v1-preview-qwen-32B Dutch CoLA test split 47.5%
- 51 gemini/gemini-2.5-flash-lite#no-thinking (zero-shot, val) Dutch CoLA test split 47.3%
- 52 amalia-llm/AMALIA-9B-0626-DPO (val) Dutch CoLA test split 46.6%
- 53 deepseek/deepseek-flash#low (zero-shot, val) Dutch CoLA test split 46.4%
- 54 gpt-5.2-2025-12-11 (zero-shot, val) Dutch CoLA test split 46.3%
- 55
Gemma 3 12B Dutch CoLA test split 45.8%
- 56 claude-haiku-4-5-20251001 (zero-shot, val) Dutch CoLA test split 45.2%
- 57 gemini/gemini-2.5-flash-lite#thinking (zero-shot, val) Dutch CoLA test split 45.1%
- 58 mistralai/Mistral-Small-3.1-24B-Base-2503 Dutch CoLA test split 43.9%
- 59 swiss-ai/Apertus-v1.5-8B (val) Dutch CoLA test split 43.9%
- 60 openai/gpt-oss-120b (val) Dutch CoLA test split 43.7%
- 61 speakleash/Bielik-11B-v3.0-Instruct (val) Dutch CoLA test split 43.1%
- 62 openeurollm/prelude-long (val) Dutch CoLA test split 43.1%
- 63 deepcogito/cogito-v1-preview-llama-8B Dutch CoLA test split 42.6%
- 64 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (val) Dutch CoLA test split 42.5%
- 65 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 (val) Dutch CoLA test split 42.5%
- 66 latam-gpt/Llama-3.1-70B-LatamGPT-SFT-1.0 (val) Dutch CoLA test split 42.2%
- 67 Qwen/Qwen3-8B#no-thinking Dutch CoLA test split 42.1%
- 68 Qwen/Qwen3.5-9B (val) Dutch CoLA test split 42%
- 69 gpt-5-nano-2025-08-07 (zero-shot, val) Dutch CoLA test split 41.9%
- 70 openai/gpt-5.4-mini-2026-03-17#medium (zero-shot, val) Dutch CoLA test split 41.6%
- 71 openai/gpt-5.4-nano-2026-03-17#medium (zero-shot, val) Dutch CoLA test split 41.6%
- 72 allenai/Llama-3.1-Tulu-3-8B-SFT Dutch CoLA test split 41.3%
- 73 google/gemma-4-12B-it (val) Dutch CoLA test split 41.3%
- 74 mistralai/Mistral-Small-3.2-24B-Instruct-2506 Dutch CoLA test split 41.2%
- 75 openeurollm/prelude-base (val) Dutch CoLA test split 40.9%
- 76 Qwen/Qwen3.8-27B (val) Dutch CoLA test split 40.7%
- 77 gpt-5.6-terra (zero-shot, val) Dutch CoLA test split 40.2%
- 78 speakleash/Bielik-Minitron-7B-v3.0-Instruct (val) Dutch CoLA test split 39.9%
- 79 openai/gpt-5.4-nano-2026-03-17#low (zero-shot, val) Dutch CoLA test split 39.8%
- 80 Qwen/Qwen3-4B#no-thinking Dutch CoLA test split 39.8%
- 81 google/gemma-4-E4B-it (val) Dutch CoLA test split 39.5%
- 82 mistralai/Mistral-Large-3-675B-Instruct-2512 (zero-shot, val) Dutch CoLA test split 39.3%
- 83 speakleash/Bielik-One-11B-v3.1-r1@75a2d83890e3c68bf7fac2fe5bbe2bff6372036c (zero-shot) Dutch CoLA test split 39.1%
- 84 nvidia/Nemotron-Cascade-2-30B-A3B (val) Dutch CoLA test split 38.9%
- 85 zai-org/GLM-4.7-Flash (val) Dutch CoLA test split 38.7%
- 86 swiss-ai/Apertus-v1.5-70B (zero-shot, val) Dutch CoLA test split 38.6%
- 87 domyn/Domyn-Small-v1.0 (val) Dutch CoLA test split 38.1%
- 88 mistralai/Ministral-3-8B-Reasoning-2512 Dutch CoLA test split 38%
- 89
Llama 3.1-70B Dutch CoLA test split 38%
- 90
Gemma 3 4B Dutch CoLA test split 37.8%
- 91 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 (val) Dutch CoLA test split 37.6%
- 92 microsoft/mdeberta-v3-base Dutch CoLA test split 37.5%
- 93 meta-llama/Llama-3.1-8B-Instruct Dutch CoLA test split 37.1%
- 94 amalia-llm/AMALIA-9B-0626-SFT (val) Dutch CoLA test split 37.1%
- 95 openai/gpt-5.4-nano-2026-03-17#high (zero-shot, val) Dutch CoLA test split 37%
- 96 mistralai/Mistral-Small-3.1-24B-Instruct-2503 Dutch CoLA test split 37%
- 97 mistralai/Mistral-Small-4-119B-2603 (zero-shot, val) Dutch CoLA test split 36.6%
- 98 gpt-oss-20b Dutch CoLA test split 36.3%
- 99 Qwen/Qwen3.5-9B-Base (val) Dutch CoLA test split 36.3%
- 100 mistralai/Ministral-3-14B-Instruct-2512 (val) Dutch CoLA test split 36.3%
- 101 openai/gpt-5.4-mini-2026-03-17#high (zero-shot, val) Dutch CoLA test split 36.1%
- 102 OpenLLM-France/Luciole-23B-Instruct-1.1 (val) Dutch CoLA test split 35.9%
- 103 Qwen/Qwen3.5-35B-A3B-Base (val) Dutch CoLA test split 35.9%
- 104 speakleash/Bielik-11B-v2.3-Instruct Dutch CoLA test split 35.2%
- 105 NbAiLab/borealis2-e4b-preview (val) Dutch CoLA test split 35.2%
- 106 CohereLabs/aya-expanse-8b (val) Dutch CoLA test split 34.7%
- 107 utter-project/EuroLLM-9B-Instruct Dutch CoLA test split 33.9%
- 108
Qwen3-4B-Thinking-2507 Dutch CoLA test split 33.6%
- 109 utter-project/EuroLLM-9B-Instruct-2512 (val) Dutch CoLA test split 33.1%
- 110 OpenLLM-France/Luciole-8B-Instruct-1.1 (val) Dutch CoLA test split 32.4%
- 111 google/gemma-4-31B (val) Dutch CoLA test split 32.2%
- 112 Qwen/Qwen3.5-2B (val) Dutch CoLA test split 31.6%
- 113 granite-4.0-micro Dutch CoLA test split 31.5%
- 114 NbAiLab/borealis2-e2b-preview (val) Dutch CoLA test split 31.1%
- 115
Gemma 3 4B PT Dutch CoLA test split 31.1%
- 116 mii-llm/nesso-4B Dutch CoLA test split 31%
- 117 allenai/Olmo-3-1125-32B Dutch CoLA test split 30.7%
- 118 danish-foundation-models/DFM-Mimir-v1.8 (val) Dutch CoLA test split 30.4%
- 119 Aleph-Alpha/Kolibri-1 (val) Dutch CoLA test split 29.8%
- 120 google/gemma-4-E2B-it (val) Dutch CoLA test split 29.8%
- 121 typesafe/jev-1.13.0 (zero-shot) Dutch CoLA test split 29.8%
- 122 mistralai/Ministral-3-3B-Instruct-2512 (val) Dutch CoLA test split 29.4%
- 123 deepcogito/cogito-v1-preview-llama-3B Dutch CoLA test split 29%
- 124 allenai/Olmo-3.1-32B-Instruct-DPO Dutch CoLA test split 28.8%
- 125 allenai/Olmo-3.1-32B-Instruct-SFT Dutch CoLA test split 28.8%
- 126 danish-foundation-models/DFM-Mimir-v1.8 (zero-shot, val) Dutch CoLA test split 28.5%
- 127
Olmo 3.1 32B Instruct Dutch CoLA test split 27.5%
- 128 ibm-granite/granite-4.0-h-micro Dutch CoLA test split 27.2%
- 129 Qwen/Qwen3.5-4B (val) Dutch CoLA test split 27.1%
- 130
Ministral 3 3B Reasoning Dutch CoLA test split 26.9%
- 131
Llama 3.1-8B Dutch CoLA test split 26.8%
- 132 danish-foundation-models/DFM-Mimir-v1.5 (val) Dutch CoLA test split 26.6%
- 133 danish-foundation-models/DFM-Mimir-v1.5 (zero-shot, val) Dutch CoLA test split 25.9%
- 134 BSC-LT/salamandra-7b-instruct Dutch CoLA test split 25.8%
- 135 amalia-llm/AMALIA-9B-0626-DPO (zero-shot, val) Dutch CoLA test split 25.5%
- 136 amalia-llm/AMALIA-9B-0626-SFT (zero-shot, val) Dutch CoLA test split 24.8%
- 137 swiss-ai/Apertus-v1.1-4B (val) Dutch CoLA test split 24.6%
- 138 google/gemma-4-26B-A4B (val) Dutch CoLA test split 23.8%
- 139 Qwen/Qwen3.5-2B-Base (val) Dutch CoLA test split 22.8%
- 140 meta-llama/Llama-3.2-3B-Instruct Dutch CoLA test split 22.6%
- 141 inclusionAI/Ling-mini-2.0 (val) Dutch CoLA test split 22.5%
- 142 OpenLLM-France/Luciole-23B-Base (val) Dutch CoLA test split 22.3%
- 143 google/gemma-4-E4B (val) Dutch CoLA test split 22%
- 144 google/gemma-4-12B (val) Dutch CoLA test split 21.4%
- 145 mistralai/Ministral-3-8B-Instruct-2512 (val) Dutch CoLA test split 21.3%
- 146 Qwen/Qwen3.5-4B-Base (val) Dutch CoLA test split 21.2%
- 147 LGAI-EXAONE/EXAONE-4.0-32B (val) Dutch CoLA test split 20.5%
- 148 Qwen/Qwen3-1.7B#no-thinking Dutch CoLA test split 20.3%
- 149 utter-project/EuroLLM-22B-2512 Dutch CoLA test split 20.1%
- 150 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 Dutch CoLA test split 19.9%
- 151 HuggingFaceTB/SmolLM3-3B-Base Dutch CoLA test split 19.9%
- 152 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 (val) Dutch CoLA test split 19.8%
- 153 danish-foundation-models/DFM-Mimir (val) Dutch CoLA test split 19.6%
- 154 Qwen/Qwen3.5-0.8B-Base (val) Dutch CoLA test split 19.3%
- 155 openai/gpt-5.4-nano-2026-03-17#none (zero-shot, val) Dutch CoLA test split 19%
- 156 jhu-clsp/mmBERT-small Dutch CoLA test split 18.5%
- 157 jhu-clsp/mmBERT-base Dutch CoLA test split 18.4%
- 158 swiss-ai/Apertus-v1.5-8B (zero-shot, val) Dutch CoLA test split 18.1%
- 159 allenai/Olmo-3-1025-7B Dutch CoLA test split 17.6%
- 160 speakleash/Bielik-One-4B-r1@567d29b0f15df28794acf0f6187f9216860d4551 (zero-shot) Dutch CoLA test split 17.3%
- 161 danish-foundation-models/DFM-Mimir-v1.8-Mini (val) Dutch CoLA test split 17.1%
- 162 JetBrains/Mellum2-12B-A2.5B-Instruct (val) Dutch CoLA test split 16.7%
- 163 openGPT-X/Teuken-7B-instruct-v0.6 (val) Dutch CoLA test split 16.2%
- 164 HuggingFaceTB/SmolLM3-3B#no-thinking Dutch CoLA test split 15.7%
- 165 mistralai/Ministral-3-8B-Base-2512 (val) Dutch CoLA test split 15.7%
- 166 occiglot/occiglot-7b-eu5-instruct Dutch CoLA test split 15.5%
- 167 JetBrains/Mellum2-12B-A2.5B-Base (val) Dutch CoLA test split 15.2%
- 168 Qwen/Qwen3.5-0.8B (val) Dutch CoLA test split 14.9%
- 169 swiss-ai/Apertus-v1.1-4B-Instruct (val) Dutch CoLA test split 14.9%
- 170 danish-foundation-models/DFM-Mimir-v1.8-Mini (zero-shot, val) Dutch CoLA test split 14.8%
- 171 allenai/Olmo-3-7B-Instruct Dutch CoLA test split 14.3%
- 172 openeurollm/oellm-9b-128k-theta32m-v3 (val) Dutch CoLA test split 13.9%
- 173 allenai/Olmo-3-7B-Instruct-DPO Dutch CoLA test split 13.6%
- 174 danish-foundation-models/DFM-Mimir (zero-shot, val) Dutch CoLA test split 13.6%
- 175 LiquidAI/LFM2.5-8B-A1B (val) Dutch CoLA test split 12.6%
- 176 intfloat/multilingual-e5-base Dutch CoLA test split 12.4%
- 177 gemini/gemini-3-flash-preview#no-thinking (zero-shot, val) Dutch CoLA test split 11.8%
- 178 intfloat/multilingual-e5-large Dutch CoLA test split 11.4%
- 179 allenai/Olmo-3-7B-Instruct-SFT Dutch CoLA test split 11.3%
- 180 EuroBERT/EuroBERT-210m Dutch CoLA test split 11.3%
- 181 LiquidAI/LFM2.5-8B-A1B-Base (val) Dutch CoLA test split 11.3%
- 182 answerdotai/ModernBERT-large Dutch CoLA test split 11.2%
- 183 EuroBERT/EuroBERT-610m Dutch CoLA test split 11.1%
- 184 CohereLabs/aya-23-8B (val) Dutch CoLA test split 9.84%
- 185 Aleph-Alpha/Kolibri-1 (zero-shot, val) Dutch CoLA test split 9.65%
- 186 speakleash/Bielik-1.5B-v3.0-Instruct Dutch CoLA test split 9.52%
- 187 utter-project/EuroLLM-9B Dutch CoLA test split 9.41%
- 188 TrustLLMeu/trustllm-v2-8b-midtrain@3c7af30bc43bb3c8607e0fab6e7f2a23dd64af49 Dutch CoLA test split 8.80%
- 189 distilbert/distilbert-base-multilingual-cased Dutch CoLA test split 7.68%
- 190 Geotrend/distilbert-base-25lang-cased Dutch CoLA test split 7.56%
- 191 google/gemma-4-E2B (val) Dutch CoLA test split 7.55%
- 192
Qwen3-0.6B Dutch CoLA test split 7.00%
- 193 swiss-ai/Apertus-v1.1-0.5B (val) Dutch CoLA test split 6.66%
- 194 answerdotai/ModernBERT-base Dutch CoLA test split 6.56%
- 195 AI-Sweden-Models/bert-large-nordic-pile-1M-steps Dutch CoLA test split 6.47%
- 196 swiss-ai/Apertus-v1.1-1.5B (val) Dutch CoLA test split 6.40%
- 197 AI-Sweden-Models/Llama-3-8B-instruct (val) Dutch CoLA test split 6.02%
- 198 OmniJev/OneJev1.6-0.8B (zero-shot) Dutch CoLA test split 5.99%
- 199 fastino/GLiNER2.5-Decide (zero-shot) Dutch CoLA test split 5.92%
- 200
Gemma 3 1B PT Dutch CoLA test split 5.90%
- 201 EuropeanParliament/EUBERT Dutch CoLA test split 5.74%
- 202 OpenLLM-France/Luciole-8B-Base (val) Dutch CoLA test split 5.50%
- 203 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f (zero-shot) Dutch CoLA test split 5.00%
- 204 TrustLLMeu/trustllm-v2-8b@d8cc924fa550fbf9474f5251e8e2f92d1c1bf894 Dutch CoLA test split 4.90%
- 205 fresh-electra-small Dutch CoLA test split 4.70%
- 206 fresh-xlm-roberta-base Dutch CoLA test split 4.62%
- 207 flwrlabs/Lizzy-7B (val) Dutch CoLA test split 4.61%
- 208 NbAiLab/borealis2-e4b-preview (zero-shot, val) Dutch CoLA test split 4.37%
- 209 syvai/claes-v1 (zero-shot) Dutch CoLA test split 4.30%
- 210 convaiinnovations/laya (zero-shot) Dutch CoLA test split 4.00%
- 211 swiss-ai/Apertus-v1.1-0.5B-Instruct (val) Dutch CoLA test split 3.89%
- 212 occiglot/occiglot-7b-eu5 Dutch CoLA test split 3.88%
- 213 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f Dutch CoLA test split 3.72%
- 214 Qwen/Qwen3-0.6B#no-thinking Dutch CoLA test split 3.60%
- 215 OpenLLM-France/Luciole-1B-Instruct-1.1 (val) Dutch CoLA test split 3.56%
- 216 LFM2-1.2B Dutch CoLA test split 3.52%
- 217 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base Dutch CoLA test split 3.37%
- 218 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 (zero-shot) Dutch CoLA test split 3.34%
- 219 JetBrains/Mellum2-12B-A2.5B-Thinking-SFT (val) Dutch CoLA test split 3.09%
- 220 PleIAs/Pleias-350m-Preview Dutch CoLA test split 2.98%
- 221 yulan-team/YuLan-Mini-Instruct Dutch CoLA test split 2.90%
- 222 meta-llama/Llama-3.2-1B-Instruct Dutch CoLA test split 2.78%
- 223 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 Dutch CoLA test split 2.62%
- 224
Gemma 3 1B Dutch CoLA test split 2.52%
- 225
Gemma 3 270M Dutch CoLA test split 2.36%
- 226
Llama 3.2 1B Dutch CoLA test split 2.35%
- 227 OpenLLM-France/Luciole-1B-Base (val) Dutch CoLA test split 2.26%
- 228 BSC-LT/salamandra-7b Dutch CoLA test split 1.95%
- 229 PleIAs/Pleias-1.2b-Preview Dutch CoLA test split 1.91%
- 230 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 (zero-shot) Dutch CoLA test split 1.82%
- 231 syvai/claes-preview (zero-shot) Dutch CoLA test split 1.76%
- 232 JetBrains/Mellum2-12B-A2.5B-Thinking (val) Dutch CoLA test split 1.62%
- 233 AIDC-AI/Marco-Mini-Global-Base (val) Dutch CoLA test split 1.34%
- 234 PleIAs/Monad Dutch CoLA test split 1.23%
- 235 LiquidAI/LFM2-350M Dutch CoLA test split 1.08%
- 236 JetBrains/Mellum2-12B-A2.5B-Instruct-SFT (val) Dutch CoLA test split 0.92%
- 237
SmolLM2-135M Dutch CoLA test split 0.70%
- 238 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 Dutch CoLA test split 0.56%
- 239
SmolLM2-360M Dutch CoLA test split 0.46%
- 240 HuggingFaceTB/SmolLM2-135M-Instruct Dutch CoLA test split 0.40%
- 241 convaiinnovations/laya-multilingual (zero-shot) Dutch CoLA test split 0.32%
- 242 google-bert/bert-base-multilingual-cased (val) Dutch CoLA test split 0.23%
- 243 NbAiLab/borealis2-e2b-preview (zero-shot, val) Dutch CoLA test split 0.23%
- 244 LiquidAI/LFM2-700M Dutch CoLA test split 0.19%
- 245 swiss-ai/Apertus-v1.1-1.5B (zero-shot) Dutch CoLA test split 0.15%
- 246 jaredpalmer/kev-0.5b (zero-shot) Dutch CoLA test split 0.14%
- 247 HuggingFaceTB/SmolLM2-360M-Instruct Dutch CoLA test split 0.06%
- 248 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct Dutch CoLA test split 0.06%
- 249 AI-Sweden-Models/Llama-3-8B-instruct (zero-shot, val) Dutch CoLA test split 0%
- 250 JetBrains/Mellum2-12B-A2.5B-Base-Pretrain (val) Dutch CoLA test split 0%
- 251 mistralai/Ministral-3-14B-Instruct-2512 (zero-shot, val) Dutch CoLA test split 0%
- 252 mistralai/Ministral-3-3B-Instruct-2512 (zero-shot, val) Dutch CoLA test split 0%
- 253 Muse-research/Muse-3B (val) Dutch CoLA test split 0%
- 254 openGPT-X/Teuken-7B-base-v0.6 (val) Dutch CoLA test split 0%
- 255 PleIAs/Baguettotron@refs/pr/6#no-thinking Dutch CoLA test split 0%
- 256 swiss-ai/Apertus-v1.1-1.5B-Instruct (val) Dutch CoLA test split 0%
- 257 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview Dutch CoLA test split 0%
Source: EuroEval’s leaderboard
- 49.4
Gemma 3 27B
- 48.5
Gemma 3 12B PT
- 48.2
Gemma 3 27B PT
- 45.8
Gemma 3 12B
- 38.0
Llama 3.1-70B
- 37.8
Gemma 3 4B
- 36.3 gpt-oss-20b
- 33.6
Qwen3-4B-Thinking-2507
- 31.5 granite-4.0-micro
- 31.1
Gemma 3 4B PT
- 27.5
Olmo 3.1 32B Instruct
- 26.9
Ministral 3 3B Reasoning
- 26.8
Llama 3.1-8B
- 7.00
Qwen3-0.6B
- 5.90
Gemma 3 1B PT
- 3.52 LFM2-1.2B
- 2.52
Gemma 3 1B
- 2.36
Gemma 3 270M
- 2.35
Llama 3.2 1B
- 0.70
SmolLM2-135M
- 0.46
SmolLM2-360M
- 1
Gemma 3 27B Dutch CoLA test split 49.4%
- 2
Gemma 3 12B PT Dutch CoLA test split 48.5%
- 3
Gemma 3 27B PT Dutch CoLA test split 48.2%
- 4
Gemma 3 12B Dutch CoLA test split 45.8%
- 5
Llama 3.1-70B Dutch CoLA test split 38%
- 6
Gemma 3 4B Dutch CoLA test split 37.8%
- 7 gpt-oss-20b Dutch CoLA test split 36.3%
- 8
Qwen3-4B-Thinking-2507 Dutch CoLA test split 33.6%
- 9 granite-4.0-micro Dutch CoLA test split 31.5%
- 10
Gemma 3 4B PT Dutch CoLA test split 31.1%
- 11
Olmo 3.1 32B Instruct Dutch CoLA test split 27.5%
- 12
Ministral 3 3B Reasoning Dutch CoLA test split 26.9%
- 13
Llama 3.1-8B Dutch CoLA test split 26.8%
- 14
Qwen3-0.6B Dutch CoLA test split 7.00%
- 15
Gemma 3 1B PT Dutch CoLA test split 5.90%
- 16 LFM2-1.2B Dutch CoLA test split 3.52%
- 17
Gemma 3 1B Dutch CoLA test split 2.52%
- 18
Gemma 3 270M Dutch CoLA test split 2.36%
- 19
Llama 3.2 1B Dutch CoLA test split 2.35%
- 20
SmolLM2-135M Dutch CoLA test split 0.70%
- 21
SmolLM2-360M Dutch CoLA test split 0.46%
Source: EuroEval’s leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | gemini/gemini-3.7-flash (zero-shot, val) | 64.8% | Dutch CoLA test split | Official leaderboard | |
| 2 | gemini/gemini-3.6-flash (zero-shot, val) | 61.4% | Dutch CoLA test split | Official leaderboard | |
| 3 | gpt-6.2-astra (zero-shot, val) | 60.5% | Dutch CoLA test split | Official leaderboard | |
| 4 | gpt-6.2-sol (zero-shot, val) | 60.3% | Dutch CoLA test split | Official leaderboard | |
| 5 | gemini/gemini-3.5-flash-lite (zero-shot, val) | 59.7% | Dutch CoLA test split | Official leaderboard | |
| 6 | mistralai/Mistral-Large-3-675B-Instruct-2512 (val) | 58.6% | Dutch CoLA test split | Official leaderboard | |
| 7 | openai/gpt-5.6-luna (zero-shot, val) | 57.8% | Dutch CoLA test split | Official leaderboard | |
| 8 | gemini/gemini-3-flash-preview#thinking (zero-shot, val) | 57.4% | Dutch CoLA test split | Official leaderboard | |
| 9 | NbAiLab/borealis2-31b-preview (val) | 57.3% | Dutch CoLA test split | Official leaderboard | |
| 10 | claude-sonnet-4-5-20250929#no-thinking (zero-shot, val) | 57.3% | Dutch CoLA test split | Official leaderboard | |
| 11 | gpt-4.1-2025-04-14 (zero-shot, val) | 56.4% | Dutch CoLA test split | Official leaderboard | |
| 12 | deepseek/deepseek-flash#no-thinking (val) | 56.2% | Dutch CoLA test split | Official leaderboard | |
| 13 | gpt-6.1-sol (zero-shot, val) | 56.1% | Dutch CoLA test split | Official leaderboard | |
| 14 | utter-project/EuroLLM-22B-Instruct-2512 | 54.7% | Dutch CoLA test split | Official leaderboard | |
| 15 | zai-org/GLM-5.3-Flash (val) | 54.6% | Dutch CoLA test split | Official leaderboard | |
| 16 | gpt-6-astra (zero-shot, val) | 54.3% | Dutch CoLA test split | Official leaderboard | |
| 17 | gpt-6-sol (zero-shot, val) | 54.2% | Dutch CoLA test split | Official leaderboard | |
| 18 | gpt-5-mini-2025-08-07 (zero-shot, val) | 54.1% | Dutch CoLA test split | Official leaderboard | |
| 19 | gemini/gemini-3.1-flash-lite-preview (zero-shot, val) | 54% | Dutch CoLA test split | Official leaderboard | |
| 20 | mistralai/Mistral-Medium-3.5-128B (val) | 53.7% | Dutch CoLA test split | Official leaderboard | |
| 21 | google/gemma-4-31B-it (val) | 53% | Dutch CoLA test split | Official leaderboard | |
| 22 | claude-sonnet-4-5-20250929#thinking (zero-shot, val) | 52.9% | Dutch CoLA test split | Official leaderboard | |
| 23 | xai/grok-4-1-fast-reasoning (zero-shot, val) | 52.6% | Dutch CoLA test split | Official leaderboard | |
| 24 | mistralai/Mistral-Small-4-119B-2603 (val) | 52.2% | Dutch CoLA test split | Official leaderboard | |
| 25 | claude-sonnet-4-6 (zero-shot, val) | 51.6% | Dutch CoLA test split | Official leaderboard | |
| 26 | openai/gpt-5.4-mini-2026-03-17#none (zero-shot, val) | 51.6% | Dutch CoLA test split | Official leaderboard | |
| 27 | openai/gpt-5.4-mini-2026-03-17#low (zero-shot, val) | 51.3% | Dutch CoLA test split | Official leaderboard | |
| 28 | Qwen/Qwen3.6-27B-FP8 (val) | 51% | Dutch CoLA test split | Official leaderboard | |
| 29 | deepcogito/cogito-v1-preview-qwen-14B | 50.8% | Dutch CoLA test split | Official leaderboard | |
| 30 | mistralai/Mistral-Medium-3.5-128B (zero-shot, val) | 50.6% | Dutch CoLA test split | Official leaderboard | |
| 31 | openai/gpt-5.4-2026-03-05#none (zero-shot, val) | 50.6% | Dutch CoLA test split | Official leaderboard | |
| 32 | NbAiLab/borealis2-26b-a4b-preview (val) | 49.8% | Dutch CoLA test split | Official leaderboard | |
| 33 | speakleash/Bielik-11B-v3.1-Instruct (val) | 49.7% | Dutch CoLA test split | Official leaderboard | |
| 34 | mistralai/Ministral-3-14B-Base-2512 (val) | 49.6% | Dutch CoLA test split | Official leaderboard | |
| 35 | deepcogito/cogito-v2-preview-llama-70B | 49.6% | Dutch CoLA test split | Official leaderboard | |
| 36 | mistralai/Ministral-3-14B-Reasoning-2512 (val) | 49.5% | Dutch CoLA test split | Official leaderboard | |
| 37 | | 49.4% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 38 | google/gemma-4-26B-A4B-it (val) | 49.1% | Dutch CoLA test split | Official leaderboard | |
| 39 | swiss-ai/Apertus-v1.5-70B (val) | 49% | Dutch CoLA test split | Official leaderboard | |
| 40 | Qwen/Qwen3.5-35B-A3B-FP8 (val) | 48.9% | Dutch CoLA test split | Official leaderboard | |
| 41 | gpt-6-luna (zero-shot, val) | 48.8% | Dutch CoLA test split | Official leaderboard | |
| 42 | NbAiLab/borealis2-26b-a4b-preview (zero-shot, val) | 48.7% | Dutch CoLA test split | Official leaderboard | |
| 43 | zai-org/GLM-5.3-Flash (zero-shot, val) | 48.7% | Dutch CoLA test split | Official leaderboard | |
| 44 | | 48.5% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 45 | | 48.2% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 46 | Qwen/Qwen3-14B#no-thinking | 48% | Dutch CoLA test split | Official leaderboard | |
| 47 | Qwen/Qwen3-32B#no-thinking | 48% | Dutch CoLA test split | Official leaderboard | |
| 48 | NbAiLab/borealis2-31b-preview (zero-shot, val) | 47.9% | Dutch CoLA test split | Official leaderboard | |
| 49 | gpt-5.6-sol (zero-shot, val) | 47.7% | Dutch CoLA test split | Official leaderboard | |
| 50 | deepcogito/cogito-v1-preview-qwen-32B | 47.5% | Dutch CoLA test split | Official leaderboard | |
| 51 | gemini/gemini-2.5-flash-lite#no-thinking (zero-shot, val) | 47.3% | Dutch CoLA test split | Official leaderboard | |
| 52 | amalia-llm/AMALIA-9B-0626-DPO (val) | 46.6% | Dutch CoLA test split | Official leaderboard | |
| 53 | deepseek/deepseek-flash#low (zero-shot, val) | 46.4% | Dutch CoLA test split | Official leaderboard | |
| 54 | gpt-5.2-2025-12-11 (zero-shot, val) | 46.3% | Dutch CoLA test split | Official leaderboard | |
| 55 | | 45.8% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 56 | claude-haiku-4-5-20251001 (zero-shot, val) | 45.2% | Dutch CoLA test split | Official leaderboard | |
| 57 | gemini/gemini-2.5-flash-lite#thinking (zero-shot, val) | 45.1% | Dutch CoLA test split | Official leaderboard | |
| 58 | mistralai/Mistral-Small-3.1-24B-Base-2503 | 43.9% | Dutch CoLA test split | Official leaderboard | |
| 59 | swiss-ai/Apertus-v1.5-8B (val) | 43.9% | Dutch CoLA test split | Official leaderboard | |
| 60 | openai/gpt-oss-120b (val) | 43.7% | Dutch CoLA test split | Official leaderboard | |
| 61 | speakleash/Bielik-11B-v3.0-Instruct (val) | 43.1% | Dutch CoLA test split | Official leaderboard | |
| 62 | openeurollm/prelude-long (val) | 43.1% | Dutch CoLA test split | Official leaderboard | |
| 63 | deepcogito/cogito-v1-preview-llama-8B | 42.6% | Dutch CoLA test split | Official leaderboard | |
| 64 | nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (val) | 42.5% | Dutch CoLA test split | Official leaderboard | |
| 65 | nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 (val) | 42.5% | Dutch CoLA test split | Official leaderboard | |
| 66 | latam-gpt/Llama-3.1-70B-LatamGPT-SFT-1.0 (val) | 42.2% | Dutch CoLA test split | Official leaderboard | |
| 67 | Qwen/Qwen3-8B#no-thinking | 42.1% | Dutch CoLA test split | Official leaderboard | |
| 68 | Qwen/Qwen3.5-9B (val) | 42% | Dutch CoLA test split | Official leaderboard | |
| 69 | gpt-5-nano-2025-08-07 (zero-shot, val) | 41.9% | Dutch CoLA test split | Official leaderboard | |
| 70 | openai/gpt-5.4-mini-2026-03-17#medium (zero-shot, val) | 41.6% | Dutch CoLA test split | Official leaderboard | |
| 71 | openai/gpt-5.4-nano-2026-03-17#medium (zero-shot, val) | 41.6% | Dutch CoLA test split | Official leaderboard | |
| 72 | allenai/Llama-3.1-Tulu-3-8B-SFT | 41.3% | Dutch CoLA test split | Official leaderboard | |
| 73 | google/gemma-4-12B-it (val) | 41.3% | Dutch CoLA test split | Official leaderboard | |
| 74 | mistralai/Mistral-Small-3.2-24B-Instruct-2506 | 41.2% | Dutch CoLA test split | Official leaderboard | |
| 75 | openeurollm/prelude-base (val) | 40.9% | Dutch CoLA test split | Official leaderboard | |
| 76 | Qwen/Qwen3.8-27B (val) | 40.7% | Dutch CoLA test split | Official leaderboard | |
| 77 | gpt-5.6-terra (zero-shot, val) | 40.2% | Dutch CoLA test split | Official leaderboard | |
| 78 | speakleash/Bielik-Minitron-7B-v3.0-Instruct (val) | 39.9% | Dutch CoLA test split | Official leaderboard | |
| 79 | openai/gpt-5.4-nano-2026-03-17#low (zero-shot, val) | 39.8% | Dutch CoLA test split | Official leaderboard | |
| 80 | Qwen/Qwen3-4B#no-thinking | 39.8% | Dutch CoLA test split | Official leaderboard | |
| 81 | google/gemma-4-E4B-it (val) | 39.5% | Dutch CoLA test split | Official leaderboard | |
| 82 | mistralai/Mistral-Large-3-675B-Instruct-2512 (zero-shot, val) | 39.3% | Dutch CoLA test split | Official leaderboard | |
| 83 | speakleash/Bielik-One-11B-v3.1-r1@75a2d83890e3c68bf7fac2fe5bbe2bff6372036c (zero-shot) | 39.1% | Dutch CoLA test split | Official leaderboard | |
| 84 | nvidia/Nemotron-Cascade-2-30B-A3B (val) | 38.9% | Dutch CoLA test split | Official leaderboard | |
| 85 | zai-org/GLM-4.7-Flash (val) | 38.7% | Dutch CoLA test split | Official leaderboard | |
| 86 | swiss-ai/Apertus-v1.5-70B (zero-shot, val) | 38.6% | Dutch CoLA test split | Official leaderboard | |
| 87 | domyn/Domyn-Small-v1.0 (val) | 38.1% | Dutch CoLA test split | Official leaderboard | |
| 88 | mistralai/Ministral-3-8B-Reasoning-2512 | 38% | Dutch CoLA test split | Official leaderboard | |
| 89 | | 38% | Dutch CoLA test split | Jul 2024 | Official leaderboard |
| 90 | | 37.8% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 91 | nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 (val) | 37.6% | Dutch CoLA test split | Official leaderboard | |
| 92 | microsoft/mdeberta-v3-base | 37.5% | Dutch CoLA test split | Official leaderboard | |
| 93 | meta-llama/Llama-3.1-8B-Instruct | 37.1% | Dutch CoLA test split | Official leaderboard | |
| 94 | amalia-llm/AMALIA-9B-0626-SFT (val) | 37.1% | Dutch CoLA test split | Official leaderboard | |
| 95 | openai/gpt-5.4-nano-2026-03-17#high (zero-shot, val) | 37% | Dutch CoLA test split | Official leaderboard | |
| 96 | mistralai/Mistral-Small-3.1-24B-Instruct-2503 | 37% | Dutch CoLA test split | Official leaderboard | |
| 97 | mistralai/Mistral-Small-4-119B-2603 (zero-shot, val) | 36.6% | Dutch CoLA test split | Official leaderboard | |
| 98 | gpt-oss-20b | 36.3% | Dutch CoLA test split | Aug 2025 | Official leaderboard |
| 99 | Qwen/Qwen3.5-9B-Base (val) | 36.3% | Dutch CoLA test split | Official leaderboard | |
| 100 | mistralai/Ministral-3-14B-Instruct-2512 (val) | 36.3% | Dutch CoLA test split | Official leaderboard | |
| 101 | openai/gpt-5.4-mini-2026-03-17#high (zero-shot, val) | 36.1% | Dutch CoLA test split | Official leaderboard | |
| 102 | OpenLLM-France/Luciole-23B-Instruct-1.1 (val) | 35.9% | Dutch CoLA test split | Official leaderboard | |
| 103 | Qwen/Qwen3.5-35B-A3B-Base (val) | 35.9% | Dutch CoLA test split | Official leaderboard | |
| 104 | speakleash/Bielik-11B-v2.3-Instruct | 35.2% | Dutch CoLA test split | Official leaderboard | |
| 105 | NbAiLab/borealis2-e4b-preview (val) | 35.2% | Dutch CoLA test split | Official leaderboard | |
| 106 | CohereLabs/aya-expanse-8b (val) | 34.7% | Dutch CoLA test split | Official leaderboard | |
| 107 | utter-project/EuroLLM-9B-Instruct | 33.9% | Dutch CoLA test split | Official leaderboard | |
| 108 | | 33.6% | Dutch CoLA test split | Aug 2025 | Official leaderboard |
| 109 | utter-project/EuroLLM-9B-Instruct-2512 (val) | 33.1% | Dutch CoLA test split | Official leaderboard | |
| 110 | OpenLLM-France/Luciole-8B-Instruct-1.1 (val) | 32.4% | Dutch CoLA test split | Official leaderboard | |
| 111 | google/gemma-4-31B (val) | 32.2% | Dutch CoLA test split | Official leaderboard | |
| 112 | Qwen/Qwen3.5-2B (val) | 31.6% | Dutch CoLA test split | Official leaderboard | |
| 113 | granite-4.0-micro | 31.5% | Dutch CoLA test split | Oct 2025 | Official leaderboard |
| 114 | NbAiLab/borealis2-e2b-preview (val) | 31.1% | Dutch CoLA test split | Official leaderboard | |
| 115 | | 31.1% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 116 | mii-llm/nesso-4B | 31% | Dutch CoLA test split | Official leaderboard | |
| 117 | allenai/Olmo-3-1125-32B | 30.7% | Dutch CoLA test split | Official leaderboard | |
| 118 | danish-foundation-models/DFM-Mimir-v1.8 (val) | 30.4% | Dutch CoLA test split | Official leaderboard | |
| 119 | Aleph-Alpha/Kolibri-1 (val) | 29.8% | Dutch CoLA test split | Official leaderboard | |
| 120 | google/gemma-4-E2B-it (val) | 29.8% | Dutch CoLA test split | Official leaderboard | |
| 121 | typesafe/jev-1.13.0 (zero-shot) | 29.8% | Dutch CoLA test split | Official leaderboard | |
| 122 | mistralai/Ministral-3-3B-Instruct-2512 (val) | 29.4% | Dutch CoLA test split | Official leaderboard | |
| 123 | deepcogito/cogito-v1-preview-llama-3B | 29% | Dutch CoLA test split | Official leaderboard | |
| 124 | allenai/Olmo-3.1-32B-Instruct-DPO | 28.8% | Dutch CoLA test split | Official leaderboard | |
| 125 | allenai/Olmo-3.1-32B-Instruct-SFT | 28.8% | Dutch CoLA test split | Official leaderboard | |
| 126 | danish-foundation-models/DFM-Mimir-v1.8 (zero-shot, val) | 28.5% | Dutch CoLA test split | Official leaderboard | |
| 127 | | 27.5% | Dutch CoLA test split | Dec 2025 | Official leaderboard |
| 128 | ibm-granite/granite-4.0-h-micro | 27.2% | Dutch CoLA test split | Official leaderboard | |
| 129 | Qwen/Qwen3.5-4B (val) | 27.1% | Dutch CoLA test split | Official leaderboard | |
| 130 | | 26.9% | Dutch CoLA test split | Dec 2025 | Official leaderboard |
| 131 | | 26.8% | Dutch CoLA test split | Jul 2024 | Official leaderboard |
| 132 | danish-foundation-models/DFM-Mimir-v1.5 (val) | 26.6% | Dutch CoLA test split | Official leaderboard | |
| 133 | danish-foundation-models/DFM-Mimir-v1.5 (zero-shot, val) | 25.9% | Dutch CoLA test split | Official leaderboard | |
| 134 | BSC-LT/salamandra-7b-instruct | 25.8% | Dutch CoLA test split | Official leaderboard | |
| 135 | amalia-llm/AMALIA-9B-0626-DPO (zero-shot, val) | 25.5% | Dutch CoLA test split | Official leaderboard | |
| 136 | amalia-llm/AMALIA-9B-0626-SFT (zero-shot, val) | 24.8% | Dutch CoLA test split | Official leaderboard | |
| 137 | swiss-ai/Apertus-v1.1-4B (val) | 24.6% | Dutch CoLA test split | Official leaderboard | |
| 138 | google/gemma-4-26B-A4B (val) | 23.8% | Dutch CoLA test split | Official leaderboard | |
| 139 | Qwen/Qwen3.5-2B-Base (val) | 22.8% | Dutch CoLA test split | Official leaderboard | |
| 140 | meta-llama/Llama-3.2-3B-Instruct | 22.6% | Dutch CoLA test split | Official leaderboard | |
| 141 | inclusionAI/Ling-mini-2.0 (val) | 22.5% | Dutch CoLA test split | Official leaderboard | |
| 142 | OpenLLM-France/Luciole-23B-Base (val) | 22.3% | Dutch CoLA test split | Official leaderboard | |
| 143 | google/gemma-4-E4B (val) | 22% | Dutch CoLA test split | Official leaderboard | |
| 144 | google/gemma-4-12B (val) | 21.4% | Dutch CoLA test split | Official leaderboard | |
| 145 | mistralai/Ministral-3-8B-Instruct-2512 (val) | 21.3% | Dutch CoLA test split | Official leaderboard | |
| 146 | Qwen/Qwen3.5-4B-Base (val) | 21.2% | Dutch CoLA test split | Official leaderboard | |
| 147 | LGAI-EXAONE/EXAONE-4.0-32B (val) | 20.5% | Dutch CoLA test split | Official leaderboard | |
| 148 | Qwen/Qwen3-1.7B#no-thinking | 20.3% | Dutch CoLA test split | Official leaderboard | |
| 149 | utter-project/EuroLLM-22B-2512 | 20.1% | Dutch CoLA test split | Official leaderboard | |
| 150 | nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 | 19.9% | Dutch CoLA test split | Official leaderboard | |
| 151 | HuggingFaceTB/SmolLM3-3B-Base | 19.9% | Dutch CoLA test split | Official leaderboard | |
| 152 | nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 (val) | 19.8% | Dutch CoLA test split | Official leaderboard | |
| 153 | danish-foundation-models/DFM-Mimir (val) | 19.6% | Dutch CoLA test split | Official leaderboard | |
| 154 | Qwen/Qwen3.5-0.8B-Base (val) | 19.3% | Dutch CoLA test split | Official leaderboard | |
| 155 | openai/gpt-5.4-nano-2026-03-17#none (zero-shot, val) | 19% | Dutch CoLA test split | Official leaderboard | |
| 156 | jhu-clsp/mmBERT-small | 18.5% | Dutch CoLA test split | Official leaderboard | |
| 157 | jhu-clsp/mmBERT-base | 18.4% | Dutch CoLA test split | Official leaderboard | |
| 158 | swiss-ai/Apertus-v1.5-8B (zero-shot, val) | 18.1% | Dutch CoLA test split | Official leaderboard | |
| 159 | allenai/Olmo-3-1025-7B | 17.6% | Dutch CoLA test split | Official leaderboard | |
| 160 | speakleash/Bielik-One-4B-r1@567d29b0f15df28794acf0f6187f9216860d4551 (zero-shot) | 17.3% | Dutch CoLA test split | Official leaderboard | |
| 161 | danish-foundation-models/DFM-Mimir-v1.8-Mini (val) | 17.1% | Dutch CoLA test split | Official leaderboard | |
| 162 | JetBrains/Mellum2-12B-A2.5B-Instruct (val) | 16.7% | Dutch CoLA test split | Official leaderboard | |
| 163 | openGPT-X/Teuken-7B-instruct-v0.6 (val) | 16.2% | Dutch CoLA test split | Official leaderboard | |
| 164 | HuggingFaceTB/SmolLM3-3B#no-thinking | 15.7% | Dutch CoLA test split | Official leaderboard | |
| 165 | mistralai/Ministral-3-8B-Base-2512 (val) | 15.7% | Dutch CoLA test split | Official leaderboard | |
| 166 | occiglot/occiglot-7b-eu5-instruct | 15.5% | Dutch CoLA test split | Official leaderboard | |
| 167 | JetBrains/Mellum2-12B-A2.5B-Base (val) | 15.2% | Dutch CoLA test split | Official leaderboard | |
| 168 | Qwen/Qwen3.5-0.8B (val) | 14.9% | Dutch CoLA test split | Official leaderboard | |
| 169 | swiss-ai/Apertus-v1.1-4B-Instruct (val) | 14.9% | Dutch CoLA test split | Official leaderboard | |
| 170 | danish-foundation-models/DFM-Mimir-v1.8-Mini (zero-shot, val) | 14.8% | Dutch CoLA test split | Official leaderboard | |
| 171 | allenai/Olmo-3-7B-Instruct | 14.3% | Dutch CoLA test split | Official leaderboard | |
| 172 | openeurollm/oellm-9b-128k-theta32m-v3 (val) | 13.9% | Dutch CoLA test split | Official leaderboard | |
| 173 | allenai/Olmo-3-7B-Instruct-DPO | 13.6% | Dutch CoLA test split | Official leaderboard | |
| 174 | danish-foundation-models/DFM-Mimir (zero-shot, val) | 13.6% | Dutch CoLA test split | Official leaderboard | |
| 175 | LiquidAI/LFM2.5-8B-A1B (val) | 12.6% | Dutch CoLA test split | Official leaderboard | |
| 176 | intfloat/multilingual-e5-base | 12.4% | Dutch CoLA test split | Official leaderboard | |
| 177 | gemini/gemini-3-flash-preview#no-thinking (zero-shot, val) | 11.8% | Dutch CoLA test split | Official leaderboard | |
| 178 | intfloat/multilingual-e5-large | 11.4% | Dutch CoLA test split | Official leaderboard | |
| 179 | allenai/Olmo-3-7B-Instruct-SFT | 11.3% | Dutch CoLA test split | Official leaderboard | |
| 180 | EuroBERT/EuroBERT-210m | 11.3% | Dutch CoLA test split | Official leaderboard | |
| 181 | LiquidAI/LFM2.5-8B-A1B-Base (val) | 11.3% | Dutch CoLA test split | Official leaderboard | |
| 182 | answerdotai/ModernBERT-large | 11.2% | Dutch CoLA test split | Official leaderboard | |
| 183 | EuroBERT/EuroBERT-610m | 11.1% | Dutch CoLA test split | Official leaderboard | |
| 184 | CohereLabs/aya-23-8B (val) | 9.84% | Dutch CoLA test split | Official leaderboard | |
| 185 | Aleph-Alpha/Kolibri-1 (zero-shot, val) | 9.65% | Dutch CoLA test split | Official leaderboard | |
| 186 | speakleash/Bielik-1.5B-v3.0-Instruct | 9.52% | Dutch CoLA test split | Official leaderboard | |
| 187 | utter-project/EuroLLM-9B | 9.41% | Dutch CoLA test split | Official leaderboard | |
| 188 | TrustLLMeu/trustllm-v2-8b-midtrain@3c7af30bc43bb3c8607e0fab6e7f2a23dd64af49 | 8.80% | Dutch CoLA test split | Official leaderboard | |
| 189 | distilbert/distilbert-base-multilingual-cased | 7.68% | Dutch CoLA test split | Official leaderboard | |
| 190 | Geotrend/distilbert-base-25lang-cased | 7.56% | Dutch CoLA test split | Official leaderboard | |
| 191 | google/gemma-4-E2B (val) | 7.55% | Dutch CoLA test split | Official leaderboard | |
| 192 | | 7.00% | Dutch CoLA test split | Apr 2025 | Official leaderboard |
| 193 | swiss-ai/Apertus-v1.1-0.5B (val) | 6.66% | Dutch CoLA test split | Official leaderboard | |
| 194 | answerdotai/ModernBERT-base | 6.56% | Dutch CoLA test split | Official leaderboard | |
| 195 | AI-Sweden-Models/bert-large-nordic-pile-1M-steps | 6.47% | Dutch CoLA test split | Official leaderboard | |
| 196 | swiss-ai/Apertus-v1.1-1.5B (val) | 6.40% | Dutch CoLA test split | Official leaderboard | |
| 197 | AI-Sweden-Models/Llama-3-8B-instruct (val) | 6.02% | Dutch CoLA test split | Official leaderboard | |
| 198 | OmniJev/OneJev1.6-0.8B (zero-shot) | 5.99% | Dutch CoLA test split | Official leaderboard | |
| 199 | fastino/GLiNER2.5-Decide (zero-shot) | 5.92% | Dutch CoLA test split | Official leaderboard | |
| 200 | | 5.90% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 201 | EuropeanParliament/EUBERT | 5.74% | Dutch CoLA test split | Official leaderboard | |
| 202 | OpenLLM-France/Luciole-8B-Base (val) | 5.50% | Dutch CoLA test split | Official leaderboard | |
| 203 | TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f (zero-shot) | 5.00% | Dutch CoLA test split | Official leaderboard | |
| 204 | TrustLLMeu/trustllm-v2-8b@d8cc924fa550fbf9474f5251e8e2f92d1c1bf894 | 4.90% | Dutch CoLA test split | Official leaderboard | |
| 205 | fresh-electra-small | 4.70% | Dutch CoLA test split | Official leaderboard | |
| 206 | fresh-xlm-roberta-base | 4.62% | Dutch CoLA test split | Official leaderboard | |
| 207 | flwrlabs/Lizzy-7B (val) | 4.61% | Dutch CoLA test split | Official leaderboard | |
| 208 | NbAiLab/borealis2-e4b-preview (zero-shot, val) | 4.37% | Dutch CoLA test split | Official leaderboard | |
| 209 | syvai/claes-v1 (zero-shot) | 4.30% | Dutch CoLA test split | Official leaderboard | |
| 210 | convaiinnovations/laya (zero-shot) | 4.00% | Dutch CoLA test split | Official leaderboard | |
| 211 | swiss-ai/Apertus-v1.1-0.5B-Instruct (val) | 3.89% | Dutch CoLA test split | Official leaderboard | |
| 212 | occiglot/occiglot-7b-eu5 | 3.88% | Dutch CoLA test split | Official leaderboard | |
| 213 | TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f | 3.72% | Dutch CoLA test split | Official leaderboard | |
| 214 | Qwen/Qwen3-0.6B#no-thinking | 3.60% | Dutch CoLA test split | Official leaderboard | |
| 215 | OpenLLM-France/Luciole-1B-Instruct-1.1 (val) | 3.56% | Dutch CoLA test split | Official leaderboard | |
| 216 | LFM2-1.2B | 3.52% | Dutch CoLA test split | Jul 2025 | Official leaderboard |
| 217 | tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base | 3.37% | Dutch CoLA test split | Official leaderboard | |
| 218 | TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 (zero-shot) | 3.34% | Dutch CoLA test split | Official leaderboard | |
| 219 | JetBrains/Mellum2-12B-A2.5B-Thinking-SFT (val) | 3.09% | Dutch CoLA test split | Official leaderboard | |
| 220 | PleIAs/Pleias-350m-Preview | 2.98% | Dutch CoLA test split | Official leaderboard | |
| 221 | yulan-team/YuLan-Mini-Instruct | 2.90% | Dutch CoLA test split | Official leaderboard | |
| 222 | meta-llama/Llama-3.2-1B-Instruct | 2.78% | Dutch CoLA test split | Official leaderboard | |
| 223 | TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 | 2.62% | Dutch CoLA test split | Official leaderboard | |
| 224 | | 2.52% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 225 | | 2.36% | Dutch CoLA test split | Aug 2025 | Official leaderboard |
| 226 | | 2.35% | Dutch CoLA test split | Sep 2024 | Official leaderboard |
| 227 | OpenLLM-France/Luciole-1B-Base (val) | 2.26% | Dutch CoLA test split | Official leaderboard | |
| 228 | BSC-LT/salamandra-7b | 1.95% | Dutch CoLA test split | Official leaderboard | |
| 229 | PleIAs/Pleias-1.2b-Preview | 1.91% | Dutch CoLA test split | Official leaderboard | |
| 230 | TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 (zero-shot) | 1.82% | Dutch CoLA test split | Official leaderboard | |
| 231 | syvai/claes-preview (zero-shot) | 1.76% | Dutch CoLA test split | Official leaderboard | |
| 232 | JetBrains/Mellum2-12B-A2.5B-Thinking (val) | 1.62% | Dutch CoLA test split | Official leaderboard | |
| 233 | AIDC-AI/Marco-Mini-Global-Base (val) | 1.34% | Dutch CoLA test split | Official leaderboard | |
| 234 | PleIAs/Monad | 1.23% | Dutch CoLA test split | Official leaderboard | |
| 235 | LiquidAI/LFM2-350M | 1.08% | Dutch CoLA test split | Official leaderboard | |
| 236 | JetBrains/Mellum2-12B-A2.5B-Instruct-SFT (val) | 0.92% | Dutch CoLA test split | Official leaderboard | |
| 237 | | 0.70% | Dutch CoLA test split | Oct 2024 | Official leaderboard |
| 238 | TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 | 0.56% | Dutch CoLA test split | Official leaderboard | |
| 239 | | 0.46% | Dutch CoLA test split | Oct 2024 | Official leaderboard |
| 240 | HuggingFaceTB/SmolLM2-135M-Instruct | 0.40% | Dutch CoLA test split | Official leaderboard | |
| 241 | convaiinnovations/laya-multilingual (zero-shot) | 0.32% | Dutch CoLA test split | Official leaderboard | |
| 242 | google-bert/bert-base-multilingual-cased (val) | 0.23% | Dutch CoLA test split | Official leaderboard | |
| 243 | NbAiLab/borealis2-e2b-preview (zero-shot, val) | 0.23% | Dutch CoLA test split | Official leaderboard | |
| 244 | LiquidAI/LFM2-700M | 0.19% | Dutch CoLA test split | Official leaderboard | |
| 245 | swiss-ai/Apertus-v1.1-1.5B (zero-shot) | 0.15% | Dutch CoLA test split | Official leaderboard | |
| 246 | jaredpalmer/kev-0.5b (zero-shot) | 0.14% | Dutch CoLA test split | Official leaderboard | |
| 247 | HuggingFaceTB/SmolLM2-360M-Instruct | 0.06% | Dutch CoLA test split | Official leaderboard | |
| 248 | tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct | 0.06% | Dutch CoLA test split | Official leaderboard | |
| 249 | AI-Sweden-Models/Llama-3-8B-instruct (zero-shot, val) | 0% | Dutch CoLA test split | Official leaderboard | |
| 250 | JetBrains/Mellum2-12B-A2.5B-Base-Pretrain (val) | 0% | Dutch CoLA test split | Official leaderboard | |
| 251 | mistralai/Ministral-3-14B-Instruct-2512 (zero-shot, val) | 0% | Dutch CoLA test split | Official leaderboard | |
| 252 | mistralai/Ministral-3-3B-Instruct-2512 (zero-shot, val) | 0% | Dutch CoLA test split | Official leaderboard | |
| 253 | Muse-research/Muse-3B (val) | 0% | Dutch CoLA test split | Official leaderboard | |
| 254 | openGPT-X/Teuken-7B-base-v0.6 (val) | 0% | Dutch CoLA test split | Official leaderboard | |
| 255 | PleIAs/Baguettotron@refs/pr/6#no-thinking | 0% | Dutch CoLA test split | Official leaderboard | |
| 256 | swiss-ai/Apertus-v1.1-1.5B-Instruct (val) | 0% | Dutch CoLA test split | Official leaderboard | |
| 257 | utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview | 0% | Dutch CoLA test split | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 49.4% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 2 | | 48.5% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 3 | | 48.2% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 4 | | 45.8% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 5 | | 38% | Dutch CoLA test split | Jul 2024 | Official leaderboard |
| 6 | | 37.8% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 7 | gpt-oss-20b | 36.3% | Dutch CoLA test split | Aug 2025 | Official leaderboard |
| 8 | | 33.6% | Dutch CoLA test split | Aug 2025 | Official leaderboard |
| 9 | granite-4.0-micro | 31.5% | Dutch CoLA test split | Oct 2025 | Official leaderboard |
| 10 | | 31.1% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 11 | | 27.5% | Dutch CoLA test split | Dec 2025 | Official leaderboard |
| 12 | | 26.9% | Dutch CoLA test split | Dec 2025 | Official leaderboard |
| 13 | | 26.8% | Dutch CoLA test split | Jul 2024 | Official leaderboard |
| 14 | | 7.00% | Dutch CoLA test split | Apr 2025 | Official leaderboard |
| 15 | | 5.90% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 16 | LFM2-1.2B | 3.52% | Dutch CoLA test split | Jul 2025 | Official leaderboard |
| 17 | | 2.52% | Dutch CoLA test split | Mar 2025 | Official leaderboard |
| 18 | | 2.36% | Dutch CoLA test split | Aug 2025 | Official leaderboard |
| 19 | | 2.35% | Dutch CoLA test split | Sep 2024 | Official leaderboard |
| 20 | | 0.70% | Dutch CoLA test split | Oct 2024 | Official leaderboard |
| 21 | | 0.46% | Dutch CoLA test split | Oct 2024 | Official leaderboard |
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: EuroEval’s leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: EuroEval’s leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5%
- MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87%
- CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7%
- Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
- MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%
Data audit
9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Instruct. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
- Pass Independent runs Scores from the official leaderboard (257 entries).
- Pass Models scored 21 models we track, plus 236 other entries on its leaderboard.
- Warning Newest score The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Instruct.
- Note Frontier models None of the top 10 has a score. The strongest tested is gpt-oss-20b (#143).
- Pass Headroom The best is 49.4%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on EuroEval: 24 rows for models we track and 236 other entries at the last read.
- Pass Last read November 21st, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.