Benchmark Almanac

BLiMP-NL

gemini/gemini-3.7-flash (zero-shot, val) 64.8% tops its leaderboard Gemma 3 27B 49.4% leads the models we track
Not in model cards

Models compare Dutch sentence pairs that differ in grammatical acceptability.

EuroEval compares the labels with human or constructed reference judgments using the Matthews correlation coefficient.

Made by
Suijkerbuijk and colleagues
Released
July 2025
Kind
Reasoning in many languages
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Measures
Capability
Language
Dutch
In and out
Text → Text
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
Based on
BLiMP
Strongest model tested
gpt-oss-20b #143 on the capability index
Last model added
December 2025
Scores refreshed
November 21st, 2026
257
Entries on EuroEval’s leaderboard, 21 models we track

gemini/gemini-3.7-flash (zero-shot, val) tops the leaderboard with 64.8% Gemma 3 27B leads with 49.4% among models released by December 2025

Best score per entry from EuroEval’s leaderboard, read on November 21st, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Gemma 3 27B, at 49.4%. This view shows only models we track, each at its best.

The newest model with a score here came out in December 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

257 entries. Scroll sideways for the rest.

  1. 64.8 gemini/gemini-3.7-flash (zero-shot, val)
  2. 61.4 gemini/gemini-3.6-flash (zero-shot, val)
  3. 60.5 gpt-6.2-astra (zero-shot, val)
  4. 60.3 gpt-6.2-sol (zero-shot, val)
  5. 59.7 gemini/gemini-3.5-flash-lite (zero-shot, val)
  6. 58.6 mistralai/Mistral-Large-3-675B-Instruct-2512 (val)
  7. 57.8 openai/gpt-5.6-luna (zero-shot, val)
  8. 57.4 gemini/gemini-3-flash-preview#thinking (zero-shot, val)
  9. 57.3 NbAiLab/borealis2-31b-preview (val)
  10. 57.3 claude-sonnet-4-5-20250929#no-thinking (zero-shot, val)
  11. 56.4 gpt-4.1-2025-04-14 (zero-shot, val)
  12. 56.2 deepseek/deepseek-flash#no-thinking (val)
  13. 56.1 gpt-6.1-sol (zero-shot, val)
  14. 54.7 utter-project/EuroLLM-22B-Instruct-2512
  15. 54.6 zai-org/GLM-5.3-Flash (val)
  16. 54.3 gpt-6-astra (zero-shot, val)
  17. 54.2 gpt-6-sol (zero-shot, val)
  18. 54.1 gpt-5-mini-2025-08-07 (zero-shot, val)
  19. 54.0 gemini/gemini-3.1-flash-lite-preview (zero-shot, val)
  20. 53.7 mistralai/Mistral-Medium-3.5-128B (val)
  21. 53.0 google/gemma-4-31B-it (val)
  22. 52.9 claude-sonnet-4-5-20250929#thinking (zero-shot, val)
  23. 52.6 xai/grok-4-1-fast-reasoning (zero-shot, val)
  24. 52.2 mistralai/Mistral-Small-4-119B-2603 (val)
  25. 51.6 claude-sonnet-4-6 (zero-shot, val)
  26. 51.6 openai/gpt-5.4-mini-2026-03-17#none (zero-shot, val)
  27. 51.3 openai/gpt-5.4-mini-2026-03-17#low (zero-shot, val)
  28. 51.0 Qwen/Qwen3.6-27B-FP8 (val)
  29. 50.8 deepcogito/cogito-v1-preview-qwen-14B
  30. 50.6 mistralai/Mistral-Medium-3.5-128B (zero-shot, val)
  31. 50.6 openai/gpt-5.4-2026-03-05#none (zero-shot, val)
  32. 49.8 NbAiLab/borealis2-26b-a4b-preview (val)
  33. 49.7 speakleash/Bielik-11B-v3.1-Instruct (val)
  34. 49.6 mistralai/Ministral-3-14B-Base-2512 (val)
  35. 49.6 deepcogito/cogito-v2-preview-llama-70B
  36. 49.5 mistralai/Ministral-3-14B-Reasoning-2512 (val)
  37. 49.4 Gemma 3 27B
  38. 49.1 google/gemma-4-26B-A4B-it (val)
  39. 49.0 swiss-ai/Apertus-v1.5-70B (val)
  40. 48.9 Qwen/Qwen3.5-35B-A3B-FP8 (val)
  41. 48.8 gpt-6-luna (zero-shot, val)
  42. 48.7 NbAiLab/borealis2-26b-a4b-preview (zero-shot, val)
  43. 48.7 zai-org/GLM-5.3-Flash (zero-shot, val)
  44. 48.5 Gemma 3 12B PT
  45. 48.2 Gemma 3 27B PT
  46. 48.0 Qwen/Qwen3-14B#no-thinking
  47. 48.0 Qwen/Qwen3-32B#no-thinking
  48. 47.9 NbAiLab/borealis2-31b-preview (zero-shot, val)
  49. 47.7 gpt-5.6-sol (zero-shot, val)
  50. 47.5 deepcogito/cogito-v1-preview-qwen-32B
  51. 47.3 gemini/gemini-2.5-flash-lite#no-thinking (zero-shot, val)
  52. 46.6 amalia-llm/AMALIA-9B-0626-DPO (val)
  53. 46.4 deepseek/deepseek-flash#low (zero-shot, val)
  54. 46.3 gpt-5.2-2025-12-11 (zero-shot, val)
  55. 45.8 Gemma 3 12B
  56. 45.2 claude-haiku-4-5-20251001 (zero-shot, val)
  57. 45.1 gemini/gemini-2.5-flash-lite#thinking (zero-shot, val)
  58. 43.9 mistralai/Mistral-Small-3.1-24B-Base-2503
  59. 43.9 swiss-ai/Apertus-v1.5-8B (val)
  60. 43.7 openai/gpt-oss-120b (val)
  61. 43.1 speakleash/Bielik-11B-v3.0-Instruct (val)
  62. 43.1 openeurollm/prelude-long (val)
  63. 42.6 deepcogito/cogito-v1-preview-llama-8B
  64. 42.5 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (val)
  65. 42.5 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 (val)
  66. 42.2 latam-gpt/Llama-3.1-70B-LatamGPT-SFT-1.0 (val)
  67. 42.1 Qwen/Qwen3-8B#no-thinking
  68. 42.0 Qwen/Qwen3.5-9B (val)
  69. 41.9 gpt-5-nano-2025-08-07 (zero-shot, val)
  70. 41.6 openai/gpt-5.4-mini-2026-03-17#medium (zero-shot, val)
  71. 41.6 openai/gpt-5.4-nano-2026-03-17#medium (zero-shot, val)
  72. 41.3 allenai/Llama-3.1-Tulu-3-8B-SFT
  73. 41.3 google/gemma-4-12B-it (val)
  74. 41.2 mistralai/Mistral-Small-3.2-24B-Instruct-2506
  75. 40.9 openeurollm/prelude-base (val)
  76. 40.7 Qwen/Qwen3.8-27B (val)
  77. 40.2 gpt-5.6-terra (zero-shot, val)
  78. 39.9 speakleash/Bielik-Minitron-7B-v3.0-Instruct (val)
  79. 39.8 openai/gpt-5.4-nano-2026-03-17#low (zero-shot, val)
  80. 39.8 Qwen/Qwen3-4B#no-thinking
  81. 39.5 google/gemma-4-E4B-it (val)
  82. 39.3 mistralai/Mistral-Large-3-675B-Instruct-2512 (zero-shot, val)
  83. 39.1 speakleash/Bielik-One-11B-v3.1-r1@75a2d83890e3c68bf7fac2fe5bbe2bff6372036c (zero-shot)
  84. 38.9 nvidia/Nemotron-Cascade-2-30B-A3B (val)
  85. 38.7 zai-org/GLM-4.7-Flash (val)
  86. 38.6 swiss-ai/Apertus-v1.5-70B (zero-shot, val)
  87. 38.1 domyn/Domyn-Small-v1.0 (val)
  88. 38.0 mistralai/Ministral-3-8B-Reasoning-2512
  89. 38.0 Llama 3.1-70B
  90. 37.8 Gemma 3 4B
  91. 37.6 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 (val)
  92. 37.5 microsoft/mdeberta-v3-base
  93. 37.1 meta-llama/Llama-3.1-8B-Instruct
  94. 37.1 amalia-llm/AMALIA-9B-0626-SFT (val)
  95. 37.0 openai/gpt-5.4-nano-2026-03-17#high (zero-shot, val)
  96. 37.0 mistralai/Mistral-Small-3.1-24B-Instruct-2503
  97. 36.6 mistralai/Mistral-Small-4-119B-2603 (zero-shot, val)
  98. 36.3 gpt-oss-20b
  99. 36.3 Qwen/Qwen3.5-9B-Base (val)
  100. 36.3 mistralai/Ministral-3-14B-Instruct-2512 (val)
  101. 36.1 openai/gpt-5.4-mini-2026-03-17#high (zero-shot, val)
  102. 35.9 OpenLLM-France/Luciole-23B-Instruct-1.1 (val)
  103. 35.9 Qwen/Qwen3.5-35B-A3B-Base (val)
  104. 35.2 speakleash/Bielik-11B-v2.3-Instruct
  105. 35.2 NbAiLab/borealis2-e4b-preview (val)
  106. 34.7 CohereLabs/aya-expanse-8b (val)
  107. 33.9 utter-project/EuroLLM-9B-Instruct
  108. 33.6 Qwen3-4B-Thinking-2507
  109. 33.1 utter-project/EuroLLM-9B-Instruct-2512 (val)
  110. 32.4 OpenLLM-France/Luciole-8B-Instruct-1.1 (val)
  111. 32.2 google/gemma-4-31B (val)
  112. 31.6 Qwen/Qwen3.5-2B (val)
  113. 31.5 granite-4.0-micro
  114. 31.1 NbAiLab/borealis2-e2b-preview (val)
  115. 31.1 Gemma 3 4B PT
  116. 31.0 mii-llm/nesso-4B
  117. 30.7 allenai/Olmo-3-1125-32B
  118. 30.4 danish-foundation-models/DFM-Mimir-v1.8 (val)
  119. 29.8 Aleph-Alpha/Kolibri-1 (val)
  120. 29.8 google/gemma-4-E2B-it (val)
  121. 29.8 typesafe/jev-1.13.0 (zero-shot)
  122. 29.4 mistralai/Ministral-3-3B-Instruct-2512 (val)
  123. 29.0 deepcogito/cogito-v1-preview-llama-3B
  124. 28.8 allenai/Olmo-3.1-32B-Instruct-DPO
  125. 28.8 allenai/Olmo-3.1-32B-Instruct-SFT
  126. 28.5 danish-foundation-models/DFM-Mimir-v1.8 (zero-shot, val)
  127. 27.5 Olmo 3.1 32B Instruct
  128. 27.2 ibm-granite/granite-4.0-h-micro
  129. 27.1 Qwen/Qwen3.5-4B (val)
  130. 26.9 Ministral 3 3B Reasoning
  131. 26.8 Llama 3.1-8B
  132. 26.6 danish-foundation-models/DFM-Mimir-v1.5 (val)
  133. 25.9 danish-foundation-models/DFM-Mimir-v1.5 (zero-shot, val)
  134. 25.8 BSC-LT/salamandra-7b-instruct
  135. 25.5 amalia-llm/AMALIA-9B-0626-DPO (zero-shot, val)
  136. 24.8 amalia-llm/AMALIA-9B-0626-SFT (zero-shot, val)
  137. 24.6 swiss-ai/Apertus-v1.1-4B (val)
  138. 23.8 google/gemma-4-26B-A4B (val)
  139. 22.8 Qwen/Qwen3.5-2B-Base (val)
  140. 22.6 meta-llama/Llama-3.2-3B-Instruct
  141. 22.5 inclusionAI/Ling-mini-2.0 (val)
  142. 22.3 OpenLLM-France/Luciole-23B-Base (val)
  143. 22.0 google/gemma-4-E4B (val)
  144. 21.4 google/gemma-4-12B (val)
  145. 21.3 mistralai/Ministral-3-8B-Instruct-2512 (val)
  146. 21.2 Qwen/Qwen3.5-4B-Base (val)
  147. 20.5 LGAI-EXAONE/EXAONE-4.0-32B (val)
  148. 20.3 Qwen/Qwen3-1.7B#no-thinking
  149. 20.1 utter-project/EuroLLM-22B-2512
  150. 19.9 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
  151. 19.9 HuggingFaceTB/SmolLM3-3B-Base
  152. 19.8 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 (val)
  153. 19.6 danish-foundation-models/DFM-Mimir (val)
  154. 19.3 Qwen/Qwen3.5-0.8B-Base (val)
  155. 19.0 openai/gpt-5.4-nano-2026-03-17#none (zero-shot, val)
  156. 18.5 jhu-clsp/mmBERT-small
  157. 18.4 jhu-clsp/mmBERT-base
  158. 18.1 swiss-ai/Apertus-v1.5-8B (zero-shot, val)
  159. 17.6 allenai/Olmo-3-1025-7B
  160. 17.3 speakleash/Bielik-One-4B-r1@567d29b0f15df28794acf0f6187f9216860d4551 (zero-shot)
  161. 17.1 danish-foundation-models/DFM-Mimir-v1.8-Mini (val)
  162. 16.7 JetBrains/Mellum2-12B-A2.5B-Instruct (val)
  163. 16.2 openGPT-X/Teuken-7B-instruct-v0.6 (val)
  164. 15.7 HuggingFaceTB/SmolLM3-3B#no-thinking
  165. 15.7 mistralai/Ministral-3-8B-Base-2512 (val)
  166. 15.5 occiglot/occiglot-7b-eu5-instruct
  167. 15.2 JetBrains/Mellum2-12B-A2.5B-Base (val)
  168. 14.9 Qwen/Qwen3.5-0.8B (val)
  169. 14.9 swiss-ai/Apertus-v1.1-4B-Instruct (val)
  170. 14.8 danish-foundation-models/DFM-Mimir-v1.8-Mini (zero-shot, val)
  171. 14.3 allenai/Olmo-3-7B-Instruct
  172. 13.9 openeurollm/oellm-9b-128k-theta32m-v3 (val)
  173. 13.6 allenai/Olmo-3-7B-Instruct-DPO
  174. 13.6 danish-foundation-models/DFM-Mimir (zero-shot, val)
  175. 12.6 LiquidAI/LFM2.5-8B-A1B (val)
  176. 12.4 intfloat/multilingual-e5-base
  177. 11.8 gemini/gemini-3-flash-preview#no-thinking (zero-shot, val)
  178. 11.4 intfloat/multilingual-e5-large
  179. 11.3 allenai/Olmo-3-7B-Instruct-SFT
  180. 11.3 EuroBERT/EuroBERT-210m
  181. 11.3 LiquidAI/LFM2.5-8B-A1B-Base (val)
  182. 11.2 answerdotai/ModernBERT-large
  183. 11.1 EuroBERT/EuroBERT-610m
  184. 9.84 CohereLabs/aya-23-8B (val)
  185. 9.65 Aleph-Alpha/Kolibri-1 (zero-shot, val)
  186. 9.52 speakleash/Bielik-1.5B-v3.0-Instruct
  187. 9.41 utter-project/EuroLLM-9B
  188. 8.80 TrustLLMeu/trustllm-v2-8b-midtrain@3c7af30bc43bb3c8607e0fab6e7f2a23dd64af49
  189. 7.68 distilbert/distilbert-base-multilingual-cased
  190. 7.56 Geotrend/distilbert-base-25lang-cased
  191. 7.55 google/gemma-4-E2B (val)
  192. 7.00 Qwen3-0.6B
  193. 6.66 swiss-ai/Apertus-v1.1-0.5B (val)
  194. 6.56 answerdotai/ModernBERT-base
  195. 6.47 AI-Sweden-Models/bert-large-nordic-pile-1M-steps
  196. 6.40 swiss-ai/Apertus-v1.1-1.5B (val)
  197. 6.02 AI-Sweden-Models/Llama-3-8B-instruct (val)
  198. 5.99 OmniJev/OneJev1.6-0.8B (zero-shot)
  199. 5.92 fastino/GLiNER2.5-Decide (zero-shot)
  200. 5.90 Gemma 3 1B PT
  201. 5.74 EuropeanParliament/EUBERT
  202. 5.50 OpenLLM-France/Luciole-8B-Base (val)
  203. 5.00 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f (zero-shot)
  204. 4.90 TrustLLMeu/trustllm-v2-8b@d8cc924fa550fbf9474f5251e8e2f92d1c1bf894
  205. 4.70 fresh-electra-small
  206. 4.62 fresh-xlm-roberta-base
  207. 4.61 flwrlabs/Lizzy-7B (val)
  208. 4.37 NbAiLab/borealis2-e4b-preview (zero-shot, val)
  209. 4.30 syvai/claes-v1 (zero-shot)
  210. 4.00 convaiinnovations/laya (zero-shot)
  211. 3.89 swiss-ai/Apertus-v1.1-0.5B-Instruct (val)
  212. 3.88 occiglot/occiglot-7b-eu5
  213. 3.72 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f
  214. 3.60 Qwen/Qwen3-0.6B#no-thinking
  215. 3.56 OpenLLM-France/Luciole-1B-Instruct-1.1 (val)
  216. 3.52 LFM2-1.2B
  217. 3.37 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base
  218. 3.34 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 (zero-shot)
  219. 3.09 JetBrains/Mellum2-12B-A2.5B-Thinking-SFT (val)
  220. 2.98 PleIAs/Pleias-350m-Preview
  221. 2.90 yulan-team/YuLan-Mini-Instruct
  222. 2.78 meta-llama/Llama-3.2-1B-Instruct
  223. 2.62 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6
  224. 2.52 Gemma 3 1B
  225. 2.36 Gemma 3 270M
  226. 2.35 Llama 3.2 1B
  227. 2.26 OpenLLM-France/Luciole-1B-Base (val)
  228. 1.95 BSC-LT/salamandra-7b
  229. 1.91 PleIAs/Pleias-1.2b-Preview
  230. 1.82 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 (zero-shot)
  231. 1.76 syvai/claes-preview (zero-shot)
  232. 1.62 JetBrains/Mellum2-12B-A2.5B-Thinking (val)
  233. 1.34 AIDC-AI/Marco-Mini-Global-Base (val)
  234. 1.23 PleIAs/Monad
  235. 1.08 LiquidAI/LFM2-350M
  236. 0.92 JetBrains/Mellum2-12B-A2.5B-Instruct-SFT (val)
  237. 0.70 SmolLM2-135M
  238. 0.56 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660
  239. 0.46 SmolLM2-360M
  240. 0.40 HuggingFaceTB/SmolLM2-135M-Instruct
  241. 0.32 convaiinnovations/laya-multilingual (zero-shot)
  242. 0.23 google-bert/bert-base-multilingual-cased (val)
  243. 0.23 NbAiLab/borealis2-e2b-preview (zero-shot, val)
  244. 0.19 LiquidAI/LFM2-700M
  245. 0.15 swiss-ai/Apertus-v1.1-1.5B (zero-shot)
  246. 0.14 jaredpalmer/kev-0.5b (zero-shot)
  247. 0.06 HuggingFaceTB/SmolLM2-360M-Instruct
  248. 0.06 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct
  249. 0.0 AI-Sweden-Models/Llama-3-8B-instruct (zero-shot, val)
  250. 0.0 JetBrains/Mellum2-12B-A2.5B-Base-Pretrain (val)
  251. 0.0 mistralai/Ministral-3-14B-Instruct-2512 (zero-shot, val)
  252. 0.0 mistralai/Ministral-3-3B-Instruct-2512 (zero-shot, val)
  253. 0.0 Muse-research/Muse-3B (val)
  254. 0.0 openGPT-X/Teuken-7B-base-v0.6 (val)
  255. 0.0 PleIAs/Baguettotron@refs/pr/6#no-thinking
  256. 0.0 swiss-ai/Apertus-v1.1-1.5B-Instruct (val)
  257. 0.0 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview
  1. 1 gemini/gemini-3.7-flash (zero-shot, val) Dutch CoLA test split 64.8%
  2. 2 gemini/gemini-3.6-flash (zero-shot, val) Dutch CoLA test split 61.4%
  3. 3 gpt-6.2-astra (zero-shot, val) Dutch CoLA test split 60.5%
  4. 4 gpt-6.2-sol (zero-shot, val) Dutch CoLA test split 60.3%
  5. 5 gemini/gemini-3.5-flash-lite (zero-shot, val) Dutch CoLA test split 59.7%
  6. 6 mistralai/Mistral-Large-3-675B-Instruct-2512 (val) Dutch CoLA test split 58.6%
  7. 7 openai/gpt-5.6-luna (zero-shot, val) Dutch CoLA test split 57.8%
  8. 8 gemini/gemini-3-flash-preview#thinking (zero-shot, val) Dutch CoLA test split 57.4%
  9. 9 NbAiLab/borealis2-31b-preview (val) Dutch CoLA test split 57.3%
  10. 10 claude-sonnet-4-5-20250929#no-thinking (zero-shot, val) Dutch CoLA test split 57.3%
  11. 11 gpt-4.1-2025-04-14 (zero-shot, val) Dutch CoLA test split 56.4%
  12. 12 deepseek/deepseek-flash#no-thinking (val) Dutch CoLA test split 56.2%
  13. 13 gpt-6.1-sol (zero-shot, val) Dutch CoLA test split 56.1%
  14. 14 utter-project/EuroLLM-22B-Instruct-2512 Dutch CoLA test split 54.7%
  15. 15 zai-org/GLM-5.3-Flash (val) Dutch CoLA test split 54.6%
  16. 16 gpt-6-astra (zero-shot, val) Dutch CoLA test split 54.3%
  17. 17 gpt-6-sol (zero-shot, val) Dutch CoLA test split 54.2%
  18. 18 gpt-5-mini-2025-08-07 (zero-shot, val) Dutch CoLA test split 54.1%
  19. 19 gemini/gemini-3.1-flash-lite-preview (zero-shot, val) Dutch CoLA test split 54%
  20. 20 mistralai/Mistral-Medium-3.5-128B (val) Dutch CoLA test split 53.7%
  21. 21 google/gemma-4-31B-it (val) Dutch CoLA test split 53%
  22. 22 claude-sonnet-4-5-20250929#thinking (zero-shot, val) Dutch CoLA test split 52.9%
  23. 23 xai/grok-4-1-fast-reasoning (zero-shot, val) Dutch CoLA test split 52.6%
  24. 24 mistralai/Mistral-Small-4-119B-2603 (val) Dutch CoLA test split 52.2%
  25. 25 claude-sonnet-4-6 (zero-shot, val) Dutch CoLA test split 51.6%
  26. 26 openai/gpt-5.4-mini-2026-03-17#none (zero-shot, val) Dutch CoLA test split 51.6%
  27. 27 openai/gpt-5.4-mini-2026-03-17#low (zero-shot, val) Dutch CoLA test split 51.3%
  28. 28 Qwen/Qwen3.6-27B-FP8 (val) Dutch CoLA test split 51%
  29. 29 deepcogito/cogito-v1-preview-qwen-14B Dutch CoLA test split 50.8%
  30. 30 mistralai/Mistral-Medium-3.5-128B (zero-shot, val) Dutch CoLA test split 50.6%
  31. 31 openai/gpt-5.4-2026-03-05#none (zero-shot, val) Dutch CoLA test split 50.6%
  32. 32 NbAiLab/borealis2-26b-a4b-preview (val) Dutch CoLA test split 49.8%
  33. 33 speakleash/Bielik-11B-v3.1-Instruct (val) Dutch CoLA test split 49.7%
  34. 34 mistralai/Ministral-3-14B-Base-2512 (val) Dutch CoLA test split 49.6%
  35. 35 deepcogito/cogito-v2-preview-llama-70B Dutch CoLA test split 49.6%
  36. 36 mistralai/Ministral-3-14B-Reasoning-2512 (val) Dutch CoLA test split 49.5%
  37. 37 Gemma 3 27B Dutch CoLA test split 49.4%
  38. 38 google/gemma-4-26B-A4B-it (val) Dutch CoLA test split 49.1%
  39. 39 swiss-ai/Apertus-v1.5-70B (val) Dutch CoLA test split 49%
  40. 40 Qwen/Qwen3.5-35B-A3B-FP8 (val) Dutch CoLA test split 48.9%
  41. 41 gpt-6-luna (zero-shot, val) Dutch CoLA test split 48.8%
  42. 42 NbAiLab/borealis2-26b-a4b-preview (zero-shot, val) Dutch CoLA test split 48.7%
  43. 43 zai-org/GLM-5.3-Flash (zero-shot, val) Dutch CoLA test split 48.7%
  44. 44 Gemma 3 12B PT Dutch CoLA test split 48.5%
  45. 45 Gemma 3 27B PT Dutch CoLA test split 48.2%
  46. 46 Qwen/Qwen3-14B#no-thinking Dutch CoLA test split 48%
  47. 47 Qwen/Qwen3-32B#no-thinking Dutch CoLA test split 48%
  48. 48 NbAiLab/borealis2-31b-preview (zero-shot, val) Dutch CoLA test split 47.9%
  49. 49 gpt-5.6-sol (zero-shot, val) Dutch CoLA test split 47.7%
  50. 50 deepcogito/cogito-v1-preview-qwen-32B Dutch CoLA test split 47.5%
  51. 51 gemini/gemini-2.5-flash-lite#no-thinking (zero-shot, val) Dutch CoLA test split 47.3%
  52. 52 amalia-llm/AMALIA-9B-0626-DPO (val) Dutch CoLA test split 46.6%
  53. 53 deepseek/deepseek-flash#low (zero-shot, val) Dutch CoLA test split 46.4%
  54. 54 gpt-5.2-2025-12-11 (zero-shot, val) Dutch CoLA test split 46.3%
  55. 55 Gemma 3 12B Dutch CoLA test split 45.8%
  56. 56 claude-haiku-4-5-20251001 (zero-shot, val) Dutch CoLA test split 45.2%
  57. 57 gemini/gemini-2.5-flash-lite#thinking (zero-shot, val) Dutch CoLA test split 45.1%
  58. 58 mistralai/Mistral-Small-3.1-24B-Base-2503 Dutch CoLA test split 43.9%
  59. 59 swiss-ai/Apertus-v1.5-8B (val) Dutch CoLA test split 43.9%
  60. 60 openai/gpt-oss-120b (val) Dutch CoLA test split 43.7%
  61. 61 speakleash/Bielik-11B-v3.0-Instruct (val) Dutch CoLA test split 43.1%
  62. 62 openeurollm/prelude-long (val) Dutch CoLA test split 43.1%
  63. 63 deepcogito/cogito-v1-preview-llama-8B Dutch CoLA test split 42.6%
  64. 64 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (val) Dutch CoLA test split 42.5%
  65. 65 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 (val) Dutch CoLA test split 42.5%
  66. 66 latam-gpt/Llama-3.1-70B-LatamGPT-SFT-1.0 (val) Dutch CoLA test split 42.2%
  67. 67 Qwen/Qwen3-8B#no-thinking Dutch CoLA test split 42.1%
  68. 68 Qwen/Qwen3.5-9B (val) Dutch CoLA test split 42%
  69. 69 gpt-5-nano-2025-08-07 (zero-shot, val) Dutch CoLA test split 41.9%
  70. 70 openai/gpt-5.4-mini-2026-03-17#medium (zero-shot, val) Dutch CoLA test split 41.6%
  71. 71 openai/gpt-5.4-nano-2026-03-17#medium (zero-shot, val) Dutch CoLA test split 41.6%
  72. 72 allenai/Llama-3.1-Tulu-3-8B-SFT Dutch CoLA test split 41.3%
  73. 73 google/gemma-4-12B-it (val) Dutch CoLA test split 41.3%
  74. 74 mistralai/Mistral-Small-3.2-24B-Instruct-2506 Dutch CoLA test split 41.2%
  75. 75 openeurollm/prelude-base (val) Dutch CoLA test split 40.9%
  76. 76 Qwen/Qwen3.8-27B (val) Dutch CoLA test split 40.7%
  77. 77 gpt-5.6-terra (zero-shot, val) Dutch CoLA test split 40.2%
  78. 78 speakleash/Bielik-Minitron-7B-v3.0-Instruct (val) Dutch CoLA test split 39.9%
  79. 79 openai/gpt-5.4-nano-2026-03-17#low (zero-shot, val) Dutch CoLA test split 39.8%
  80. 80 Qwen/Qwen3-4B#no-thinking Dutch CoLA test split 39.8%
  81. 81 google/gemma-4-E4B-it (val) Dutch CoLA test split 39.5%
  82. 82 mistralai/Mistral-Large-3-675B-Instruct-2512 (zero-shot, val) Dutch CoLA test split 39.3%
  83. 83 speakleash/Bielik-One-11B-v3.1-r1@75a2d83890e3c68bf7fac2fe5bbe2bff6372036c (zero-shot) Dutch CoLA test split 39.1%
  84. 84 nvidia/Nemotron-Cascade-2-30B-A3B (val) Dutch CoLA test split 38.9%
  85. 85 zai-org/GLM-4.7-Flash (val) Dutch CoLA test split 38.7%
  86. 86 swiss-ai/Apertus-v1.5-70B (zero-shot, val) Dutch CoLA test split 38.6%
  87. 87 domyn/Domyn-Small-v1.0 (val) Dutch CoLA test split 38.1%
  88. 88 mistralai/Ministral-3-8B-Reasoning-2512 Dutch CoLA test split 38%
  89. 89 Llama 3.1-70B Dutch CoLA test split 38%
  90. 90 Gemma 3 4B Dutch CoLA test split 37.8%
  91. 91 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 (val) Dutch CoLA test split 37.6%
  92. 92 microsoft/mdeberta-v3-base Dutch CoLA test split 37.5%
  93. 93 meta-llama/Llama-3.1-8B-Instruct Dutch CoLA test split 37.1%
  94. 94 amalia-llm/AMALIA-9B-0626-SFT (val) Dutch CoLA test split 37.1%
  95. 95 openai/gpt-5.4-nano-2026-03-17#high (zero-shot, val) Dutch CoLA test split 37%
  96. 96 mistralai/Mistral-Small-3.1-24B-Instruct-2503 Dutch CoLA test split 37%
  97. 97 mistralai/Mistral-Small-4-119B-2603 (zero-shot, val) Dutch CoLA test split 36.6%
  98. 98 gpt-oss-20b Dutch CoLA test split 36.3%
  99. 99 Qwen/Qwen3.5-9B-Base (val) Dutch CoLA test split 36.3%
  100. 100 mistralai/Ministral-3-14B-Instruct-2512 (val) Dutch CoLA test split 36.3%
  101. 101 openai/gpt-5.4-mini-2026-03-17#high (zero-shot, val) Dutch CoLA test split 36.1%
  102. 102 OpenLLM-France/Luciole-23B-Instruct-1.1 (val) Dutch CoLA test split 35.9%
  103. 103 Qwen/Qwen3.5-35B-A3B-Base (val) Dutch CoLA test split 35.9%
  104. 104 speakleash/Bielik-11B-v2.3-Instruct Dutch CoLA test split 35.2%
  105. 105 NbAiLab/borealis2-e4b-preview (val) Dutch CoLA test split 35.2%
  106. 106 CohereLabs/aya-expanse-8b (val) Dutch CoLA test split 34.7%
  107. 107 utter-project/EuroLLM-9B-Instruct Dutch CoLA test split 33.9%
  108. 108 Qwen3-4B-Thinking-2507 Dutch CoLA test split 33.6%
  109. 109 utter-project/EuroLLM-9B-Instruct-2512 (val) Dutch CoLA test split 33.1%
  110. 110 OpenLLM-France/Luciole-8B-Instruct-1.1 (val) Dutch CoLA test split 32.4%
  111. 111 google/gemma-4-31B (val) Dutch CoLA test split 32.2%
  112. 112 Qwen/Qwen3.5-2B (val) Dutch CoLA test split 31.6%
  113. 113 granite-4.0-micro Dutch CoLA test split 31.5%
  114. 114 NbAiLab/borealis2-e2b-preview (val) Dutch CoLA test split 31.1%
  115. 115 Gemma 3 4B PT Dutch CoLA test split 31.1%
  116. 116 mii-llm/nesso-4B Dutch CoLA test split 31%
  117. 117 allenai/Olmo-3-1125-32B Dutch CoLA test split 30.7%
  118. 118 danish-foundation-models/DFM-Mimir-v1.8 (val) Dutch CoLA test split 30.4%
  119. 119 Aleph-Alpha/Kolibri-1 (val) Dutch CoLA test split 29.8%
  120. 120 google/gemma-4-E2B-it (val) Dutch CoLA test split 29.8%
  121. 121 typesafe/jev-1.13.0 (zero-shot) Dutch CoLA test split 29.8%
  122. 122 mistralai/Ministral-3-3B-Instruct-2512 (val) Dutch CoLA test split 29.4%
  123. 123 deepcogito/cogito-v1-preview-llama-3B Dutch CoLA test split 29%
  124. 124 allenai/Olmo-3.1-32B-Instruct-DPO Dutch CoLA test split 28.8%
  125. 125 allenai/Olmo-3.1-32B-Instruct-SFT Dutch CoLA test split 28.8%
  126. 126 danish-foundation-models/DFM-Mimir-v1.8 (zero-shot, val) Dutch CoLA test split 28.5%
  127. 127 Olmo 3.1 32B Instruct Dutch CoLA test split 27.5%
  128. 128 ibm-granite/granite-4.0-h-micro Dutch CoLA test split 27.2%
  129. 129 Qwen/Qwen3.5-4B (val) Dutch CoLA test split 27.1%
  130. 130 Ministral 3 3B Reasoning Dutch CoLA test split 26.9%
  131. 131 Llama 3.1-8B Dutch CoLA test split 26.8%
  132. 132 danish-foundation-models/DFM-Mimir-v1.5 (val) Dutch CoLA test split 26.6%
  133. 133 danish-foundation-models/DFM-Mimir-v1.5 (zero-shot, val) Dutch CoLA test split 25.9%
  134. 134 BSC-LT/salamandra-7b-instruct Dutch CoLA test split 25.8%
  135. 135 amalia-llm/AMALIA-9B-0626-DPO (zero-shot, val) Dutch CoLA test split 25.5%
  136. 136 amalia-llm/AMALIA-9B-0626-SFT (zero-shot, val) Dutch CoLA test split 24.8%
  137. 137 swiss-ai/Apertus-v1.1-4B (val) Dutch CoLA test split 24.6%
  138. 138 google/gemma-4-26B-A4B (val) Dutch CoLA test split 23.8%
  139. 139 Qwen/Qwen3.5-2B-Base (val) Dutch CoLA test split 22.8%
  140. 140 meta-llama/Llama-3.2-3B-Instruct Dutch CoLA test split 22.6%
  141. 141 inclusionAI/Ling-mini-2.0 (val) Dutch CoLA test split 22.5%
  142. 142 OpenLLM-France/Luciole-23B-Base (val) Dutch CoLA test split 22.3%
  143. 143 google/gemma-4-E4B (val) Dutch CoLA test split 22%
  144. 144 google/gemma-4-12B (val) Dutch CoLA test split 21.4%
  145. 145 mistralai/Ministral-3-8B-Instruct-2512 (val) Dutch CoLA test split 21.3%
  146. 146 Qwen/Qwen3.5-4B-Base (val) Dutch CoLA test split 21.2%
  147. 147 LGAI-EXAONE/EXAONE-4.0-32B (val) Dutch CoLA test split 20.5%
  148. 148 Qwen/Qwen3-1.7B#no-thinking Dutch CoLA test split 20.3%
  149. 149 utter-project/EuroLLM-22B-2512 Dutch CoLA test split 20.1%
  150. 150 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 Dutch CoLA test split 19.9%
  151. 151 HuggingFaceTB/SmolLM3-3B-Base Dutch CoLA test split 19.9%
  152. 152 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 (val) Dutch CoLA test split 19.8%
  153. 153 danish-foundation-models/DFM-Mimir (val) Dutch CoLA test split 19.6%
  154. 154 Qwen/Qwen3.5-0.8B-Base (val) Dutch CoLA test split 19.3%
  155. 155 openai/gpt-5.4-nano-2026-03-17#none (zero-shot, val) Dutch CoLA test split 19%
  156. 156 jhu-clsp/mmBERT-small Dutch CoLA test split 18.5%
  157. 157 jhu-clsp/mmBERT-base Dutch CoLA test split 18.4%
  158. 158 swiss-ai/Apertus-v1.5-8B (zero-shot, val) Dutch CoLA test split 18.1%
  159. 159 allenai/Olmo-3-1025-7B Dutch CoLA test split 17.6%
  160. 160 speakleash/Bielik-One-4B-r1@567d29b0f15df28794acf0f6187f9216860d4551 (zero-shot) Dutch CoLA test split 17.3%
  161. 161 danish-foundation-models/DFM-Mimir-v1.8-Mini (val) Dutch CoLA test split 17.1%
  162. 162 JetBrains/Mellum2-12B-A2.5B-Instruct (val) Dutch CoLA test split 16.7%
  163. 163 openGPT-X/Teuken-7B-instruct-v0.6 (val) Dutch CoLA test split 16.2%
  164. 164 HuggingFaceTB/SmolLM3-3B#no-thinking Dutch CoLA test split 15.7%
  165. 165 mistralai/Ministral-3-8B-Base-2512 (val) Dutch CoLA test split 15.7%
  166. 166 occiglot/occiglot-7b-eu5-instruct Dutch CoLA test split 15.5%
  167. 167 JetBrains/Mellum2-12B-A2.5B-Base (val) Dutch CoLA test split 15.2%
  168. 168 Qwen/Qwen3.5-0.8B (val) Dutch CoLA test split 14.9%
  169. 169 swiss-ai/Apertus-v1.1-4B-Instruct (val) Dutch CoLA test split 14.9%
  170. 170 danish-foundation-models/DFM-Mimir-v1.8-Mini (zero-shot, val) Dutch CoLA test split 14.8%
  171. 171 allenai/Olmo-3-7B-Instruct Dutch CoLA test split 14.3%
  172. 172 openeurollm/oellm-9b-128k-theta32m-v3 (val) Dutch CoLA test split 13.9%
  173. 173 allenai/Olmo-3-7B-Instruct-DPO Dutch CoLA test split 13.6%
  174. 174 danish-foundation-models/DFM-Mimir (zero-shot, val) Dutch CoLA test split 13.6%
  175. 175 LiquidAI/LFM2.5-8B-A1B (val) Dutch CoLA test split 12.6%
  176. 176 intfloat/multilingual-e5-base Dutch CoLA test split 12.4%
  177. 177 gemini/gemini-3-flash-preview#no-thinking (zero-shot, val) Dutch CoLA test split 11.8%
  178. 178 intfloat/multilingual-e5-large Dutch CoLA test split 11.4%
  179. 179 allenai/Olmo-3-7B-Instruct-SFT Dutch CoLA test split 11.3%
  180. 180 EuroBERT/EuroBERT-210m Dutch CoLA test split 11.3%
  181. 181 LiquidAI/LFM2.5-8B-A1B-Base (val) Dutch CoLA test split 11.3%
  182. 182 answerdotai/ModernBERT-large Dutch CoLA test split 11.2%
  183. 183 EuroBERT/EuroBERT-610m Dutch CoLA test split 11.1%
  184. 184 CohereLabs/aya-23-8B (val) Dutch CoLA test split 9.84%
  185. 185 Aleph-Alpha/Kolibri-1 (zero-shot, val) Dutch CoLA test split 9.65%
  186. 186 speakleash/Bielik-1.5B-v3.0-Instruct Dutch CoLA test split 9.52%
  187. 187 utter-project/EuroLLM-9B Dutch CoLA test split 9.41%
  188. 188 TrustLLMeu/trustllm-v2-8b-midtrain@3c7af30bc43bb3c8607e0fab6e7f2a23dd64af49 Dutch CoLA test split 8.80%
  189. 189 distilbert/distilbert-base-multilingual-cased Dutch CoLA test split 7.68%
  190. 190 Geotrend/distilbert-base-25lang-cased Dutch CoLA test split 7.56%
  191. 191 google/gemma-4-E2B (val) Dutch CoLA test split 7.55%
  192. 192 Qwen3-0.6B Dutch CoLA test split 7.00%
  193. 193 swiss-ai/Apertus-v1.1-0.5B (val) Dutch CoLA test split 6.66%
  194. 194 answerdotai/ModernBERT-base Dutch CoLA test split 6.56%
  195. 195 AI-Sweden-Models/bert-large-nordic-pile-1M-steps Dutch CoLA test split 6.47%
  196. 196 swiss-ai/Apertus-v1.1-1.5B (val) Dutch CoLA test split 6.40%
  197. 197 AI-Sweden-Models/Llama-3-8B-instruct (val) Dutch CoLA test split 6.02%
  198. 198 OmniJev/OneJev1.6-0.8B (zero-shot) Dutch CoLA test split 5.99%
  199. 199 fastino/GLiNER2.5-Decide (zero-shot) Dutch CoLA test split 5.92%
  200. 200 Gemma 3 1B PT Dutch CoLA test split 5.90%
  201. 201 EuropeanParliament/EUBERT Dutch CoLA test split 5.74%
  202. 202 OpenLLM-France/Luciole-8B-Base (val) Dutch CoLA test split 5.50%
  203. 203 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f (zero-shot) Dutch CoLA test split 5.00%
  204. 204 TrustLLMeu/trustllm-v2-8b@d8cc924fa550fbf9474f5251e8e2f92d1c1bf894 Dutch CoLA test split 4.90%
  205. 205 fresh-electra-small Dutch CoLA test split 4.70%
  206. 206 fresh-xlm-roberta-base Dutch CoLA test split 4.62%
  207. 207 flwrlabs/Lizzy-7B (val) Dutch CoLA test split 4.61%
  208. 208 NbAiLab/borealis2-e4b-preview (zero-shot, val) Dutch CoLA test split 4.37%
  209. 209 syvai/claes-v1 (zero-shot) Dutch CoLA test split 4.30%
  210. 210 convaiinnovations/laya (zero-shot) Dutch CoLA test split 4.00%
  211. 211 swiss-ai/Apertus-v1.1-0.5B-Instruct (val) Dutch CoLA test split 3.89%
  212. 212 occiglot/occiglot-7b-eu5 Dutch CoLA test split 3.88%
  213. 213 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f Dutch CoLA test split 3.72%
  214. 214 Qwen/Qwen3-0.6B#no-thinking Dutch CoLA test split 3.60%
  215. 215 OpenLLM-France/Luciole-1B-Instruct-1.1 (val) Dutch CoLA test split 3.56%
  216. 216 LFM2-1.2B Dutch CoLA test split 3.52%
  217. 217 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base Dutch CoLA test split 3.37%
  218. 218 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 (zero-shot) Dutch CoLA test split 3.34%
  219. 219 JetBrains/Mellum2-12B-A2.5B-Thinking-SFT (val) Dutch CoLA test split 3.09%
  220. 220 PleIAs/Pleias-350m-Preview Dutch CoLA test split 2.98%
  221. 221 yulan-team/YuLan-Mini-Instruct Dutch CoLA test split 2.90%
  222. 222 meta-llama/Llama-3.2-1B-Instruct Dutch CoLA test split 2.78%
  223. 223 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 Dutch CoLA test split 2.62%
  224. 224 Gemma 3 1B Dutch CoLA test split 2.52%
  225. 225 Gemma 3 270M Dutch CoLA test split 2.36%
  226. 226 Llama 3.2 1B Dutch CoLA test split 2.35%
  227. 227 OpenLLM-France/Luciole-1B-Base (val) Dutch CoLA test split 2.26%
  228. 228 BSC-LT/salamandra-7b Dutch CoLA test split 1.95%
  229. 229 PleIAs/Pleias-1.2b-Preview Dutch CoLA test split 1.91%
  230. 230 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 (zero-shot) Dutch CoLA test split 1.82%
  231. 231 syvai/claes-preview (zero-shot) Dutch CoLA test split 1.76%
  232. 232 JetBrains/Mellum2-12B-A2.5B-Thinking (val) Dutch CoLA test split 1.62%
  233. 233 AIDC-AI/Marco-Mini-Global-Base (val) Dutch CoLA test split 1.34%
  234. 234 PleIAs/Monad Dutch CoLA test split 1.23%
  235. 235 LiquidAI/LFM2-350M Dutch CoLA test split 1.08%
  236. 236 JetBrains/Mellum2-12B-A2.5B-Instruct-SFT (val) Dutch CoLA test split 0.92%
  237. 237 SmolLM2-135M Dutch CoLA test split 0.70%
  238. 238 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 Dutch CoLA test split 0.56%
  239. 239 SmolLM2-360M Dutch CoLA test split 0.46%
  240. 240 HuggingFaceTB/SmolLM2-135M-Instruct Dutch CoLA test split 0.40%
  241. 241 convaiinnovations/laya-multilingual (zero-shot) Dutch CoLA test split 0.32%
  242. 242 google-bert/bert-base-multilingual-cased (val) Dutch CoLA test split 0.23%
  243. 243 NbAiLab/borealis2-e2b-preview (zero-shot, val) Dutch CoLA test split 0.23%
  244. 244 LiquidAI/LFM2-700M Dutch CoLA test split 0.19%
  245. 245 swiss-ai/Apertus-v1.1-1.5B (zero-shot) Dutch CoLA test split 0.15%
  246. 246 jaredpalmer/kev-0.5b (zero-shot) Dutch CoLA test split 0.14%
  247. 247 HuggingFaceTB/SmolLM2-360M-Instruct Dutch CoLA test split 0.06%
  248. 248 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct Dutch CoLA test split 0.06%
  249. 249 AI-Sweden-Models/Llama-3-8B-instruct (zero-shot, val) Dutch CoLA test split 0%
  250. 250 JetBrains/Mellum2-12B-A2.5B-Base-Pretrain (val) Dutch CoLA test split 0%
  251. 251 mistralai/Ministral-3-14B-Instruct-2512 (zero-shot, val) Dutch CoLA test split 0%
  252. 252 mistralai/Ministral-3-3B-Instruct-2512 (zero-shot, val) Dutch CoLA test split 0%
  253. 253 Muse-research/Muse-3B (val) Dutch CoLA test split 0%
  254. 254 openGPT-X/Teuken-7B-base-v0.6 (val) Dutch CoLA test split 0%
  255. 255 PleIAs/Baguettotron@refs/pr/6#no-thinking Dutch CoLA test split 0%
  256. 256 swiss-ai/Apertus-v1.1-1.5B-Instruct (val) Dutch CoLA test split 0%
  257. 257 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview Dutch CoLA test split 0%

Source: EuroEval’s leaderboard

#EntryScoreSettingDateSource
1 gemini/gemini-3.7-flash (zero-shot, val) 64.8% Dutch CoLA test split Official leaderboard
2 gemini/gemini-3.6-flash (zero-shot, val) 61.4% Dutch CoLA test split Official leaderboard
3 gpt-6.2-astra (zero-shot, val) 60.5% Dutch CoLA test split Official leaderboard
4 gpt-6.2-sol (zero-shot, val) 60.3% Dutch CoLA test split Official leaderboard
5 gemini/gemini-3.5-flash-lite (zero-shot, val) 59.7% Dutch CoLA test split Official leaderboard
6 mistralai/Mistral-Large-3-675B-Instruct-2512 (val) 58.6% Dutch CoLA test split Official leaderboard
7 openai/gpt-5.6-luna (zero-shot, val) 57.8% Dutch CoLA test split Official leaderboard
8 gemini/gemini-3-flash-preview#thinking (zero-shot, val) 57.4% Dutch CoLA test split Official leaderboard
9 NbAiLab/borealis2-31b-preview (val) 57.3% Dutch CoLA test split Official leaderboard
10 claude-sonnet-4-5-20250929#no-thinking (zero-shot, val) 57.3% Dutch CoLA test split Official leaderboard
11 gpt-4.1-2025-04-14 (zero-shot, val) 56.4% Dutch CoLA test split Official leaderboard
12 deepseek/deepseek-flash#no-thinking (val) 56.2% Dutch CoLA test split Official leaderboard
13 gpt-6.1-sol (zero-shot, val) 56.1% Dutch CoLA test split Official leaderboard
14 utter-project/EuroLLM-22B-Instruct-2512 54.7% Dutch CoLA test split Official leaderboard
15 zai-org/GLM-5.3-Flash (val) 54.6% Dutch CoLA test split Official leaderboard
16 gpt-6-astra (zero-shot, val) 54.3% Dutch CoLA test split Official leaderboard
17 gpt-6-sol (zero-shot, val) 54.2% Dutch CoLA test split Official leaderboard
18 gpt-5-mini-2025-08-07 (zero-shot, val) 54.1% Dutch CoLA test split Official leaderboard
19 gemini/gemini-3.1-flash-lite-preview (zero-shot, val) 54% Dutch CoLA test split Official leaderboard
20 mistralai/Mistral-Medium-3.5-128B (val) 53.7% Dutch CoLA test split Official leaderboard
21 google/gemma-4-31B-it (val) 53% Dutch CoLA test split Official leaderboard
22 claude-sonnet-4-5-20250929#thinking (zero-shot, val) 52.9% Dutch CoLA test split Official leaderboard
23 xai/grok-4-1-fast-reasoning (zero-shot, val) 52.6% Dutch CoLA test split Official leaderboard
24 mistralai/Mistral-Small-4-119B-2603 (val) 52.2% Dutch CoLA test split Official leaderboard
25 claude-sonnet-4-6 (zero-shot, val) 51.6% Dutch CoLA test split Official leaderboard
26 openai/gpt-5.4-mini-2026-03-17#none (zero-shot, val) 51.6% Dutch CoLA test split Official leaderboard
27 openai/gpt-5.4-mini-2026-03-17#low (zero-shot, val) 51.3% Dutch CoLA test split Official leaderboard
28 Qwen/Qwen3.6-27B-FP8 (val) 51% Dutch CoLA test split Official leaderboard
29 deepcogito/cogito-v1-preview-qwen-14B 50.8% Dutch CoLA test split Official leaderboard
30 mistralai/Mistral-Medium-3.5-128B (zero-shot, val) 50.6% Dutch CoLA test split Official leaderboard
31 openai/gpt-5.4-2026-03-05#none (zero-shot, val) 50.6% Dutch CoLA test split Official leaderboard
32 NbAiLab/borealis2-26b-a4b-preview (val) 49.8% Dutch CoLA test split Official leaderboard
33 speakleash/Bielik-11B-v3.1-Instruct (val) 49.7% Dutch CoLA test split Official leaderboard
34 mistralai/Ministral-3-14B-Base-2512 (val) 49.6% Dutch CoLA test split Official leaderboard
35 deepcogito/cogito-v2-preview-llama-70B 49.6% Dutch CoLA test split Official leaderboard
36 mistralai/Ministral-3-14B-Reasoning-2512 (val) 49.5% Dutch CoLA test split Official leaderboard
37 Gemma 3 27B 49.4% Dutch CoLA test split Mar 2025 Official leaderboard
38 google/gemma-4-26B-A4B-it (val) 49.1% Dutch CoLA test split Official leaderboard
39 swiss-ai/Apertus-v1.5-70B (val) 49% Dutch CoLA test split Official leaderboard
40 Qwen/Qwen3.5-35B-A3B-FP8 (val) 48.9% Dutch CoLA test split Official leaderboard
41 gpt-6-luna (zero-shot, val) 48.8% Dutch CoLA test split Official leaderboard
42 NbAiLab/borealis2-26b-a4b-preview (zero-shot, val) 48.7% Dutch CoLA test split Official leaderboard
43 zai-org/GLM-5.3-Flash (zero-shot, val) 48.7% Dutch CoLA test split Official leaderboard
44 Gemma 3 12B PT 48.5% Dutch CoLA test split Mar 2025 Official leaderboard
45 Gemma 3 27B PT 48.2% Dutch CoLA test split Mar 2025 Official leaderboard
46 Qwen/Qwen3-14B#no-thinking 48% Dutch CoLA test split Official leaderboard
47 Qwen/Qwen3-32B#no-thinking 48% Dutch CoLA test split Official leaderboard
48 NbAiLab/borealis2-31b-preview (zero-shot, val) 47.9% Dutch CoLA test split Official leaderboard
49 gpt-5.6-sol (zero-shot, val) 47.7% Dutch CoLA test split Official leaderboard
50 deepcogito/cogito-v1-preview-qwen-32B 47.5% Dutch CoLA test split Official leaderboard
51 gemini/gemini-2.5-flash-lite#no-thinking (zero-shot, val) 47.3% Dutch CoLA test split Official leaderboard
52 amalia-llm/AMALIA-9B-0626-DPO (val) 46.6% Dutch CoLA test split Official leaderboard
53 deepseek/deepseek-flash#low (zero-shot, val) 46.4% Dutch CoLA test split Official leaderboard
54 gpt-5.2-2025-12-11 (zero-shot, val) 46.3% Dutch CoLA test split Official leaderboard
55 Gemma 3 12B 45.8% Dutch CoLA test split Mar 2025 Official leaderboard
56 claude-haiku-4-5-20251001 (zero-shot, val) 45.2% Dutch CoLA test split Official leaderboard
57 gemini/gemini-2.5-flash-lite#thinking (zero-shot, val) 45.1% Dutch CoLA test split Official leaderboard
58 mistralai/Mistral-Small-3.1-24B-Base-2503 43.9% Dutch CoLA test split Official leaderboard
59 swiss-ai/Apertus-v1.5-8B (val) 43.9% Dutch CoLA test split Official leaderboard
60 openai/gpt-oss-120b (val) 43.7% Dutch CoLA test split Official leaderboard
61 speakleash/Bielik-11B-v3.0-Instruct (val) 43.1% Dutch CoLA test split Official leaderboard
62 openeurollm/prelude-long (val) 43.1% Dutch CoLA test split Official leaderboard
63 deepcogito/cogito-v1-preview-llama-8B 42.6% Dutch CoLA test split Official leaderboard
64 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (val) 42.5% Dutch CoLA test split Official leaderboard
65 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 (val) 42.5% Dutch CoLA test split Official leaderboard
66 latam-gpt/Llama-3.1-70B-LatamGPT-SFT-1.0 (val) 42.2% Dutch CoLA test split Official leaderboard
67 Qwen/Qwen3-8B#no-thinking 42.1% Dutch CoLA test split Official leaderboard
68 Qwen/Qwen3.5-9B (val) 42% Dutch CoLA test split Official leaderboard
69 gpt-5-nano-2025-08-07 (zero-shot, val) 41.9% Dutch CoLA test split Official leaderboard
70 openai/gpt-5.4-mini-2026-03-17#medium (zero-shot, val) 41.6% Dutch CoLA test split Official leaderboard
71 openai/gpt-5.4-nano-2026-03-17#medium (zero-shot, val) 41.6% Dutch CoLA test split Official leaderboard
72 allenai/Llama-3.1-Tulu-3-8B-SFT 41.3% Dutch CoLA test split Official leaderboard
73 google/gemma-4-12B-it (val) 41.3% Dutch CoLA test split Official leaderboard
74 mistralai/Mistral-Small-3.2-24B-Instruct-2506 41.2% Dutch CoLA test split Official leaderboard
75 openeurollm/prelude-base (val) 40.9% Dutch CoLA test split Official leaderboard
76 Qwen/Qwen3.8-27B (val) 40.7% Dutch CoLA test split Official leaderboard
77 gpt-5.6-terra (zero-shot, val) 40.2% Dutch CoLA test split Official leaderboard
78 speakleash/Bielik-Minitron-7B-v3.0-Instruct (val) 39.9% Dutch CoLA test split Official leaderboard
79 openai/gpt-5.4-nano-2026-03-17#low (zero-shot, val) 39.8% Dutch CoLA test split Official leaderboard
80 Qwen/Qwen3-4B#no-thinking 39.8% Dutch CoLA test split Official leaderboard
81 google/gemma-4-E4B-it (val) 39.5% Dutch CoLA test split Official leaderboard
82 mistralai/Mistral-Large-3-675B-Instruct-2512 (zero-shot, val) 39.3% Dutch CoLA test split Official leaderboard
83 speakleash/Bielik-One-11B-v3.1-r1@75a2d83890e3c68bf7fac2fe5bbe2bff6372036c (zero-shot) 39.1% Dutch CoLA test split Official leaderboard
84 nvidia/Nemotron-Cascade-2-30B-A3B (val) 38.9% Dutch CoLA test split Official leaderboard
85 zai-org/GLM-4.7-Flash (val) 38.7% Dutch CoLA test split Official leaderboard
86 swiss-ai/Apertus-v1.5-70B (zero-shot, val) 38.6% Dutch CoLA test split Official leaderboard
87 domyn/Domyn-Small-v1.0 (val) 38.1% Dutch CoLA test split Official leaderboard
88 mistralai/Ministral-3-8B-Reasoning-2512 38% Dutch CoLA test split Official leaderboard
89 Llama 3.1-70B 38% Dutch CoLA test split Jul 2024 Official leaderboard
90 Gemma 3 4B 37.8% Dutch CoLA test split Mar 2025 Official leaderboard
91 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 (val) 37.6% Dutch CoLA test split Official leaderboard
92 microsoft/mdeberta-v3-base 37.5% Dutch CoLA test split Official leaderboard
93 meta-llama/Llama-3.1-8B-Instruct 37.1% Dutch CoLA test split Official leaderboard
94 amalia-llm/AMALIA-9B-0626-SFT (val) 37.1% Dutch CoLA test split Official leaderboard
95 openai/gpt-5.4-nano-2026-03-17#high (zero-shot, val) 37% Dutch CoLA test split Official leaderboard
96 mistralai/Mistral-Small-3.1-24B-Instruct-2503 37% Dutch CoLA test split Official leaderboard
97 mistralai/Mistral-Small-4-119B-2603 (zero-shot, val) 36.6% Dutch CoLA test split Official leaderboard
98 gpt-oss-20b 36.3% Dutch CoLA test split Aug 2025 Official leaderboard
99 Qwen/Qwen3.5-9B-Base (val) 36.3% Dutch CoLA test split Official leaderboard
100 mistralai/Ministral-3-14B-Instruct-2512 (val) 36.3% Dutch CoLA test split Official leaderboard
101 openai/gpt-5.4-mini-2026-03-17#high (zero-shot, val) 36.1% Dutch CoLA test split Official leaderboard
102 OpenLLM-France/Luciole-23B-Instruct-1.1 (val) 35.9% Dutch CoLA test split Official leaderboard
103 Qwen/Qwen3.5-35B-A3B-Base (val) 35.9% Dutch CoLA test split Official leaderboard
104 speakleash/Bielik-11B-v2.3-Instruct 35.2% Dutch CoLA test split Official leaderboard
105 NbAiLab/borealis2-e4b-preview (val) 35.2% Dutch CoLA test split Official leaderboard
106 CohereLabs/aya-expanse-8b (val) 34.7% Dutch CoLA test split Official leaderboard
107 utter-project/EuroLLM-9B-Instruct 33.9% Dutch CoLA test split Official leaderboard
108 Qwen3-4B-Thinking-2507 33.6% Dutch CoLA test split Aug 2025 Official leaderboard
109 utter-project/EuroLLM-9B-Instruct-2512 (val) 33.1% Dutch CoLA test split Official leaderboard
110 OpenLLM-France/Luciole-8B-Instruct-1.1 (val) 32.4% Dutch CoLA test split Official leaderboard
111 google/gemma-4-31B (val) 32.2% Dutch CoLA test split Official leaderboard
112 Qwen/Qwen3.5-2B (val) 31.6% Dutch CoLA test split Official leaderboard
113 granite-4.0-micro 31.5% Dutch CoLA test split Oct 2025 Official leaderboard
114 NbAiLab/borealis2-e2b-preview (val) 31.1% Dutch CoLA test split Official leaderboard
115 Gemma 3 4B PT 31.1% Dutch CoLA test split Mar 2025 Official leaderboard
116 mii-llm/nesso-4B 31% Dutch CoLA test split Official leaderboard
117 allenai/Olmo-3-1125-32B 30.7% Dutch CoLA test split Official leaderboard
118 danish-foundation-models/DFM-Mimir-v1.8 (val) 30.4% Dutch CoLA test split Official leaderboard
119 Aleph-Alpha/Kolibri-1 (val) 29.8% Dutch CoLA test split Official leaderboard
120 google/gemma-4-E2B-it (val) 29.8% Dutch CoLA test split Official leaderboard
121 typesafe/jev-1.13.0 (zero-shot) 29.8% Dutch CoLA test split Official leaderboard
122 mistralai/Ministral-3-3B-Instruct-2512 (val) 29.4% Dutch CoLA test split Official leaderboard
123 deepcogito/cogito-v1-preview-llama-3B 29% Dutch CoLA test split Official leaderboard
124 allenai/Olmo-3.1-32B-Instruct-DPO 28.8% Dutch CoLA test split Official leaderboard
125 allenai/Olmo-3.1-32B-Instruct-SFT 28.8% Dutch CoLA test split Official leaderboard
126 danish-foundation-models/DFM-Mimir-v1.8 (zero-shot, val) 28.5% Dutch CoLA test split Official leaderboard
127 Olmo 3.1 32B Instruct 27.5% Dutch CoLA test split Dec 2025 Official leaderboard
128 ibm-granite/granite-4.0-h-micro 27.2% Dutch CoLA test split Official leaderboard
129 Qwen/Qwen3.5-4B (val) 27.1% Dutch CoLA test split Official leaderboard
130 Ministral 3 3B Reasoning 26.9% Dutch CoLA test split Dec 2025 Official leaderboard
131 Llama 3.1-8B 26.8% Dutch CoLA test split Jul 2024 Official leaderboard
132 danish-foundation-models/DFM-Mimir-v1.5 (val) 26.6% Dutch CoLA test split Official leaderboard
133 danish-foundation-models/DFM-Mimir-v1.5 (zero-shot, val) 25.9% Dutch CoLA test split Official leaderboard
134 BSC-LT/salamandra-7b-instruct 25.8% Dutch CoLA test split Official leaderboard
135 amalia-llm/AMALIA-9B-0626-DPO (zero-shot, val) 25.5% Dutch CoLA test split Official leaderboard
136 amalia-llm/AMALIA-9B-0626-SFT (zero-shot, val) 24.8% Dutch CoLA test split Official leaderboard
137 swiss-ai/Apertus-v1.1-4B (val) 24.6% Dutch CoLA test split Official leaderboard
138 google/gemma-4-26B-A4B (val) 23.8% Dutch CoLA test split Official leaderboard
139 Qwen/Qwen3.5-2B-Base (val) 22.8% Dutch CoLA test split Official leaderboard
140 meta-llama/Llama-3.2-3B-Instruct 22.6% Dutch CoLA test split Official leaderboard
141 inclusionAI/Ling-mini-2.0 (val) 22.5% Dutch CoLA test split Official leaderboard
142 OpenLLM-France/Luciole-23B-Base (val) 22.3% Dutch CoLA test split Official leaderboard
143 google/gemma-4-E4B (val) 22% Dutch CoLA test split Official leaderboard
144 google/gemma-4-12B (val) 21.4% Dutch CoLA test split Official leaderboard
145 mistralai/Ministral-3-8B-Instruct-2512 (val) 21.3% Dutch CoLA test split Official leaderboard
146 Qwen/Qwen3.5-4B-Base (val) 21.2% Dutch CoLA test split Official leaderboard
147 LGAI-EXAONE/EXAONE-4.0-32B (val) 20.5% Dutch CoLA test split Official leaderboard
148 Qwen/Qwen3-1.7B#no-thinking 20.3% Dutch CoLA test split Official leaderboard
149 utter-project/EuroLLM-22B-2512 20.1% Dutch CoLA test split Official leaderboard
150 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 19.9% Dutch CoLA test split Official leaderboard
151 HuggingFaceTB/SmolLM3-3B-Base 19.9% Dutch CoLA test split Official leaderboard
152 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 (val) 19.8% Dutch CoLA test split Official leaderboard
153 danish-foundation-models/DFM-Mimir (val) 19.6% Dutch CoLA test split Official leaderboard
154 Qwen/Qwen3.5-0.8B-Base (val) 19.3% Dutch CoLA test split Official leaderboard
155 openai/gpt-5.4-nano-2026-03-17#none (zero-shot, val) 19% Dutch CoLA test split Official leaderboard
156 jhu-clsp/mmBERT-small 18.5% Dutch CoLA test split Official leaderboard
157 jhu-clsp/mmBERT-base 18.4% Dutch CoLA test split Official leaderboard
158 swiss-ai/Apertus-v1.5-8B (zero-shot, val) 18.1% Dutch CoLA test split Official leaderboard
159 allenai/Olmo-3-1025-7B 17.6% Dutch CoLA test split Official leaderboard
160 speakleash/Bielik-One-4B-r1@567d29b0f15df28794acf0f6187f9216860d4551 (zero-shot) 17.3% Dutch CoLA test split Official leaderboard
161 danish-foundation-models/DFM-Mimir-v1.8-Mini (val) 17.1% Dutch CoLA test split Official leaderboard
162 JetBrains/Mellum2-12B-A2.5B-Instruct (val) 16.7% Dutch CoLA test split Official leaderboard
163 openGPT-X/Teuken-7B-instruct-v0.6 (val) 16.2% Dutch CoLA test split Official leaderboard
164 HuggingFaceTB/SmolLM3-3B#no-thinking 15.7% Dutch CoLA test split Official leaderboard
165 mistralai/Ministral-3-8B-Base-2512 (val) 15.7% Dutch CoLA test split Official leaderboard
166 occiglot/occiglot-7b-eu5-instruct 15.5% Dutch CoLA test split Official leaderboard
167 JetBrains/Mellum2-12B-A2.5B-Base (val) 15.2% Dutch CoLA test split Official leaderboard
168 Qwen/Qwen3.5-0.8B (val) 14.9% Dutch CoLA test split Official leaderboard
169 swiss-ai/Apertus-v1.1-4B-Instruct (val) 14.9% Dutch CoLA test split Official leaderboard
170 danish-foundation-models/DFM-Mimir-v1.8-Mini (zero-shot, val) 14.8% Dutch CoLA test split Official leaderboard
171 allenai/Olmo-3-7B-Instruct 14.3% Dutch CoLA test split Official leaderboard
172 openeurollm/oellm-9b-128k-theta32m-v3 (val) 13.9% Dutch CoLA test split Official leaderboard
173 allenai/Olmo-3-7B-Instruct-DPO 13.6% Dutch CoLA test split Official leaderboard
174 danish-foundation-models/DFM-Mimir (zero-shot, val) 13.6% Dutch CoLA test split Official leaderboard
175 LiquidAI/LFM2.5-8B-A1B (val) 12.6% Dutch CoLA test split Official leaderboard
176 intfloat/multilingual-e5-base 12.4% Dutch CoLA test split Official leaderboard
177 gemini/gemini-3-flash-preview#no-thinking (zero-shot, val) 11.8% Dutch CoLA test split Official leaderboard
178 intfloat/multilingual-e5-large 11.4% Dutch CoLA test split Official leaderboard
179 allenai/Olmo-3-7B-Instruct-SFT 11.3% Dutch CoLA test split Official leaderboard
180 EuroBERT/EuroBERT-210m 11.3% Dutch CoLA test split Official leaderboard
181 LiquidAI/LFM2.5-8B-A1B-Base (val) 11.3% Dutch CoLA test split Official leaderboard
182 answerdotai/ModernBERT-large 11.2% Dutch CoLA test split Official leaderboard
183 EuroBERT/EuroBERT-610m 11.1% Dutch CoLA test split Official leaderboard
184 CohereLabs/aya-23-8B (val) 9.84% Dutch CoLA test split Official leaderboard
185 Aleph-Alpha/Kolibri-1 (zero-shot, val) 9.65% Dutch CoLA test split Official leaderboard
186 speakleash/Bielik-1.5B-v3.0-Instruct 9.52% Dutch CoLA test split Official leaderboard
187 utter-project/EuroLLM-9B 9.41% Dutch CoLA test split Official leaderboard
188 TrustLLMeu/trustllm-v2-8b-midtrain@3c7af30bc43bb3c8607e0fab6e7f2a23dd64af49 8.80% Dutch CoLA test split Official leaderboard
189 distilbert/distilbert-base-multilingual-cased 7.68% Dutch CoLA test split Official leaderboard
190 Geotrend/distilbert-base-25lang-cased 7.56% Dutch CoLA test split Official leaderboard
191 google/gemma-4-E2B (val) 7.55% Dutch CoLA test split Official leaderboard
192 Qwen3-0.6B 7.00% Dutch CoLA test split Apr 2025 Official leaderboard
193 swiss-ai/Apertus-v1.1-0.5B (val) 6.66% Dutch CoLA test split Official leaderboard
194 answerdotai/ModernBERT-base 6.56% Dutch CoLA test split Official leaderboard
195 AI-Sweden-Models/bert-large-nordic-pile-1M-steps 6.47% Dutch CoLA test split Official leaderboard
196 swiss-ai/Apertus-v1.1-1.5B (val) 6.40% Dutch CoLA test split Official leaderboard
197 AI-Sweden-Models/Llama-3-8B-instruct (val) 6.02% Dutch CoLA test split Official leaderboard
198 OmniJev/OneJev1.6-0.8B (zero-shot) 5.99% Dutch CoLA test split Official leaderboard
199 fastino/GLiNER2.5-Decide (zero-shot) 5.92% Dutch CoLA test split Official leaderboard
200 Gemma 3 1B PT 5.90% Dutch CoLA test split Mar 2025 Official leaderboard
201 EuropeanParliament/EUBERT 5.74% Dutch CoLA test split Official leaderboard
202 OpenLLM-France/Luciole-8B-Base (val) 5.50% Dutch CoLA test split Official leaderboard
203 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f (zero-shot) 5.00% Dutch CoLA test split Official leaderboard
204 TrustLLMeu/trustllm-v2-8b@d8cc924fa550fbf9474f5251e8e2f92d1c1bf894 4.90% Dutch CoLA test split Official leaderboard
205 fresh-electra-small 4.70% Dutch CoLA test split Official leaderboard
206 fresh-xlm-roberta-base 4.62% Dutch CoLA test split Official leaderboard
207 flwrlabs/Lizzy-7B (val) 4.61% Dutch CoLA test split Official leaderboard
208 NbAiLab/borealis2-e4b-preview (zero-shot, val) 4.37% Dutch CoLA test split Official leaderboard
209 syvai/claes-v1 (zero-shot) 4.30% Dutch CoLA test split Official leaderboard
210 convaiinnovations/laya (zero-shot) 4.00% Dutch CoLA test split Official leaderboard
211 swiss-ai/Apertus-v1.1-0.5B-Instruct (val) 3.89% Dutch CoLA test split Official leaderboard
212 occiglot/occiglot-7b-eu5 3.88% Dutch CoLA test split Official leaderboard
213 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f 3.72% Dutch CoLA test split Official leaderboard
214 Qwen/Qwen3-0.6B#no-thinking 3.60% Dutch CoLA test split Official leaderboard
215 OpenLLM-France/Luciole-1B-Instruct-1.1 (val) 3.56% Dutch CoLA test split Official leaderboard
216 LFM2-1.2B 3.52% Dutch CoLA test split Jul 2025 Official leaderboard
217 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base 3.37% Dutch CoLA test split Official leaderboard
218 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 (zero-shot) 3.34% Dutch CoLA test split Official leaderboard
219 JetBrains/Mellum2-12B-A2.5B-Thinking-SFT (val) 3.09% Dutch CoLA test split Official leaderboard
220 PleIAs/Pleias-350m-Preview 2.98% Dutch CoLA test split Official leaderboard
221 yulan-team/YuLan-Mini-Instruct 2.90% Dutch CoLA test split Official leaderboard
222 meta-llama/Llama-3.2-1B-Instruct 2.78% Dutch CoLA test split Official leaderboard
223 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 2.62% Dutch CoLA test split Official leaderboard
224 Gemma 3 1B 2.52% Dutch CoLA test split Mar 2025 Official leaderboard
225 Gemma 3 270M 2.36% Dutch CoLA test split Aug 2025 Official leaderboard
226 Llama 3.2 1B 2.35% Dutch CoLA test split Sep 2024 Official leaderboard
227 OpenLLM-France/Luciole-1B-Base (val) 2.26% Dutch CoLA test split Official leaderboard
228 BSC-LT/salamandra-7b 1.95% Dutch CoLA test split Official leaderboard
229 PleIAs/Pleias-1.2b-Preview 1.91% Dutch CoLA test split Official leaderboard
230 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 (zero-shot) 1.82% Dutch CoLA test split Official leaderboard
231 syvai/claes-preview (zero-shot) 1.76% Dutch CoLA test split Official leaderboard
232 JetBrains/Mellum2-12B-A2.5B-Thinking (val) 1.62% Dutch CoLA test split Official leaderboard
233 AIDC-AI/Marco-Mini-Global-Base (val) 1.34% Dutch CoLA test split Official leaderboard
234 PleIAs/Monad 1.23% Dutch CoLA test split Official leaderboard
235 LiquidAI/LFM2-350M 1.08% Dutch CoLA test split Official leaderboard
236 JetBrains/Mellum2-12B-A2.5B-Instruct-SFT (val) 0.92% Dutch CoLA test split Official leaderboard
237 SmolLM2-135M 0.70% Dutch CoLA test split Oct 2024 Official leaderboard
238 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 0.56% Dutch CoLA test split Official leaderboard
239 SmolLM2-360M 0.46% Dutch CoLA test split Oct 2024 Official leaderboard
240 HuggingFaceTB/SmolLM2-135M-Instruct 0.40% Dutch CoLA test split Official leaderboard
241 convaiinnovations/laya-multilingual (zero-shot) 0.32% Dutch CoLA test split Official leaderboard
242 google-bert/bert-base-multilingual-cased (val) 0.23% Dutch CoLA test split Official leaderboard
243 NbAiLab/borealis2-e2b-preview (zero-shot, val) 0.23% Dutch CoLA test split Official leaderboard
244 LiquidAI/LFM2-700M 0.19% Dutch CoLA test split Official leaderboard
245 swiss-ai/Apertus-v1.1-1.5B (zero-shot) 0.15% Dutch CoLA test split Official leaderboard
246 jaredpalmer/kev-0.5b (zero-shot) 0.14% Dutch CoLA test split Official leaderboard
247 HuggingFaceTB/SmolLM2-360M-Instruct 0.06% Dutch CoLA test split Official leaderboard
248 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct 0.06% Dutch CoLA test split Official leaderboard
249 AI-Sweden-Models/Llama-3-8B-instruct (zero-shot, val) 0% Dutch CoLA test split Official leaderboard
250 JetBrains/Mellum2-12B-A2.5B-Base-Pretrain (val) 0% Dutch CoLA test split Official leaderboard
251 mistralai/Ministral-3-14B-Instruct-2512 (zero-shot, val) 0% Dutch CoLA test split Official leaderboard
252 mistralai/Ministral-3-3B-Instruct-2512 (zero-shot, val) 0% Dutch CoLA test split Official leaderboard
253 Muse-research/Muse-3B (val) 0% Dutch CoLA test split Official leaderboard
254 openGPT-X/Teuken-7B-base-v0.6 (val) 0% Dutch CoLA test split Official leaderboard
255 PleIAs/Baguettotron@refs/pr/6#no-thinking 0% Dutch CoLA test split Official leaderboard
256 swiss-ai/Apertus-v1.1-1.5B-Instruct (val) 0% Dutch CoLA test split Official leaderboard
257 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview 0% Dutch CoLA test split Official leaderboard
#ModelScoreSettingDateSource
1 Gemma 3 27B 49.4% Dutch CoLA test split Mar 2025 Official leaderboard
2 Gemma 3 12B PT 48.5% Dutch CoLA test split Mar 2025 Official leaderboard
3 Gemma 3 27B PT 48.2% Dutch CoLA test split Mar 2025 Official leaderboard
4 Gemma 3 12B 45.8% Dutch CoLA test split Mar 2025 Official leaderboard
5 Llama 3.1-70B 38% Dutch CoLA test split Jul 2024 Official leaderboard
6 Gemma 3 4B 37.8% Dutch CoLA test split Mar 2025 Official leaderboard
7 gpt-oss-20b 36.3% Dutch CoLA test split Aug 2025 Official leaderboard
8 Qwen3-4B-Thinking-2507 33.6% Dutch CoLA test split Aug 2025 Official leaderboard
9 granite-4.0-micro 31.5% Dutch CoLA test split Oct 2025 Official leaderboard
10 Gemma 3 4B PT 31.1% Dutch CoLA test split Mar 2025 Official leaderboard
11 Olmo 3.1 32B Instruct 27.5% Dutch CoLA test split Dec 2025 Official leaderboard
12 Ministral 3 3B Reasoning 26.9% Dutch CoLA test split Dec 2025 Official leaderboard
13 Llama 3.1-8B 26.8% Dutch CoLA test split Jul 2024 Official leaderboard
14 Qwen3-0.6B 7.00% Dutch CoLA test split Apr 2025 Official leaderboard
15 Gemma 3 1B PT 5.90% Dutch CoLA test split Mar 2025 Official leaderboard
16 LFM2-1.2B 3.52% Dutch CoLA test split Jul 2025 Official leaderboard
17 Gemma 3 1B 2.52% Dutch CoLA test split Mar 2025 Official leaderboard
18 Gemma 3 270M 2.36% Dutch CoLA test split Aug 2025 Official leaderboard
19 Llama 3.2 1B 2.35% Dutch CoLA test split Sep 2024 Official leaderboard
20 SmolLM2-135M 0.70% Dutch CoLA test split Oct 2024 Official leaderboard
21 SmolLM2-360M 0.46% Dutch CoLA test split Oct 2024 Official leaderboard

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind
EuroEval leaderboard Not tested since Dec 2025
Gemma 3 27B 49.4%

Source: EuroEval’s leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On EuroEval’s leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Gemma 3 27B Llama 3.1-70B 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Gemma 3 27B Llama 3.1-70B

Source: EuroEval’s leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Multilingual

All 545 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%

Data audit

9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Instruct. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (257 entries).
  • Pass Models scored 21 models we track, plus 236 other entries on its leaderboard.
  • Warning Newest score The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Instruct.
  • Note Frontier models None of the top 10 has a score. The strongest tested is gpt-oss-20b (#143).
  • Pass Headroom The best is 49.4%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on EuroEval: 24 rows for models we track and 236 other entries at the last read.
  • Pass Last read November 21st, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check