Benchmark Almanac

GuiaCat

openeurollm/oellm-9b-128k-theta32m-v3 (val) 78.3% tops its leaderboard Apertus 70B 69.5% leads the models we track
Not in model cards

Models classify the sentiment of restaurant reviews written in Catalan.

EuroEval scores sentiment labels with the Matthews correlation coefficient.

Made by
Projecte Aina
Released
October 2022
Kind
Reading in many languages
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Measures
Capability
Language
Catalan
In and out
Text → Text
Interaction
Single turn
Items from
Real work and use
Refresh
One fixed set
Run by
Anyone
Strongest model tested
gpt-oss-20b #143 on the capability index
Last model added
January 2026
Scores refreshed
November 21st, 2026
320
Entries on its own leaderboard, 35 models we track

openeurollm/oellm-9b-128k-theta32m-v3 (val) tops the leaderboard with 78.3% Apertus 70B leads with 69.5% among models released by January 2026

Best score per entry from the benchmark's own leaderboard, read on November 21st, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Apertus 70B, at 69.5%. This view shows only models we track, each at its best.

The newest model with a score here came out in January 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

320 entries. Scroll sideways for the rest.

  1. 78.3 openeurollm/oellm-9b-128k-theta32m-v3 (val)
  2. 77.2 Qwen/Qwen3.5-9B (val)
  3. 76.9 Qwen/Qwen3.5-27B (val)
  4. 76.1 openai/gpt-5.4-mini-2026-03-17#high (zero-shot, val)
  5. 76.0 CohereLabs/tiny-aya-base (val)
  6. 76.0 zai-org/GLM-4.7-Flash (val)
  7. 75.9 gpt-5.6-sol (zero-shot, val)
  8. 75.2 CohereLabs/tiny-aya-global (val)
  9. 75.1 swiss-ai/Apertus-v1.1-4B (val)
  10. 74.4 CohereLabs/tiny-aya-water (val)
  11. 74.1 gemini/gemini-3.6-flash (zero-shot, val)
  12. 74.0 Qwen/Qwen3.6-27B-FP8 (val)
  13. 74.0 gemini/gemini-3-flash-preview#thinking (zero-shot, val)
  14. 74.0 gpt-5.6-terra (zero-shot, val)
  15. 73.8 gpt-5-mini-2025-08-07 (zero-shot, val)
  16. 73.8 openai/gpt-5.4-mini-2026-03-17#medium (zero-shot, val)
  17. 73.7 AI-Sweden-Models/Llama-3-8B-instruct (val)
  18. 73.7 gemini/gemini-3.7-flash (zero-shot, val)
  19. 73.6 speakleash/Bielik-11B-v2.3-Instruct (val)
  20. 73.4 gpt-6.2-sol (zero-shot, val)
  21. 73.1 Qwen/Qwen3.5-27B-FP8 (val)
  22. 72.8 Qwen/Qwen3.5-35B-A3B-FP8 (val)
  23. 72.7 Qwen/Qwen3.8-27B (val)
  24. 72.6 AI-Sweden-Models/Llama-3-8B-instruct (zero-shot, val)
  25. 72.5 Qwen/Qwen3.5-4B (val)
  26. 72.5 GadflyII/GLM-4.7-Flash-NVFP4 (val)
  27. 72.4 gpt-6-sol (zero-shot, val)
  28. 72.4 mistralai/Ministral-3-14B-Instruct-2512 (zero-shot, val)
  29. 72.4 openai/gpt-5.4-mini-2026-03-17#low (zero-shot, val)
  30. 72.4 google/gemma-4-31B-it (val)
  31. 72.4 gpt-4.1-2025-04-14 (zero-shot, val)
  32. 72.2 mistralai/Ministral-3-14B-Base-2512 (val)
  33. 72.2 claude-sonnet-4-6 (zero-shot, val)
  34. 72.0 gpt-6-astra (zero-shot, val)
  35. 72.0 OpenLLM-France/Lucie-7B-Instruct-v1.1 (val)
  36. 71.9 mistralai/Ministral-3-8B-Base-2512 (val)
  37. 71.8 mistralai/Ministral-3-14B-Reasoning-2512 (val)
  38. 71.8 swiss-ai/Apertus-8B-2509
  39. 71.7 gpt-6.1-sol (zero-shot, val)
  40. 71.5 google/gemma-4-26B-A4B-it (val)
  41. 71.5 gpt-5-2025-08-07 (zero-shot, val)
  42. 71.3 gpt-6.2-astra (zero-shot, val)
  43. 71.1 swiss-ai/Apertus-v1.1-4B-Instruct (val)
  44. 71.0 mistralai/Ministral-3-8B-Instruct-2512 (val)
  45. 71.0 swiss-ai/Apertus-v1.5-70B (zero-shot, val)
  46. 70.7 latam-gpt/Llama-3.1-70B-LatamGPT-SFT-1.0 (val)
  47. 70.7 openai/gpt-5.4-nano-2026-03-17#low (zero-shot, val)
  48. 70.7 openeurollm/prelude-base (val)
  49. 70.5 mistralai/Mistral-Small-4-119B-2603 (zero-shot, val)
  50. 70.3 gpt-6-luna (zero-shot, val)
  51. 70.3 mistralai/Mistral-Medium-3.5-128B (zero-shot, val)
  52. 70.2 CohereLabs/aya-23-8B (val)
  53. 69.7 deepseek/deepseek-flash#low (zero-shot, val)
  54. 69.5 zai-org/GLM-5.3-Flash (val)
  55. 69.5 Apertus 70B
  56. 69.5 Llama 3.1-8B
  57. 69.4 Qwen/Qwen3.5-2B (val)
  58. 69.4 domyn/Domyn-Small-v1.0 (val)
  59. 69.3 openai/gpt-5.4-nano-2026-03-17#medium (zero-shot, val)
  60. 69.0 gemini/gemini-3.5-flash-lite (zero-shot, val)
  61. 68.9 mistralai/Mistral-Small-4-119B-2603 (val)
  62. 68.7 gemini/gemini-2.5-pro (zero-shot, val)
  63. 68.6 gemini/gemini-2.5-flash-lite#thinking (zero-shot, val)
  64. 68.6 openai/gpt-5.4-mini-2026-03-17#none (zero-shot, val)
  65. 68.5 gemini/gemini-2.5-flash#thinking (zero-shot, val)
  66. 68.5 OpenLLM-France/Luciole-8B-Base (val)
  67. 68.5 NbAiLab/borealis2-31b-preview (zero-shot, val)
  68. 68.5 claude-haiku-4-5-20251001 (zero-shot, val)
  69. 68.4 openai/gpt-5.4-2026-03-05#none (zero-shot, val)
  70. 68.3 gpt-5-nano-2025-08-07 (zero-shot, val)
  71. 68.3 utter-project/EuroLLM-22B-2512
  72. 68.2 Unbabel/Tower-Plus-72B (val)
  73. 68.1 claude-sonnet-4-5-20250929#no-thinking (zero-shot, val)
  74. 68.1 OpenLLM-France/Luciole-8B-Instruct-1.1 (val)
  75. 68.1 openai/gpt-5.4-nano-2026-03-17#none (zero-shot, val)
  76. 68.0 openai/gpt-5.6-luna (zero-shot, val)
  77. 67.8 xai/grok-4-1-fast-reasoning (zero-shot, val)
  78. 67.7 gemini/gemini-3.1-flash-lite-preview (zero-shot, val)
  79. 67.5 Gemma 3 12B PT
  80. 67.4 openeurollm/prelude-long (val)
  81. 67.3 NbAiLab/borealis2-31b-preview (val)
  82. 67.3 nvidia/Nemotron-Cascade-2-30B-A3B (val)
  83. 67.3 gemini/gemini-3-pro-preview (zero-shot, val)
  84. 67.2 OpenLLM-France/Luciole-23B-Instruct-1.1 (val)
  85. 67.2 Llama 3.1-70B
  86. 67.2 gpt-5.2-2025-12-11 (zero-shot, val)
  87. 67.1 swiss-ai/Apertus-v1.5-70B (val)
  88. 67.1 mistralai/Mistral-Medium-3.5-128B (val)
  89. 67.1 utter-project/EuroLLM-22B-Instruct-2512
  90. 67.0 openai/gpt-5.4-nano-2026-03-17#high (zero-shot, val)
  91. 66.9 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (val)
  92. 66.7 google/gemma-4-12B (val)
  93. 66.7 mii-llm/nesso-4B
  94. 66.5 zai-org/GLM-5.3-Flash (zero-shot, val)
  95. 66.5 TrustLLMeu/baseline-7-8b_2-3t-tokens_llama
  96. 66.5 CohereLabs/aya-expanse-8b (val)
  97. 66.2 mistralai/Mistral-Large-3-675B-Instruct-2512 (zero-shot, val)
  98. 66.2 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 (val)
  99. 66.0 gemini/gemini-2.5-flash#no-thinking (zero-shot, val)
  100. 66.0 gemini/gemini-3-flash-preview#no-thinking (zero-shot, val)
  101. 66.0 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 (val)
  102. 66.0 danish-foundation-models/DFM-Mimir-v1.8-Mini (zero-shot, val)
  103. 65.9 OpenLLM-France/Lucie-7B-Instruct-human-data (val)
  104. 65.8 intfloat/multilingual-e5-large
  105. 65.5 openai/gpt-oss-120b (val)
  106. 65.5 mistralai/Ministral-3-3B-Base-2512
  107. 65.5 danish-foundation-models/DFM-Mimir-v1.8 (zero-shot, val)
  108. 65.5 utter-project/EuroLLM-22B-Preview
  109. 65.3 danish-foundation-models/DFM-Mimir (zero-shot, val)
  110. 65.3 speakleash/Bielik-11B-v2.6-Instruct
  111. 65.1 mistralai/Ministral-3-14B-Instruct-2512 (val)
  112. 64.5 swiss-ai/Apertus-v1.1-1.5B (val)
  113. 64.4 NbAiLab/borealis2-26b-a4b-preview (zero-shot, val)
  114. 64.2 xai/grok-4.20-beta-0309-reasoning (zero-shot, val)
  115. 64.0 BSC-LT/ALIA-40b
  116. 64.0 deepseek/deepseek-flash#no-thinking (val)
  117. 63.7 Gemma 3 4B PT
  118. 63.6 google/gemma-4-12B-it (val)
  119. 63.5 Qwen3-4B-Thinking-2507
  120. 63.4 Qwen/Qwen3.5-35B-A3B-Base (val)
  121. 63.3 claude-sonnet-4-5-20250929#thinking (zero-shot, val)
  122. 62.9 NbAiLab/borealis2-26b-a4b-preview (val)
  123. 62.9 swiss-ai/Apertus-v1.1-0.5B (val)
  124. 62.7 ibm-granite/granite-4.0-h-tiny
  125. 62.4 mistralai/Mistral-Large-3-675B-Instruct-2512 (val)
  126. 62.3 Gemma 3 27B PT
  127. 62.3 gpt-4.1-mini-2025-04-14 (zero-shot, val)
  128. 62.1 allenai/Olmo-3.1-32B-Think (val)
  129. 62.0 Granite 4.0 H Small
  130. 61.9 LiquidAI/LFM2.5-8B-A1B-Base (val)
  131. 61.8 OpenLLM-Ro/RoMistral-7b-Instruct-DPO
  132. 61.7 mistralai/Mistral-Small-3.1-24B-Base-2503
  133. 61.3 OpenLLM-Ro/RoLlama3.1-8b-Instruct-DPO
  134. 61.2 typesafe/jev-1.13.0 (zero-shot)
  135. 61.1 amalia-llm/AMALIA-9B-0626-DPO (zero-shot, val)
  136. 61.1 gemini/gemini-2.5-flash-lite#no-thinking (zero-shot, val)
  137. 61.0 danish-foundation-models/DFM-Mimir-v1.8 (val)
  138. 60.9 swiss-ai/Apertus-v1.5-8B (zero-shot, val)
  139. 60.9 speakleash/Bielik-11B-v2.3-Instruct
  140. 60.8 allenai/Llama-3.1-Tulu-3-8B-SFT
  141. 60.5 utter-project/EuroLLM-1.7B-Instruct (val)
  142. 60.5 danish-foundation-models/DFM-Mimir-v1.8-Mini (val)
  143. 60.3 deepcogito/cogito-v1-preview-qwen-14B
  144. 60.0 mistralai/Mistral-Small-3.2-24B-Instruct-2506
  145. 59.9 Gemma 3 1B PT
  146. 59.7 gpt-oss-20b
  147. 59.6 amalia-llm/AMALIA-9B-0626-SFT (zero-shot, val)
  148. 59.6 BSC-LT/salamandra-7b-instruct
  149. 59.6 utter-project/EuroLLM-9B-Instruct-2512 (val)
  150. 59.5 syvai/claes-preview (zero-shot)
  151. 59.3 Qwen/Qwen3-1.7B#no-thinking
  152. 59.1 speakleash/Bielik-One-4B-r1@567d29b0f15df28794acf0f6187f9216860d4551 (zero-shot)
  153. 59.0 google/gemma-4-31B (val)
  154. 58.9 NbAiLab/borealis2-e4b-preview (zero-shot, val)
  155. 58.9 danish-foundation-models/DFM-Mimir (val)
  156. 58.9 deepcogito/cogito-v1-preview-qwen-32B
  157. 58.7 amalia-llm/AMALIA-9B-0626-DPO (val)
  158. 58.6 google/gemma-4-E4B-it (val)
  159. 58.6 Llama 3.3 70B Instruct
  160. 58.5 speakleash/Bielik-One-11B-v3.1-r1@75a2d83890e3c68bf7fac2fe5bbe2bff6372036c (zero-shot)
  161. 58.4 meta-llama/Llama-3.1-8B-Instruct
  162. 58.4 Qwen/Qwen3-32B#no-thinking
  163. 58.4 Qwen/Qwen3-4B#no-thinking
  164. 58.4 HuggingFaceTB/SmolLM3-3B-Base
  165. 58.4 speakleash/Bielik-Minitron-7B-v3.0-Instruct (val)
  166. 58.0 mistralai/Mistral-Small-3.1-24B-Instruct-2503
  167. 57.6 allenai/Olmo-3.1-32B-Instruct-SFT
  168. 57.4 google/gemma-4-E2B-it (val)
  169. 57.3 gpt-4.1-nano-2025-04-14 (zero-shot, val)
  170. 57.3 LGAI-EXAONE/EXAONE-4.0-32B (val)
  171. 57.2 NbAiLab/borealis2-e2b-preview (zero-shot, val)
  172. 57.2 occiglot/occiglot-7b-eu5-instruct
  173. 56.9 NbAiLab/borealis2-e2b-preview (val)
  174. 56.5 Gemma 3 4B
  175. 56.4 danish-foundation-models/DFM-Mimir-v1.5 (val)
  176. 56.3 Qwen/Qwen3-14B#no-thinking
  177. 56.2 swiss-ai/Apertus-v1.5-8B (val)
  178. 56.1 Qwen/Qwen2.5-1.5B-Instruct
  179. 56.0 mistralai/Ministral-3-3B-Instruct-2512 (zero-shot, val)
  180. 55.9 Qwen2.5-72B
  181. 55.8 speakleash/Bielik-1.5B-v3.0-Instruct
  182. 55.8 Cogito v1 Preview 70B
  183. 55.8 speakleash/Bielik-11B-v3.0-Instruct (val)
  184. 55.8 Qwen/Qwen3-8B#no-thinking
  185. 55.7 OpenLLM-Ro/RoGemma2-9b-Instruct-DPO
  186. 55.6 kurakurai/Luth-LFM2-1.2B (val)
  187. 55.4 amalia-llm/AMALIA-9B-0626-SFT (val)
  188. 55.2 granite-4.0-micro
  189. 55.2 Voxtral Small
  190. 54.8 utter-project/EuroLLM-22B-Instruct-Preview
  191. 54.6 Unbabel/Tower-Plus-9B (val)
  192. 54.5 ibm-granite/granite-4.0-h-micro
  193. 54.5 Gemma 3 27B
  194. 54.5 Llama 3.2 1B
  195. 54.4 OpenLLM-France/Luciole-23B-Base (val)
  196. 54.4 deepcogito/cogito-v2-preview-llama-70B
  197. 53.8 NbAiLab/borealis2-e4b-preview (val)
  198. 53.6 speakleash/Bielik-11B-v3.1-Instruct (val)
  199. 53.5 google/rembert
  200. 53.3 baidu/ERNIE-4.5-21B-A3B-PT
  201. 53.2 deepcogito/cogito-v1-preview-llama-8B
  202. 53.2 yulan-team/YuLan-Mini-Instruct
  203. 53.2 danish-foundation-models/DFM-Mimir-v1.5 (zero-shot, val)
  204. 52.7 Qwen/Qwen3-0.6B#no-thinking
  205. 52.6 meta-llama/Llama-3.2-3B-Instruct
  206. 52.2 arcee-ai/AFM-4.5B-Base
  207. 52.2 Gemma 3 12B
  208. 51.9 mistralai/Ministral-3-3B-Instruct-2512 (val)
  209. 51.9 google/gemma-4-26B-A4B (val)
  210. 51.8 allenai/Olmo-3.1-32B-Instruct-DPO
  211. 51.2 Olmo 3.1 32B Instruct
  212. 51.1 LFM2-1.2B
  213. 51.1 setu4993/LaBSE
  214. 51.0 Aleph-Alpha/Kolibri-1 (zero-shot, val)
  215. 51.0 intfloat/multilingual-e5-base
  216. 50.6 Aleph-Alpha/Kolibri-1 (val)
  217. 50.3 jhu-clsp/mmBERT-base
  218. 50.3 mistralai/Ministral-3-8B-Reasoning-2512
  219. 50.2 speakleash/Bielik-4.5B-v3.0-Instruct
  220. 49.8 Unbabel/Tower-Plus-2B (val)
  221. 49.0 kurakurai/Luth-0.6B-Instruct
  222. 48.8 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 (val)
  223. 48.8 LiquidAI/LFM2-2.6B
  224. 47.6 Gemma 3n E2B
  225. 46.7 OLMo 2 32B Instruct
  226. 46.6 Trinity Mini
  227. 46.3 swiss-ai/Apertus-v1.1-1.5B-Instruct (val)
  228. 46.2 sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
  229. 46.1 Qwen3-0.6B
  230. 46.1 Qwen/Qwen3.5-0.8B (val)
  231. 46.0 Gemma 3n E4B
  232. 45.5 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
  233. 45.5 allenai/Olmo-3-1025-7B
  234. 44.9 deepcogito/cogito-v1-preview-llama-3B
  235. 44.9 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16#no-thinking
  236. 44.8 upstage/SOLAR-10.7B-v1.0
  237. 44.7 Qwen/Qwen3.5-9B-Base (val)
  238. 44.3 AFM-4.5B
  239. 44.3 Apertus 8B Instruct
  240. 44.1 swiss-ai/Apertus-v1.1-0.5B-Instruct (val)
  241. 44.1 utter-project/EuroLLM-9B
  242. 43.9 Ministral 3 3B Reasoning
  243. 43.8 Qwen/Qwen3.5-2B-Base (val)
  244. 43.6 LFM2.5-1.2B-Thinking
  245. 43.5 allenai/Olmo-3-7B-Instruct
  246. 43.4 utter-project/EuroLLM-9B-Instruct
  247. 42.9 PleIAs/Pleias-3b-Preview
  248. 42.6 allenai/Olmo-3-7B-Instruct-DPO
  249. 42.2 tencent/Hunyuan-7B-Instruct
  250. 42.1 allenai/Olmo-3-7B-Instruct-SFT
  251. 41.8 LiquidAI/LFM2.5-350M (val)
  252. 41.7 google/gemma-4-E4B (val)
  253. 41.7 Gemma 3 1B
  254. 41.5 LiquidAI/LFM2-700M
  255. 41.4 HuggingFaceTB/SmolLM3-3B#no-thinking
  256. 41.0 syvai/claes-v1 (zero-shot)
  257. 39.9 Qwen/Qwen3.5-4B-Base (val)
  258. 39.4 OpenLLM-France/Luciole-1B-Instruct-1.1 (val)
  259. 39.0 google/gemma-4-E2B (val)
  260. 38.7 fastino/GLiNER2.5-Decide (zero-shot)
  261. 36.3 google-bert/bert-base-multilingual-cased (val)
  262. 35.9 OmniJev/OneJev1.6-0.8B (zero-shot)
  263. 35.8 jhu-clsp/mmBERT-small
  264. 34.8 LFM2.5-1.2B-Instruct
  265. 33.8 kurakurai/Luth-LFM2-350M (val)
  266. 32.5 LiquidAI/LFM2.5-1.2B-Base
  267. 31.9 FacebookAI/xlm-roberta-large
  268. 29.8 microsoft/mdeberta-v3-base
  269. 29.4 SmolLM2-360M
  270. 29.0 PleIAs/Pleias-1.2b-Preview
  271. 28.4 AI-Sweden-Models/ModernBERT-large
  272. 27.4 utter-project/EuroMoE-2.6B-A0.6B-Preview
  273. 25.6 EuroBERT/EuroBERT-210m
  274. 25.4 arcee-ai/Trinity-Nano-Base
  275. 25.0 allenai/Olmo-3-1125-32B
  276. 24.1 convaiinnovations/laya (zero-shot)
  277. 23.7 AI-Sweden-Models/bert-large-nordic-pile-1M-steps
  278. 21.5 arcee-ai/Trinity-Nano-Preview
  279. 21.2 LiquidAI/LFM2-350M
  280. 21.0 EuroBERT/EuroBERT-610m
  281. 20.6 convaiinnovations/laya-multilingual (zero-shot)
  282. 19.7 jaredpalmer/kev-0.5b (zero-shot)
  283. 19.6 mii-llm/nesso-0.4B-agentic (val)
  284. 19.3 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview
  285. 18.9 answerdotai/ModernBERT-base
  286. 17.7 Qwen/Qwen3.5-0.8B-Base (val)
  287. 17.6 Gemma 3 270M
  288. 17.5 answerdotai/ModernBERT-large
  289. 17.4 EuropeanParliament/EUBERT
  290. 16.4 PleIAs/Pleias-350m-Preview
  291. 15.9 OpenLLM-France/Luciole-1B-Base (val)
  292. 15.8 speakleash/Bielik-11B-v3-Base-20250730
  293. 15.2 occiglot/occiglot-7b-eu5
  294. 15.2 BSC-LT/salamandra-7b
  295. 13.7 google-bert/bert-base-multilingual-uncased (val)
  296. 11.5 allenai/Olmo-3-7B-Think
  297. 10.6 fresh-xlm-roberta-base
  298. 10.6 baidu/ERNIE-4.5-0.3B-PT
  299. 9.84 HuggingFaceTB/SmolLM2-360M-Instruct
  300. 9.15 mii-llm/zagreus-0.4B-ita (val)
  301. 9.04 meta-llama/Llama-3.2-1B-Instruct
  302. 8.87 SmolLM2-135M
  303. 8.78 TildeAI/TildeOpen-30b (val)
  304. 8.64 distilbert/distilbert-base-multilingual-cased
  305. 7.08 LiquidAI/LFM2.5-8B-A1B (val)
  306. 6.13 openbmb/MiniCPM5-1B (val)
  307. 4.50 Geotrend/distilbert-base-25lang-cased
  308. 3.30 HuggingFaceTB/SmolLM2-135M-Instruct
  309. 3.04 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base
  310. 1.35 openGPT-X/Teuken-7B-instruct-v0.6 (val)
  311. 1.08 AIDC-AI/Marco-Mini-Global-Base (val)
  312. 0.99 inclusionAI/Ling-mini-base-2.0 (val)
  313. 0.57 inclusionAI/Ling-mini-2.0 (val)
  314. 0.51 PleIAs/Monad
  315. 0.25 Muse-research/Muse-3B (val)
  316. 0.14 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct
  317. 0.0 fresh-electra-small
  318. 0.0 openGPT-X/Teuken-7B-base-v0.6 (val)
  319. 0.0 PleIAs/Baguettotron@refs/pr/6#thinking
  320. 0.0 utter-project/EuroLLM-1.7B (val)
  1. 1 openeurollm/oellm-9b-128k-theta32m-v3 (val) val 78.3%
  2. 2 Qwen/Qwen3.5-9B (val) val 77.2%
  3. 3 Qwen/Qwen3.5-27B (val) val 76.9%
  4. 4 openai/gpt-5.4-mini-2026-03-17#high (zero-shot, val) zero-shot, val 76.1%
  5. 5 CohereLabs/tiny-aya-base (val) val 76%
  6. 6 zai-org/GLM-4.7-Flash (val) val 76%
  7. 7 gpt-5.6-sol (zero-shot, val) zero-shot, val 75.9%
  8. 8 CohereLabs/tiny-aya-global (val) val 75.2%
  9. 9 swiss-ai/Apertus-v1.1-4B (val) val 75.1%
  10. 10 CohereLabs/tiny-aya-water (val) val 74.4%
  11. 11 gemini/gemini-3.6-flash (zero-shot, val) zero-shot, val 74.1%
  12. 12 Qwen/Qwen3.6-27B-FP8 (val) val 74%
  13. 13 gemini/gemini-3-flash-preview#thinking (zero-shot, val) zero-shot, val 74%
  14. 14 gpt-5.6-terra (zero-shot, val) zero-shot, val 74%
  15. 15 gpt-5-mini-2025-08-07 (zero-shot, val) zero-shot, val 73.8%
  16. 16 openai/gpt-5.4-mini-2026-03-17#medium (zero-shot, val) zero-shot, val 73.8%
  17. 17 AI-Sweden-Models/Llama-3-8B-instruct (val) val 73.7%
  18. 18 gemini/gemini-3.7-flash (zero-shot, val) zero-shot, val 73.7%
  19. 19 speakleash/Bielik-11B-v2.3-Instruct (val) val 73.6%
  20. 20 gpt-6.2-sol (zero-shot, val) zero-shot, val 73.4%
  21. 21 Qwen/Qwen3.5-27B-FP8 (val) val 73.1%
  22. 22 Qwen/Qwen3.5-35B-A3B-FP8 (val) val 72.8%
  23. 23 Qwen/Qwen3.8-27B (val) val 72.7%
  24. 24 AI-Sweden-Models/Llama-3-8B-instruct (zero-shot, val) zero-shot, val 72.6%
  25. 25 Qwen/Qwen3.5-4B (val) val 72.5%
  26. 26 GadflyII/GLM-4.7-Flash-NVFP4 (val) val 72.5%
  27. 27 gpt-6-sol (zero-shot, val) zero-shot, val 72.4%
  28. 28 mistralai/Ministral-3-14B-Instruct-2512 (zero-shot, val) zero-shot, val 72.4%
  29. 29 openai/gpt-5.4-mini-2026-03-17#low (zero-shot, val) zero-shot, val 72.4%
  30. 30 google/gemma-4-31B-it (val) val 72.4%
  31. 31 gpt-4.1-2025-04-14 (zero-shot, val) zero-shot, val 72.4%
  32. 32 mistralai/Ministral-3-14B-Base-2512 (val) val 72.2%
  33. 33 claude-sonnet-4-6 (zero-shot, val) zero-shot, val 72.2%
  34. 34 gpt-6-astra (zero-shot, val) zero-shot, val 72%
  35. 35 OpenLLM-France/Lucie-7B-Instruct-v1.1 (val) val 72%
  36. 36 mistralai/Ministral-3-8B-Base-2512 (val) val 71.9%
  37. 37 mistralai/Ministral-3-14B-Reasoning-2512 (val) val 71.8%
  38. 38 swiss-ai/Apertus-8B-2509 71.8%
  39. 39 gpt-6.1-sol (zero-shot, val) zero-shot, val 71.7%
  40. 40 google/gemma-4-26B-A4B-it (val) val 71.5%
  41. 41 gpt-5-2025-08-07 (zero-shot, val) zero-shot, val 71.5%
  42. 42 gpt-6.2-astra (zero-shot, val) zero-shot, val 71.3%
  43. 43 swiss-ai/Apertus-v1.1-4B-Instruct (val) val 71.1%
  44. 44 mistralai/Ministral-3-8B-Instruct-2512 (val) val 71%
  45. 45 swiss-ai/Apertus-v1.5-70B (zero-shot, val) zero-shot, val 71%
  46. 46 latam-gpt/Llama-3.1-70B-LatamGPT-SFT-1.0 (val) val 70.7%
  47. 47 openai/gpt-5.4-nano-2026-03-17#low (zero-shot, val) zero-shot, val 70.7%
  48. 48 openeurollm/prelude-base (val) val 70.7%
  49. 49 mistralai/Mistral-Small-4-119B-2603 (zero-shot, val) zero-shot, val 70.5%
  50. 50 gpt-6-luna (zero-shot, val) zero-shot, val 70.3%
  51. 51 mistralai/Mistral-Medium-3.5-128B (zero-shot, val) zero-shot, val 70.3%
  52. 52 CohereLabs/aya-23-8B (val) val 70.2%
  53. 53 deepseek/deepseek-flash#low (zero-shot, val) zero-shot, val 69.7%
  54. 54 zai-org/GLM-5.3-Flash (val) val 69.5%
  55. 55 Apertus 70B 69.5%
  56. 56 Llama 3.1-8B 69.5%
  57. 57 Qwen/Qwen3.5-2B (val) val 69.4%
  58. 58 domyn/Domyn-Small-v1.0 (val) val 69.4%
  59. 59 openai/gpt-5.4-nano-2026-03-17#medium (zero-shot, val) zero-shot, val 69.3%
  60. 60 gemini/gemini-3.5-flash-lite (zero-shot, val) zero-shot, val 69%
  61. 61 mistralai/Mistral-Small-4-119B-2603 (val) val 68.9%
  62. 62 gemini/gemini-2.5-pro (zero-shot, val) zero-shot, val 68.7%
  63. 63 gemini/gemini-2.5-flash-lite#thinking (zero-shot, val) zero-shot, val 68.6%
  64. 64 openai/gpt-5.4-mini-2026-03-17#none (zero-shot, val) zero-shot, val 68.6%
  65. 65 gemini/gemini-2.5-flash#thinking (zero-shot, val) zero-shot, val 68.5%
  66. 66 OpenLLM-France/Luciole-8B-Base (val) val 68.5%
  67. 67 NbAiLab/borealis2-31b-preview (zero-shot, val) zero-shot, val 68.5%
  68. 68 claude-haiku-4-5-20251001 (zero-shot, val) zero-shot, val 68.5%
  69. 69 openai/gpt-5.4-2026-03-05#none (zero-shot, val) zero-shot, val 68.4%
  70. 70 gpt-5-nano-2025-08-07 (zero-shot, val) zero-shot, val 68.3%
  71. 71 utter-project/EuroLLM-22B-2512 68.3%
  72. 72 Unbabel/Tower-Plus-72B (val) val 68.2%
  73. 73 claude-sonnet-4-5-20250929#no-thinking (zero-shot, val) zero-shot, val 68.1%
  74. 74 OpenLLM-France/Luciole-8B-Instruct-1.1 (val) val 68.1%
  75. 75 openai/gpt-5.4-nano-2026-03-17#none (zero-shot, val) zero-shot, val 68.1%
  76. 76 openai/gpt-5.6-luna (zero-shot, val) zero-shot, val 68%
  77. 77 xai/grok-4-1-fast-reasoning (zero-shot, val) zero-shot, val 67.8%
  78. 78 gemini/gemini-3.1-flash-lite-preview (zero-shot, val) zero-shot, val 67.7%
  79. 79 Gemma 3 12B PT 67.5%
  80. 80 openeurollm/prelude-long (val) val 67.4%
  81. 81 NbAiLab/borealis2-31b-preview (val) val 67.3%
  82. 82 nvidia/Nemotron-Cascade-2-30B-A3B (val) val 67.3%
  83. 83 gemini/gemini-3-pro-preview (zero-shot, val) zero-shot, val 67.3%
  84. 84 OpenLLM-France/Luciole-23B-Instruct-1.1 (val) val 67.2%
  85. 85 Llama 3.1-70B 67.2%
  86. 86 gpt-5.2-2025-12-11 (zero-shot, val) zero-shot, val 67.2%
  87. 87 swiss-ai/Apertus-v1.5-70B (val) val 67.1%
  88. 88 mistralai/Mistral-Medium-3.5-128B (val) val 67.1%
  89. 89 utter-project/EuroLLM-22B-Instruct-2512 67.1%
  90. 90 openai/gpt-5.4-nano-2026-03-17#high (zero-shot, val) zero-shot, val 67%
  91. 91 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (val) val 66.9%
  92. 92 google/gemma-4-12B (val) val 66.7%
  93. 93 mii-llm/nesso-4B 66.7%
  94. 94 zai-org/GLM-5.3-Flash (zero-shot, val) zero-shot, val 66.5%
  95. 95 TrustLLMeu/baseline-7-8b_2-3t-tokens_llama 66.5%
  96. 96 CohereLabs/aya-expanse-8b (val) val 66.5%
  97. 97 mistralai/Mistral-Large-3-675B-Instruct-2512 (zero-shot, val) zero-shot, val 66.2%
  98. 98 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 (val) val 66.2%
  99. 99 gemini/gemini-2.5-flash#no-thinking (zero-shot, val) zero-shot, val 66%
  100. 100 gemini/gemini-3-flash-preview#no-thinking (zero-shot, val) zero-shot, val 66%
  101. 101 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 (val) val 66%
  102. 102 danish-foundation-models/DFM-Mimir-v1.8-Mini (zero-shot, val) zero-shot, val 66%
  103. 103 OpenLLM-France/Lucie-7B-Instruct-human-data (val) val 65.9%
  104. 104 intfloat/multilingual-e5-large 65.8%
  105. 105 openai/gpt-oss-120b (val) val 65.5%
  106. 106 mistralai/Ministral-3-3B-Base-2512 65.5%
  107. 107 danish-foundation-models/DFM-Mimir-v1.8 (zero-shot, val) zero-shot, val 65.5%
  108. 108 utter-project/EuroLLM-22B-Preview 65.5%
  109. 109 danish-foundation-models/DFM-Mimir (zero-shot, val) zero-shot, val 65.3%
  110. 110 speakleash/Bielik-11B-v2.6-Instruct 65.3%
  111. 111 mistralai/Ministral-3-14B-Instruct-2512 (val) val 65.1%
  112. 112 swiss-ai/Apertus-v1.1-1.5B (val) val 64.5%
  113. 113 NbAiLab/borealis2-26b-a4b-preview (zero-shot, val) zero-shot, val 64.4%
  114. 114 xai/grok-4.20-beta-0309-reasoning (zero-shot, val) zero-shot, val 64.2%
  115. 115 BSC-LT/ALIA-40b 64%
  116. 116 deepseek/deepseek-flash#no-thinking (val) val 64%
  117. 117 Gemma 3 4B PT 63.7%
  118. 118 google/gemma-4-12B-it (val) val 63.6%
  119. 119 Qwen3-4B-Thinking-2507 63.5%
  120. 120 Qwen/Qwen3.5-35B-A3B-Base (val) val 63.4%
  121. 121 claude-sonnet-4-5-20250929#thinking (zero-shot, val) zero-shot, val 63.3%
  122. 122 NbAiLab/borealis2-26b-a4b-preview (val) val 62.9%
  123. 123 swiss-ai/Apertus-v1.1-0.5B (val) val 62.9%
  124. 124 ibm-granite/granite-4.0-h-tiny 62.7%
  125. 125 mistralai/Mistral-Large-3-675B-Instruct-2512 (val) val 62.4%
  126. 126 Gemma 3 27B PT 62.3%
  127. 127 gpt-4.1-mini-2025-04-14 (zero-shot, val) zero-shot, val 62.3%
  128. 128 allenai/Olmo-3.1-32B-Think (val) val 62.1%
  129. 129 Granite 4.0 H Small 62%
  130. 130 LiquidAI/LFM2.5-8B-A1B-Base (val) val 61.9%
  131. 131 OpenLLM-Ro/RoMistral-7b-Instruct-DPO 61.8%
  132. 132 mistralai/Mistral-Small-3.1-24B-Base-2503 61.7%
  133. 133 OpenLLM-Ro/RoLlama3.1-8b-Instruct-DPO 61.3%
  134. 134 typesafe/jev-1.13.0 (zero-shot) zero-shot 61.2%
  135. 135 amalia-llm/AMALIA-9B-0626-DPO (zero-shot, val) zero-shot, val 61.1%
  136. 136 gemini/gemini-2.5-flash-lite#no-thinking (zero-shot, val) zero-shot, val 61.1%
  137. 137 danish-foundation-models/DFM-Mimir-v1.8 (val) val 61%
  138. 138 swiss-ai/Apertus-v1.5-8B (zero-shot, val) zero-shot, val 60.9%
  139. 139 speakleash/Bielik-11B-v2.3-Instruct 60.9%
  140. 140 allenai/Llama-3.1-Tulu-3-8B-SFT 60.8%
  141. 141 utter-project/EuroLLM-1.7B-Instruct (val) val 60.5%
  142. 142 danish-foundation-models/DFM-Mimir-v1.8-Mini (val) val 60.5%
  143. 143 deepcogito/cogito-v1-preview-qwen-14B 60.3%
  144. 144 mistralai/Mistral-Small-3.2-24B-Instruct-2506 60%
  145. 145 Gemma 3 1B PT 59.9%
  146. 146 gpt-oss-20b medium 59.7%
  147. 147 amalia-llm/AMALIA-9B-0626-SFT (zero-shot, val) zero-shot, val 59.6%
  148. 148 BSC-LT/salamandra-7b-instruct 59.6%
  149. 149 utter-project/EuroLLM-9B-Instruct-2512 (val) val 59.6%
  150. 150 syvai/claes-preview (zero-shot) zero-shot 59.5%
  151. 151 Qwen/Qwen3-1.7B#no-thinking 59.3%
  152. 152 speakleash/Bielik-One-4B-r1@567d29b0f15df28794acf0f6187f9216860d4551 (zero-shot) zero-shot 59.1%
  153. 153 google/gemma-4-31B (val) val 59%
  154. 154 NbAiLab/borealis2-e4b-preview (zero-shot, val) zero-shot, val 58.9%
  155. 155 danish-foundation-models/DFM-Mimir (val) val 58.9%
  156. 156 deepcogito/cogito-v1-preview-qwen-32B 58.9%
  157. 157 amalia-llm/AMALIA-9B-0626-DPO (val) val 58.7%
  158. 158 google/gemma-4-E4B-it (val) val 58.6%
  159. 159 Llama 3.3 70B Instruct 58.6%
  160. 160 speakleash/Bielik-One-11B-v3.1-r1@75a2d83890e3c68bf7fac2fe5bbe2bff6372036c (zero-shot) zero-shot 58.5%
  161. 161 meta-llama/Llama-3.1-8B-Instruct 58.4%
  162. 162 Qwen/Qwen3-32B#no-thinking 58.4%
  163. 163 Qwen/Qwen3-4B#no-thinking 58.4%
  164. 164 HuggingFaceTB/SmolLM3-3B-Base 58.4%
  165. 165 speakleash/Bielik-Minitron-7B-v3.0-Instruct (val) val 58.4%
  166. 166 mistralai/Mistral-Small-3.1-24B-Instruct-2503 58%
  167. 167 allenai/Olmo-3.1-32B-Instruct-SFT 57.6%
  168. 168 google/gemma-4-E2B-it (val) val 57.4%
  169. 169 gpt-4.1-nano-2025-04-14 (zero-shot, val) zero-shot, val 57.3%
  170. 170 LGAI-EXAONE/EXAONE-4.0-32B (val) val 57.3%
  171. 171 NbAiLab/borealis2-e2b-preview (zero-shot, val) zero-shot, val 57.2%
  172. 172 occiglot/occiglot-7b-eu5-instruct 57.2%
  173. 173 NbAiLab/borealis2-e2b-preview (val) val 56.9%
  174. 174 Gemma 3 4B 56.5%
  175. 175 danish-foundation-models/DFM-Mimir-v1.5 (val) val 56.4%
  176. 176 Qwen/Qwen3-14B#no-thinking 56.3%
  177. 177 swiss-ai/Apertus-v1.5-8B (val) val 56.2%
  178. 178 Qwen/Qwen2.5-1.5B-Instruct 56.1%
  179. 179 mistralai/Ministral-3-3B-Instruct-2512 (zero-shot, val) zero-shot, val 56%
  180. 180 Qwen2.5-72B 55.9%
  181. 181 speakleash/Bielik-1.5B-v3.0-Instruct 55.8%
  182. 182 Cogito v1 Preview 70B 55.8%
  183. 183 speakleash/Bielik-11B-v3.0-Instruct (val) val 55.8%
  184. 184 Qwen/Qwen3-8B#no-thinking 55.8%
  185. 185 OpenLLM-Ro/RoGemma2-9b-Instruct-DPO 55.7%
  186. 186 kurakurai/Luth-LFM2-1.2B (val) val 55.6%
  187. 187 amalia-llm/AMALIA-9B-0626-SFT (val) val 55.4%
  188. 188 granite-4.0-micro 55.2%
  189. 189 Voxtral Small 55.2%
  190. 190 utter-project/EuroLLM-22B-Instruct-Preview 54.8%
  191. 191 Unbabel/Tower-Plus-9B (val) val 54.6%
  192. 192 ibm-granite/granite-4.0-h-micro 54.5%
  193. 193 Gemma 3 27B 54.5%
  194. 194 Llama 3.2 1B 54.5%
  195. 195 OpenLLM-France/Luciole-23B-Base (val) val 54.4%
  196. 196 deepcogito/cogito-v2-preview-llama-70B 54.4%
  197. 197 NbAiLab/borealis2-e4b-preview (val) val 53.8%
  198. 198 speakleash/Bielik-11B-v3.1-Instruct (val) val 53.6%
  199. 199 google/rembert 53.5%
  200. 200 baidu/ERNIE-4.5-21B-A3B-PT 53.3%
  201. 201 deepcogito/cogito-v1-preview-llama-8B 53.2%
  202. 202 yulan-team/YuLan-Mini-Instruct 53.2%
  203. 203 danish-foundation-models/DFM-Mimir-v1.5 (zero-shot, val) zero-shot, val 53.2%
  204. 204 Qwen/Qwen3-0.6B#no-thinking 52.7%
  205. 205 meta-llama/Llama-3.2-3B-Instruct 52.6%
  206. 206 arcee-ai/AFM-4.5B-Base 52.2%
  207. 207 Gemma 3 12B 52.2%
  208. 208 mistralai/Ministral-3-3B-Instruct-2512 (val) val 51.9%
  209. 209 google/gemma-4-26B-A4B (val) val 51.9%
  210. 210 allenai/Olmo-3.1-32B-Instruct-DPO 51.8%
  211. 211 Olmo 3.1 32B Instruct 51.2%
  212. 212 LFM2-1.2B 51.1%
  213. 213 setu4993/LaBSE 51.1%
  214. 214 Aleph-Alpha/Kolibri-1 (zero-shot, val) zero-shot, val 51%
  215. 215 intfloat/multilingual-e5-base 51%
  216. 216 Aleph-Alpha/Kolibri-1 (val) val 50.6%
  217. 217 jhu-clsp/mmBERT-base 50.3%
  218. 218 mistralai/Ministral-3-8B-Reasoning-2512 50.3%
  219. 219 speakleash/Bielik-4.5B-v3.0-Instruct 50.2%
  220. 220 Unbabel/Tower-Plus-2B (val) val 49.8%
  221. 221 kurakurai/Luth-0.6B-Instruct 49%
  222. 222 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 (val) val 48.8%
  223. 223 LiquidAI/LFM2-2.6B 48.8%
  224. 224 Gemma 3n E2B 47.6%
  225. 225 OLMo 2 32B Instruct 46.7%
  226. 226 Trinity Mini 46.6%
  227. 227 swiss-ai/Apertus-v1.1-1.5B-Instruct (val) val 46.3%
  228. 228 sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 46.2%
  229. 229 Qwen3-0.6B 46.1%
  230. 230 Qwen/Qwen3.5-0.8B (val) val 46.1%
  231. 231 Gemma 3n E4B 46%
  232. 232 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 45.5%
  233. 233 allenai/Olmo-3-1025-7B 45.5%
  234. 234 deepcogito/cogito-v1-preview-llama-3B 44.9%
  235. 235 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16#no-thinking 44.9%
  236. 236 upstage/SOLAR-10.7B-v1.0 44.8%
  237. 237 Qwen/Qwen3.5-9B-Base (val) val 44.7%
  238. 238 AFM-4.5B 44.3%
  239. 239 Apertus 8B Instruct 44.3%
  240. 240 swiss-ai/Apertus-v1.1-0.5B-Instruct (val) val 44.1%
  241. 241 utter-project/EuroLLM-9B 44.1%
  242. 242 Ministral 3 3B Reasoning 43.9%
  243. 243 Qwen/Qwen3.5-2B-Base (val) val 43.8%
  244. 244 LFM2.5-1.2B-Thinking 43.6%
  245. 245 allenai/Olmo-3-7B-Instruct 43.5%
  246. 246 utter-project/EuroLLM-9B-Instruct 43.4%
  247. 247 PleIAs/Pleias-3b-Preview 42.9%
  248. 248 allenai/Olmo-3-7B-Instruct-DPO 42.6%
  249. 249 tencent/Hunyuan-7B-Instruct 42.2%
  250. 250 allenai/Olmo-3-7B-Instruct-SFT 42.1%
  251. 251 LiquidAI/LFM2.5-350M (val) val 41.8%
  252. 252 google/gemma-4-E4B (val) val 41.7%
  253. 253 Gemma 3 1B 41.7%
  254. 254 LiquidAI/LFM2-700M 41.5%
  255. 255 HuggingFaceTB/SmolLM3-3B#no-thinking 41.4%
  256. 256 syvai/claes-v1 (zero-shot) zero-shot 41%
  257. 257 Qwen/Qwen3.5-4B-Base (val) val 39.9%
  258. 258 OpenLLM-France/Luciole-1B-Instruct-1.1 (val) val 39.4%
  259. 259 google/gemma-4-E2B (val) val 39%
  260. 260 fastino/GLiNER2.5-Decide (zero-shot) zero-shot 38.7%
  261. 261 google-bert/bert-base-multilingual-cased (val) val 36.3%
  262. 262 OmniJev/OneJev1.6-0.8B (zero-shot) zero-shot 35.9%
  263. 263 jhu-clsp/mmBERT-small 35.8%
  264. 264 LFM2.5-1.2B-Instruct 34.8%
  265. 265 kurakurai/Luth-LFM2-350M (val) val 33.8%
  266. 266 LiquidAI/LFM2.5-1.2B-Base 32.5%
  267. 267 FacebookAI/xlm-roberta-large 31.9%
  268. 268 microsoft/mdeberta-v3-base 29.8%
  269. 269 SmolLM2-360M 29.4%
  270. 270 PleIAs/Pleias-1.2b-Preview 29%
  271. 271 AI-Sweden-Models/ModernBERT-large 28.4%
  272. 272 utter-project/EuroMoE-2.6B-A0.6B-Preview 27.4%
  273. 273 EuroBERT/EuroBERT-210m 25.6%
  274. 274 arcee-ai/Trinity-Nano-Base 25.4%
  275. 275 allenai/Olmo-3-1125-32B 25%
  276. 276 convaiinnovations/laya (zero-shot) zero-shot 24.1%
  277. 277 AI-Sweden-Models/bert-large-nordic-pile-1M-steps 23.7%
  278. 278 arcee-ai/Trinity-Nano-Preview 21.5%
  279. 279 LiquidAI/LFM2-350M 21.2%
  280. 280 EuroBERT/EuroBERT-610m 21%
  281. 281 convaiinnovations/laya-multilingual (zero-shot) zero-shot 20.6%
  282. 282 jaredpalmer/kev-0.5b (zero-shot) zero-shot 19.7%
  283. 283 mii-llm/nesso-0.4B-agentic (val) val 19.6%
  284. 284 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview 19.3%
  285. 285 answerdotai/ModernBERT-base 18.9%
  286. 286 Qwen/Qwen3.5-0.8B-Base (val) val 17.7%
  287. 287 Gemma 3 270M 17.6%
  288. 288 answerdotai/ModernBERT-large 17.5%
  289. 289 EuropeanParliament/EUBERT 17.4%
  290. 290 PleIAs/Pleias-350m-Preview 16.4%
  291. 291 OpenLLM-France/Luciole-1B-Base (val) val 15.9%
  292. 292 speakleash/Bielik-11B-v3-Base-20250730 15.8%
  293. 293 occiglot/occiglot-7b-eu5 15.2%
  294. 294 BSC-LT/salamandra-7b 15.2%
  295. 295 google-bert/bert-base-multilingual-uncased (val) val 13.7%
  296. 296 allenai/Olmo-3-7B-Think 11.5%
  297. 297 fresh-xlm-roberta-base 10.6%
  298. 298 baidu/ERNIE-4.5-0.3B-PT 10.6%
  299. 299 HuggingFaceTB/SmolLM2-360M-Instruct 9.84%
  300. 300 mii-llm/zagreus-0.4B-ita (val) val 9.15%
  301. 301 meta-llama/Llama-3.2-1B-Instruct 9.04%
  302. 302 SmolLM2-135M 8.87%
  303. 303 TildeAI/TildeOpen-30b (val) val 8.78%
  304. 304 distilbert/distilbert-base-multilingual-cased 8.64%
  305. 305 LiquidAI/LFM2.5-8B-A1B (val) val 7.08%
  306. 306 openbmb/MiniCPM5-1B (val) val 6.13%
  307. 307 Geotrend/distilbert-base-25lang-cased 4.50%
  308. 308 HuggingFaceTB/SmolLM2-135M-Instruct 3.30%
  309. 309 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base 3.04%
  310. 310 openGPT-X/Teuken-7B-instruct-v0.6 (val) val 1.35%
  311. 311 AIDC-AI/Marco-Mini-Global-Base (val) val 1.08%
  312. 312 inclusionAI/Ling-mini-base-2.0 (val) val 0.99%
  313. 313 inclusionAI/Ling-mini-2.0 (val) val 0.57%
  314. 314 PleIAs/Monad 0.51%
  315. 315 Muse-research/Muse-3B (val) val 0.25%
  316. 316 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct 0.14%
  317. 317 fresh-electra-small 0%
  318. 318 openGPT-X/Teuken-7B-base-v0.6 (val) val 0%
  319. 319 PleIAs/Baguettotron@refs/pr/6#thinking 0%
  320. 320 utter-project/EuroLLM-1.7B (val) val 0%

Source: Official GuiaCat leaderboard

Source: Official GuiaCat leaderboard

#EntryScoreSettingDateSource
1 openeurollm/oellm-9b-128k-theta32m-v3 (val) 78.3% val Official leaderboard
2 Qwen/Qwen3.5-9B (val) 77.2% val Official leaderboard
3 Qwen/Qwen3.5-27B (val) 76.9% val Official leaderboard
4 openai/gpt-5.4-mini-2026-03-17#high (zero-shot, val) 76.1% zero-shot, val Official leaderboard
5 CohereLabs/tiny-aya-base (val) 76% val Official leaderboard
6 zai-org/GLM-4.7-Flash (val) 76% val Official leaderboard
7 gpt-5.6-sol (zero-shot, val) 75.9% zero-shot, val Official leaderboard
8 CohereLabs/tiny-aya-global (val) 75.2% val Official leaderboard
9 swiss-ai/Apertus-v1.1-4B (val) 75.1% val Official leaderboard
10 CohereLabs/tiny-aya-water (val) 74.4% val Official leaderboard
11 gemini/gemini-3.6-flash (zero-shot, val) 74.1% zero-shot, val Official leaderboard
12 Qwen/Qwen3.6-27B-FP8 (val) 74% val Official leaderboard
13 gemini/gemini-3-flash-preview#thinking (zero-shot, val) 74% zero-shot, val Official leaderboard
14 gpt-5.6-terra (zero-shot, val) 74% zero-shot, val Official leaderboard
15 gpt-5-mini-2025-08-07 (zero-shot, val) 73.8% zero-shot, val Official leaderboard
16 openai/gpt-5.4-mini-2026-03-17#medium (zero-shot, val) 73.8% zero-shot, val Official leaderboard
17 AI-Sweden-Models/Llama-3-8B-instruct (val) 73.7% val Official leaderboard
18 gemini/gemini-3.7-flash (zero-shot, val) 73.7% zero-shot, val Official leaderboard
19 speakleash/Bielik-11B-v2.3-Instruct (val) 73.6% val Official leaderboard
20 gpt-6.2-sol (zero-shot, val) 73.4% zero-shot, val Official leaderboard
21 Qwen/Qwen3.5-27B-FP8 (val) 73.1% val Official leaderboard
22 Qwen/Qwen3.5-35B-A3B-FP8 (val) 72.8% val Official leaderboard
23 Qwen/Qwen3.8-27B (val) 72.7% val Official leaderboard
24 AI-Sweden-Models/Llama-3-8B-instruct (zero-shot, val) 72.6% zero-shot, val Official leaderboard
25 Qwen/Qwen3.5-4B (val) 72.5% val Official leaderboard
26 GadflyII/GLM-4.7-Flash-NVFP4 (val) 72.5% val Official leaderboard
27 gpt-6-sol (zero-shot, val) 72.4% zero-shot, val Official leaderboard
28 mistralai/Ministral-3-14B-Instruct-2512 (zero-shot, val) 72.4% zero-shot, val Official leaderboard
29 openai/gpt-5.4-mini-2026-03-17#low (zero-shot, val) 72.4% zero-shot, val Official leaderboard
30 google/gemma-4-31B-it (val) 72.4% val Official leaderboard
31 gpt-4.1-2025-04-14 (zero-shot, val) 72.4% zero-shot, val Official leaderboard
32 mistralai/Ministral-3-14B-Base-2512 (val) 72.2% val Official leaderboard
33 claude-sonnet-4-6 (zero-shot, val) 72.2% zero-shot, val Official leaderboard
34 gpt-6-astra (zero-shot, val) 72% zero-shot, val Official leaderboard
35 OpenLLM-France/Lucie-7B-Instruct-v1.1 (val) 72% val Official leaderboard
36 mistralai/Ministral-3-8B-Base-2512 (val) 71.9% val Official leaderboard
37 mistralai/Ministral-3-14B-Reasoning-2512 (val) 71.8% val Official leaderboard
38 swiss-ai/Apertus-8B-2509 71.8% Official leaderboard
39 gpt-6.1-sol (zero-shot, val) 71.7% zero-shot, val Official leaderboard
40 google/gemma-4-26B-A4B-it (val) 71.5% val Official leaderboard
41 gpt-5-2025-08-07 (zero-shot, val) 71.5% zero-shot, val Official leaderboard
42 gpt-6.2-astra (zero-shot, val) 71.3% zero-shot, val Official leaderboard
43 swiss-ai/Apertus-v1.1-4B-Instruct (val) 71.1% val Official leaderboard
44 mistralai/Ministral-3-8B-Instruct-2512 (val) 71% val Official leaderboard
45 swiss-ai/Apertus-v1.5-70B (zero-shot, val) 71% zero-shot, val Official leaderboard
46 latam-gpt/Llama-3.1-70B-LatamGPT-SFT-1.0 (val) 70.7% val Official leaderboard
47 openai/gpt-5.4-nano-2026-03-17#low (zero-shot, val) 70.7% zero-shot, val Official leaderboard
48 openeurollm/prelude-base (val) 70.7% val Official leaderboard
49 mistralai/Mistral-Small-4-119B-2603 (zero-shot, val) 70.5% zero-shot, val Official leaderboard
50 gpt-6-luna (zero-shot, val) 70.3% zero-shot, val Official leaderboard
51 mistralai/Mistral-Medium-3.5-128B (zero-shot, val) 70.3% zero-shot, val Official leaderboard
52 CohereLabs/aya-23-8B (val) 70.2% val Official leaderboard
53 deepseek/deepseek-flash#low (zero-shot, val) 69.7% zero-shot, val Official leaderboard
54 zai-org/GLM-5.3-Flash (val) 69.5% val Official leaderboard
55 Apertus 70B 69.5% Sep 2025 Official leaderboard
56 Llama 3.1-8B 69.5% Jul 2024 Official leaderboard
57 Qwen/Qwen3.5-2B (val) 69.4% val Official leaderboard
58 domyn/Domyn-Small-v1.0 (val) 69.4% val Official leaderboard
59 openai/gpt-5.4-nano-2026-03-17#medium (zero-shot, val) 69.3% zero-shot, val Official leaderboard
60 gemini/gemini-3.5-flash-lite (zero-shot, val) 69% zero-shot, val Official leaderboard
61 mistralai/Mistral-Small-4-119B-2603 (val) 68.9% val Official leaderboard
62 gemini/gemini-2.5-pro (zero-shot, val) 68.7% zero-shot, val Official leaderboard
63 gemini/gemini-2.5-flash-lite#thinking (zero-shot, val) 68.6% zero-shot, val Official leaderboard
64 openai/gpt-5.4-mini-2026-03-17#none (zero-shot, val) 68.6% zero-shot, val Official leaderboard
65 gemini/gemini-2.5-flash#thinking (zero-shot, val) 68.5% zero-shot, val Official leaderboard
66 OpenLLM-France/Luciole-8B-Base (val) 68.5% val Official leaderboard
67 NbAiLab/borealis2-31b-preview (zero-shot, val) 68.5% zero-shot, val Official leaderboard
68 claude-haiku-4-5-20251001 (zero-shot, val) 68.5% zero-shot, val Official leaderboard
69 openai/gpt-5.4-2026-03-05#none (zero-shot, val) 68.4% zero-shot, val Official leaderboard
70 gpt-5-nano-2025-08-07 (zero-shot, val) 68.3% zero-shot, val Official leaderboard
71 utter-project/EuroLLM-22B-2512 68.3% Official leaderboard
72 Unbabel/Tower-Plus-72B (val) 68.2% val Official leaderboard
73 claude-sonnet-4-5-20250929#no-thinking (zero-shot, val) 68.1% zero-shot, val Official leaderboard
74 OpenLLM-France/Luciole-8B-Instruct-1.1 (val) 68.1% val Official leaderboard
75 openai/gpt-5.4-nano-2026-03-17#none (zero-shot, val) 68.1% zero-shot, val Official leaderboard
76 openai/gpt-5.6-luna (zero-shot, val) 68% zero-shot, val Official leaderboard
77 xai/grok-4-1-fast-reasoning (zero-shot, val) 67.8% zero-shot, val Official leaderboard
78 gemini/gemini-3.1-flash-lite-preview (zero-shot, val) 67.7% zero-shot, val Official leaderboard
79 Gemma 3 12B PT 67.5% Mar 2025 Official leaderboard
80 openeurollm/prelude-long (val) 67.4% val Official leaderboard
81 NbAiLab/borealis2-31b-preview (val) 67.3% val Official leaderboard
82 nvidia/Nemotron-Cascade-2-30B-A3B (val) 67.3% val Official leaderboard
83 gemini/gemini-3-pro-preview (zero-shot, val) 67.3% zero-shot, val Official leaderboard
84 OpenLLM-France/Luciole-23B-Instruct-1.1 (val) 67.2% val Official leaderboard
85 Llama 3.1-70B 67.2% Jul 2024 Official leaderboard
86 gpt-5.2-2025-12-11 (zero-shot, val) 67.2% zero-shot, val Official leaderboard
87 swiss-ai/Apertus-v1.5-70B (val) 67.1% val Official leaderboard
88 mistralai/Mistral-Medium-3.5-128B (val) 67.1% val Official leaderboard
89 utter-project/EuroLLM-22B-Instruct-2512 67.1% Official leaderboard
90 openai/gpt-5.4-nano-2026-03-17#high (zero-shot, val) 67% zero-shot, val Official leaderboard
91 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (val) 66.9% val Official leaderboard
92 google/gemma-4-12B (val) 66.7% val Official leaderboard
93 mii-llm/nesso-4B 66.7% Official leaderboard
94 zai-org/GLM-5.3-Flash (zero-shot, val) 66.5% zero-shot, val Official leaderboard
95 TrustLLMeu/baseline-7-8b_2-3t-tokens_llama 66.5% Official leaderboard
96 CohereLabs/aya-expanse-8b (val) 66.5% val Official leaderboard
97 mistralai/Mistral-Large-3-675B-Instruct-2512 (zero-shot, val) 66.2% zero-shot, val Official leaderboard
98 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 (val) 66.2% val Official leaderboard
99 gemini/gemini-2.5-flash#no-thinking (zero-shot, val) 66% zero-shot, val Official leaderboard
100 gemini/gemini-3-flash-preview#no-thinking (zero-shot, val) 66% zero-shot, val Official leaderboard
101 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 (val) 66% val Official leaderboard
102 danish-foundation-models/DFM-Mimir-v1.8-Mini (zero-shot, val) 66% zero-shot, val Official leaderboard
103 OpenLLM-France/Lucie-7B-Instruct-human-data (val) 65.9% val Official leaderboard
104 intfloat/multilingual-e5-large 65.8% Official leaderboard
105 openai/gpt-oss-120b (val) 65.5% val Official leaderboard
106 mistralai/Ministral-3-3B-Base-2512 65.5% Official leaderboard
107 danish-foundation-models/DFM-Mimir-v1.8 (zero-shot, val) 65.5% zero-shot, val Official leaderboard
108 utter-project/EuroLLM-22B-Preview 65.5% Official leaderboard
109 danish-foundation-models/DFM-Mimir (zero-shot, val) 65.3% zero-shot, val Official leaderboard
110 speakleash/Bielik-11B-v2.6-Instruct 65.3% Official leaderboard
111 mistralai/Ministral-3-14B-Instruct-2512 (val) 65.1% val Official leaderboard
112 swiss-ai/Apertus-v1.1-1.5B (val) 64.5% val Official leaderboard
113 NbAiLab/borealis2-26b-a4b-preview (zero-shot, val) 64.4% zero-shot, val Official leaderboard
114 xai/grok-4.20-beta-0309-reasoning (zero-shot, val) 64.2% zero-shot, val Official leaderboard
115 BSC-LT/ALIA-40b 64% Official leaderboard
116 deepseek/deepseek-flash#no-thinking (val) 64% val Official leaderboard
117 Gemma 3 4B PT 63.7% Mar 2025 Official leaderboard
118 google/gemma-4-12B-it (val) 63.6% val Official leaderboard
119 Qwen3-4B-Thinking-2507 63.5% Aug 2025 Official leaderboard
120 Qwen/Qwen3.5-35B-A3B-Base (val) 63.4% val Official leaderboard
121 claude-sonnet-4-5-20250929#thinking (zero-shot, val) 63.3% zero-shot, val Official leaderboard
122 NbAiLab/borealis2-26b-a4b-preview (val) 62.9% val Official leaderboard
123 swiss-ai/Apertus-v1.1-0.5B (val) 62.9% val Official leaderboard
124 ibm-granite/granite-4.0-h-tiny 62.7% Official leaderboard
125 mistralai/Mistral-Large-3-675B-Instruct-2512 (val) 62.4% val Official leaderboard
126 Gemma 3 27B PT 62.3% Mar 2025 Official leaderboard
127 gpt-4.1-mini-2025-04-14 (zero-shot, val) 62.3% zero-shot, val Official leaderboard
128 allenai/Olmo-3.1-32B-Think (val) 62.1% val Official leaderboard
129 Granite 4.0 H Small 62% Oct 2025 Official leaderboard
130 LiquidAI/LFM2.5-8B-A1B-Base (val) 61.9% val Official leaderboard
131 OpenLLM-Ro/RoMistral-7b-Instruct-DPO 61.8% Official leaderboard
132 mistralai/Mistral-Small-3.1-24B-Base-2503 61.7% Official leaderboard
133 OpenLLM-Ro/RoLlama3.1-8b-Instruct-DPO 61.3% Official leaderboard
134 typesafe/jev-1.13.0 (zero-shot) 61.2% zero-shot Official leaderboard
135 amalia-llm/AMALIA-9B-0626-DPO (zero-shot, val) 61.1% zero-shot, val Official leaderboard
136 gemini/gemini-2.5-flash-lite#no-thinking (zero-shot, val) 61.1% zero-shot, val Official leaderboard
137 danish-foundation-models/DFM-Mimir-v1.8 (val) 61% val Official leaderboard
138 swiss-ai/Apertus-v1.5-8B (zero-shot, val) 60.9% zero-shot, val Official leaderboard
139 speakleash/Bielik-11B-v2.3-Instruct 60.9% Official leaderboard
140 allenai/Llama-3.1-Tulu-3-8B-SFT 60.8% Official leaderboard
141 utter-project/EuroLLM-1.7B-Instruct (val) 60.5% val Official leaderboard
142 danish-foundation-models/DFM-Mimir-v1.8-Mini (val) 60.5% val Official leaderboard
143 deepcogito/cogito-v1-preview-qwen-14B 60.3% Official leaderboard
144 mistralai/Mistral-Small-3.2-24B-Instruct-2506 60% Official leaderboard
145 Gemma 3 1B PT 59.9% Mar 2025 Official leaderboard
146 gpt-oss-20b 59.7% medium Aug 2025 Official leaderboard
147 amalia-llm/AMALIA-9B-0626-SFT (zero-shot, val) 59.6% zero-shot, val Official leaderboard
148 BSC-LT/salamandra-7b-instruct 59.6% Official leaderboard
149 utter-project/EuroLLM-9B-Instruct-2512 (val) 59.6% val Official leaderboard
150 syvai/claes-preview (zero-shot) 59.5% zero-shot Official leaderboard
151 Qwen/Qwen3-1.7B#no-thinking 59.3% Official leaderboard
152 speakleash/Bielik-One-4B-r1@567d29b0f15df28794acf0f6187f9216860d4551 (zero-shot) 59.1% zero-shot Official leaderboard
153 google/gemma-4-31B (val) 59% val Official leaderboard
154 NbAiLab/borealis2-e4b-preview (zero-shot, val) 58.9% zero-shot, val Official leaderboard
155 danish-foundation-models/DFM-Mimir (val) 58.9% val Official leaderboard
156 deepcogito/cogito-v1-preview-qwen-32B 58.9% Official leaderboard
157 amalia-llm/AMALIA-9B-0626-DPO (val) 58.7% val Official leaderboard
158 google/gemma-4-E4B-it (val) 58.6% val Official leaderboard
159 Llama 3.3 70B Instruct 58.6% Dec 2024 Official leaderboard
160 speakleash/Bielik-One-11B-v3.1-r1@75a2d83890e3c68bf7fac2fe5bbe2bff6372036c (zero-shot) 58.5% zero-shot Official leaderboard
161 meta-llama/Llama-3.1-8B-Instruct 58.4% Official leaderboard
162 Qwen/Qwen3-32B#no-thinking 58.4% Official leaderboard
163 Qwen/Qwen3-4B#no-thinking 58.4% Official leaderboard
164 HuggingFaceTB/SmolLM3-3B-Base 58.4% Official leaderboard
165 speakleash/Bielik-Minitron-7B-v3.0-Instruct (val) 58.4% val Official leaderboard
166 mistralai/Mistral-Small-3.1-24B-Instruct-2503 58% Official leaderboard
167 allenai/Olmo-3.1-32B-Instruct-SFT 57.6% Official leaderboard
168 google/gemma-4-E2B-it (val) 57.4% val Official leaderboard
169 gpt-4.1-nano-2025-04-14 (zero-shot, val) 57.3% zero-shot, val Official leaderboard
170 LGAI-EXAONE/EXAONE-4.0-32B (val) 57.3% val Official leaderboard
171 NbAiLab/borealis2-e2b-preview (zero-shot, val) 57.2% zero-shot, val Official leaderboard
172 occiglot/occiglot-7b-eu5-instruct 57.2% Official leaderboard
173 NbAiLab/borealis2-e2b-preview (val) 56.9% val Official leaderboard
174 Gemma 3 4B 56.5% Mar 2025 Official leaderboard
175 danish-foundation-models/DFM-Mimir-v1.5 (val) 56.4% val Official leaderboard
176 Qwen/Qwen3-14B#no-thinking 56.3% Official leaderboard
177 swiss-ai/Apertus-v1.5-8B (val) 56.2% val Official leaderboard
178 Qwen/Qwen2.5-1.5B-Instruct 56.1% Official leaderboard
179 mistralai/Ministral-3-3B-Instruct-2512 (zero-shot, val) 56% zero-shot, val Official leaderboard
180 Qwen2.5-72B 55.9% Sep 2024 Official leaderboard
181 speakleash/Bielik-1.5B-v3.0-Instruct 55.8% Official leaderboard
182 Cogito v1 Preview 70B 55.8% Apr 2025 Official leaderboard
183 speakleash/Bielik-11B-v3.0-Instruct (val) 55.8% val Official leaderboard
184 Qwen/Qwen3-8B#no-thinking 55.8% Official leaderboard
185 OpenLLM-Ro/RoGemma2-9b-Instruct-DPO 55.7% Official leaderboard
186 kurakurai/Luth-LFM2-1.2B (val) 55.6% val Official leaderboard
187 amalia-llm/AMALIA-9B-0626-SFT (val) 55.4% val Official leaderboard
188 granite-4.0-micro 55.2% Oct 2025 Official leaderboard
189 Voxtral Small 55.2% Jul 2025 Official leaderboard
190 utter-project/EuroLLM-22B-Instruct-Preview 54.8% Official leaderboard
191 Unbabel/Tower-Plus-9B (val) 54.6% val Official leaderboard
192 ibm-granite/granite-4.0-h-micro 54.5% Official leaderboard
193 Gemma 3 27B 54.5% Mar 2025 Official leaderboard
194 Llama 3.2 1B 54.5% Sep 2024 Official leaderboard
195 OpenLLM-France/Luciole-23B-Base (val) 54.4% val Official leaderboard
196 deepcogito/cogito-v2-preview-llama-70B 54.4% Official leaderboard
197 NbAiLab/borealis2-e4b-preview (val) 53.8% val Official leaderboard
198 speakleash/Bielik-11B-v3.1-Instruct (val) 53.6% val Official leaderboard
199 google/rembert 53.5% Official leaderboard
200 baidu/ERNIE-4.5-21B-A3B-PT 53.3% Official leaderboard
201 deepcogito/cogito-v1-preview-llama-8B 53.2% Official leaderboard
202 yulan-team/YuLan-Mini-Instruct 53.2% Official leaderboard
203 danish-foundation-models/DFM-Mimir-v1.5 (zero-shot, val) 53.2% zero-shot, val Official leaderboard
204 Qwen/Qwen3-0.6B#no-thinking 52.7% Official leaderboard
205 meta-llama/Llama-3.2-3B-Instruct 52.6% Official leaderboard
206 arcee-ai/AFM-4.5B-Base 52.2% Official leaderboard
207 Gemma 3 12B 52.2% Mar 2025 Official leaderboard
208 mistralai/Ministral-3-3B-Instruct-2512 (val) 51.9% val Official leaderboard
209 google/gemma-4-26B-A4B (val) 51.9% val Official leaderboard
210 allenai/Olmo-3.1-32B-Instruct-DPO 51.8% Official leaderboard
211 Olmo 3.1 32B Instruct 51.2% Dec 2025 Official leaderboard
212 LFM2-1.2B 51.1% Jul 2025 Official leaderboard
213 setu4993/LaBSE 51.1% Official leaderboard
214 Aleph-Alpha/Kolibri-1 (zero-shot, val) 51% zero-shot, val Official leaderboard
215 intfloat/multilingual-e5-base 51% Official leaderboard
216 Aleph-Alpha/Kolibri-1 (val) 50.6% val Official leaderboard
217 jhu-clsp/mmBERT-base 50.3% Official leaderboard
218 mistralai/Ministral-3-8B-Reasoning-2512 50.3% Official leaderboard
219 speakleash/Bielik-4.5B-v3.0-Instruct 50.2% Official leaderboard
220 Unbabel/Tower-Plus-2B (val) 49.8% val Official leaderboard
221 kurakurai/Luth-0.6B-Instruct 49% Official leaderboard
222 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 (val) 48.8% val Official leaderboard
223 LiquidAI/LFM2-2.6B 48.8% Official leaderboard
224 Gemma 3n E2B 47.6% Jun 2025 Official leaderboard
225 OLMo 2 32B Instruct 46.7% Mar 2025 Official leaderboard
226 Trinity Mini 46.6% Dec 2025 Official leaderboard
227 swiss-ai/Apertus-v1.1-1.5B-Instruct (val) 46.3% val Official leaderboard
228 sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 46.2% Official leaderboard
229 Qwen3-0.6B 46.1% Apr 2025 Official leaderboard
230 Qwen/Qwen3.5-0.8B (val) 46.1% val Official leaderboard
231 Gemma 3n E4B 46% Jun 2025 Official leaderboard
232 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 45.5% Official leaderboard
233 allenai/Olmo-3-1025-7B 45.5% Official leaderboard
234 deepcogito/cogito-v1-preview-llama-3B 44.9% Official leaderboard
235 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16#no-thinking 44.9% Official leaderboard
236 upstage/SOLAR-10.7B-v1.0 44.8% Official leaderboard
237 Qwen/Qwen3.5-9B-Base (val) 44.7% val Official leaderboard
238 AFM-4.5B 44.3% Jul 2025 Official leaderboard
239 Apertus 8B Instruct 44.3% Sep 2025 Official leaderboard
240 swiss-ai/Apertus-v1.1-0.5B-Instruct (val) 44.1% val Official leaderboard
241 utter-project/EuroLLM-9B 44.1% Official leaderboard
242 Ministral 3 3B Reasoning 43.9% Dec 2025 Official leaderboard
243 Qwen/Qwen3.5-2B-Base (val) 43.8% val Official leaderboard
244 LFM2.5-1.2B-Thinking 43.6% Jan 2026 Official leaderboard
245 allenai/Olmo-3-7B-Instruct 43.5% Official leaderboard
246 utter-project/EuroLLM-9B-Instruct 43.4% Official leaderboard
247 PleIAs/Pleias-3b-Preview 42.9% Official leaderboard
248 allenai/Olmo-3-7B-Instruct-DPO 42.6% Official leaderboard
249 tencent/Hunyuan-7B-Instruct 42.2% Official leaderboard
250 allenai/Olmo-3-7B-Instruct-SFT 42.1% Official leaderboard
251 LiquidAI/LFM2.5-350M (val) 41.8% val Official leaderboard
252 google/gemma-4-E4B (val) 41.7% val Official leaderboard
253 Gemma 3 1B 41.7% Mar 2025 Official leaderboard
254 LiquidAI/LFM2-700M 41.5% Official leaderboard
255 HuggingFaceTB/SmolLM3-3B#no-thinking 41.4% Official leaderboard
256 syvai/claes-v1 (zero-shot) 41% zero-shot Official leaderboard
257 Qwen/Qwen3.5-4B-Base (val) 39.9% val Official leaderboard
258 OpenLLM-France/Luciole-1B-Instruct-1.1 (val) 39.4% val Official leaderboard
259 google/gemma-4-E2B (val) 39% val Official leaderboard
260 fastino/GLiNER2.5-Decide (zero-shot) 38.7% zero-shot Official leaderboard
261 google-bert/bert-base-multilingual-cased (val) 36.3% val Official leaderboard
262 OmniJev/OneJev1.6-0.8B (zero-shot) 35.9% zero-shot Official leaderboard
263 jhu-clsp/mmBERT-small 35.8% Official leaderboard
264 LFM2.5-1.2B-Instruct 34.8% Jan 2026 Official leaderboard
265 kurakurai/Luth-LFM2-350M (val) 33.8% val Official leaderboard
266 LiquidAI/LFM2.5-1.2B-Base 32.5% Official leaderboard
267 FacebookAI/xlm-roberta-large 31.9% Official leaderboard
268 microsoft/mdeberta-v3-base 29.8% Official leaderboard
269 SmolLM2-360M 29.4% Oct 2024 Official leaderboard
270 PleIAs/Pleias-1.2b-Preview 29% Official leaderboard
271 AI-Sweden-Models/ModernBERT-large 28.4% Official leaderboard
272 utter-project/EuroMoE-2.6B-A0.6B-Preview 27.4% Official leaderboard
273 EuroBERT/EuroBERT-210m 25.6% Official leaderboard
274 arcee-ai/Trinity-Nano-Base 25.4% Official leaderboard
275 allenai/Olmo-3-1125-32B 25% Official leaderboard
276 convaiinnovations/laya (zero-shot) 24.1% zero-shot Official leaderboard
277 AI-Sweden-Models/bert-large-nordic-pile-1M-steps 23.7% Official leaderboard
278 arcee-ai/Trinity-Nano-Preview 21.5% Official leaderboard
279 LiquidAI/LFM2-350M 21.2% Official leaderboard
280 EuroBERT/EuroBERT-610m 21% Official leaderboard
281 convaiinnovations/laya-multilingual (zero-shot) 20.6% zero-shot Official leaderboard
282 jaredpalmer/kev-0.5b (zero-shot) 19.7% zero-shot Official leaderboard
283 mii-llm/nesso-0.4B-agentic (val) 19.6% val Official leaderboard
284 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview 19.3% Official leaderboard
285 answerdotai/ModernBERT-base 18.9% Official leaderboard
286 Qwen/Qwen3.5-0.8B-Base (val) 17.7% val Official leaderboard
287 Gemma 3 270M 17.6% Aug 2025 Official leaderboard
288 answerdotai/ModernBERT-large 17.5% Official leaderboard
289 EuropeanParliament/EUBERT 17.4% Official leaderboard
290 PleIAs/Pleias-350m-Preview 16.4% Official leaderboard
291 OpenLLM-France/Luciole-1B-Base (val) 15.9% val Official leaderboard
292 speakleash/Bielik-11B-v3-Base-20250730 15.8% Official leaderboard
293 occiglot/occiglot-7b-eu5 15.2% Official leaderboard
294 BSC-LT/salamandra-7b 15.2% Official leaderboard
295 google-bert/bert-base-multilingual-uncased (val) 13.7% val Official leaderboard
296 allenai/Olmo-3-7B-Think 11.5% Official leaderboard
297 fresh-xlm-roberta-base 10.6% Official leaderboard
298 baidu/ERNIE-4.5-0.3B-PT 10.6% Official leaderboard
299 HuggingFaceTB/SmolLM2-360M-Instruct 9.84% Official leaderboard
300 mii-llm/zagreus-0.4B-ita (val) 9.15% val Official leaderboard
301 meta-llama/Llama-3.2-1B-Instruct 9.04% Official leaderboard
302 SmolLM2-135M 8.87% Oct 2024 Official leaderboard
303 TildeAI/TildeOpen-30b (val) 8.78% val Official leaderboard
304 distilbert/distilbert-base-multilingual-cased 8.64% Official leaderboard
305 LiquidAI/LFM2.5-8B-A1B (val) 7.08% val Official leaderboard
306 openbmb/MiniCPM5-1B (val) 6.13% val Official leaderboard
307 Geotrend/distilbert-base-25lang-cased 4.50% Official leaderboard
308 HuggingFaceTB/SmolLM2-135M-Instruct 3.30% Official leaderboard
309 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base 3.04% Official leaderboard
310 openGPT-X/Teuken-7B-instruct-v0.6 (val) 1.35% val Official leaderboard
311 AIDC-AI/Marco-Mini-Global-Base (val) 1.08% val Official leaderboard
312 inclusionAI/Ling-mini-base-2.0 (val) 0.99% val Official leaderboard
313 inclusionAI/Ling-mini-2.0 (val) 0.57% val Official leaderboard
314 PleIAs/Monad 0.51% Official leaderboard
315 Muse-research/Muse-3B (val) 0.25% val Official leaderboard
316 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct 0.14% Official leaderboard
317 fresh-electra-small 0% Official leaderboard
318 openGPT-X/Teuken-7B-base-v0.6 (val) 0% val Official leaderboard
319 PleIAs/Baguettotron@refs/pr/6#thinking 0% Official leaderboard
320 utter-project/EuroLLM-1.7B (val) 0% val Official leaderboard
#ModelScoreSettingDateSource
1 Apertus 70B 69.5% Sep 2025 Official leaderboard
2 Llama 3.1-8B 69.5% Jul 2024 Official leaderboard
3 Gemma 3 12B PT 67.5% Mar 2025 Official leaderboard
4 Llama 3.1-70B 67.2% Jul 2024 Official leaderboard
5 Gemma 3 4B PT 63.7% Mar 2025 Official leaderboard
6 Qwen3-4B-Thinking-2507 63.5% Aug 2025 Official leaderboard
7 Gemma 3 27B PT 62.3% Mar 2025 Official leaderboard
8 Granite 4.0 H Small 62% Oct 2025 Official leaderboard
9 Gemma 3 1B PT 59.9% Mar 2025 Official leaderboard
10 gpt-oss-20b 59.7% medium Aug 2025 Official leaderboard
11 Llama 3.3 70B Instruct 58.6% Dec 2024 Official leaderboard
12 Gemma 3 4B 56.5% Mar 2025 Official leaderboard
13 Qwen2.5-72B 55.9% Sep 2024 Official leaderboard
14 Cogito v1 Preview 70B 55.8% Apr 2025 Official leaderboard
15 granite-4.0-micro 55.2% Oct 2025 Official leaderboard
16 Voxtral Small 55.2% Jul 2025 Official leaderboard
17 Gemma 3 27B 54.5% Mar 2025 Official leaderboard
18 Llama 3.2 1B 54.5% Sep 2024 Official leaderboard
19 Gemma 3 12B 52.2% Mar 2025 Official leaderboard
20 Olmo 3.1 32B Instruct 51.2% Dec 2025 Official leaderboard
21 LFM2-1.2B 51.1% Jul 2025 Official leaderboard
22 Gemma 3n E2B 47.6% Jun 2025 Official leaderboard
23 OLMo 2 32B Instruct 46.7% Mar 2025 Official leaderboard
24 Trinity Mini 46.6% Dec 2025 Official leaderboard
25 Qwen3-0.6B 46.1% Apr 2025 Official leaderboard
26 Gemma 3n E4B 46% Jun 2025 Official leaderboard
27 AFM-4.5B 44.3% Jul 2025 Official leaderboard
28 Apertus 8B Instruct 44.3% Sep 2025 Official leaderboard
29 Ministral 3 3B Reasoning 43.9% Dec 2025 Official leaderboard
30 LFM2.5-1.2B-Thinking 43.6% Jan 2026 Official leaderboard
31 Gemma 3 1B 41.7% Mar 2025 Official leaderboard
32 LFM2.5-1.2B-Instruct 34.8% Jan 2026 Official leaderboard
33 SmolLM2-360M 29.4% Oct 2024 Official leaderboard
34 Gemma 3 270M 17.6% Aug 2025 Official leaderboard
35 SmolLM2-135M 8.87% Oct 2024 Official leaderboard

2 models have held number one on the official leaderboard since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Meta Swiss AI Initiative
Own leaderboard Not tested since Jan 2026
Apertus 70B 69.5%

Source: Official GuiaCat leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Apertus 70B Llama 3.1-8B 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Apertus 70B Llama 3.1-8B

Source: Official GuiaCat leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Multilingual

All 545 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%

Data audit

9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 10 months ago, in January 2026, for LFM2.5-1.2B-Thinking. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (320 entries).
  • Pass Models scored 35 models we track, plus 285 other entries on its leaderboard.
  • Warning Newest score The newest came 10 months ago, in January 2026, for LFM2.5-1.2B-Thinking.
  • Note Frontier models None of the top 10 has a score. The strongest tested is gpt-oss-20b (#143).
  • Pass Headroom The best is 69.5%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on euroeval.com: 39 rows for models we track and 285 other entries at the last read.
  • Pass Last read November 21st, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check