Benchmark Almanac

GermanQuAD

allenai/Olmo-3-1125-32B 72.4% tops its leaderboard Gemma 2 27B 70% leads the models we track
Not in model cards

Models extract answers to German questions from manually annotated Wikipedia passages.

EuroEval reports character-level F1 and exact match.

Made by
Timo Möller and colleagues
Released
April 2021
Kind
Reading in many languages
Level
School
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Measures
Capability
Language
German
In and out
Text → Text
Interaction
Single turn
Items from
Real work and use
Refresh
One fixed set
Run by
Anyone
Strongest model tested
Qwen3-235B-A22B #131 on the capability index
Last model added
March 2026
Scores refreshed
November 20th, 2026
271
Entries on EuroEval’s leaderboard, 88 models we track

allenai/Olmo-3-1125-32B tops the leaderboard with 72.4% Gemma 2 27B leads with 70% among models released by March 2026

Best score per entry from EuroEval’s leaderboard, read on November 20th, 2026. EuroEval reports character-level F1 on the GermanQuAD-mini test split, scaled by 100. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Gemma 2 27B, at 70%. This view shows only models we track, each at its best.

The newest model with a score here came out in March 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

271 entries. Scroll sideways for the rest.

  1. 72.4 allenai/Olmo-3-1125-32B
  2. 70.0 Gemma 2 27B
  3. 69.4 Llama 3.1-405B
  4. 69.2 Llama 2-70B
  5. 69.0 mistralai/Mistral-Small-3.1-24B-Base-2503
  6. 68.4 allenai/OLMo-2-0325-32B
  7. 68.4 Gemma 3 12B PT
  8. 68.4 Gemma 3n E4B
  9. 68.3 allenai/OLMo-2-1124-13B
  10. 68.2 Yi-1.5-34B
  11. 68.1 Llama 3.1-70B
  12. 68.1 utter-project/EuroLLM-22B-Preview
  13. 68.0 allenai/Olmo-3-1025-7B
  14. 68.0 upstage/SOLAR-10.7B-v1.0
  15. 68.0 Gemma 3 27B PT
  16. 68.0 utter-project/EuroLLM-22B-2512
  17. 67.6 Apertus 70B
  18. 67.5 speakleash/Bielik-11B-v3-Base-20250730
  19. 67.0 mistralai/Ministral-3-3B-Base-2512
  20. 66.8 01-ai/Yi-1.5-9B
  21. 66.8 allenai/OLMo-2-1124-7B
  22. 66.8 tiiuae/Falcon3-10B-Base
  23. 66.5 Llama 3.1-8B
  24. 66.3 speakleash/Bielik-11B-v2.3-Instruct
  25. 65.8 Llama 3-8B
  26. 65.6 swiss-ai/Apertus-8B-2509
  27. 65.5 ibm-granite/granite-34b-code-base
  28. 65.5 mistralai/Magistral-Small-2509
  29. 65.4 Mistral Small 3.1
  30. 65.2 mistralai/Ministral-3-8B-Reasoning-2512
  31. 65.2 Gemma 2 9B
  32. 64.9 allenai/Olmo-3.1-32B-Instruct-SFT
  33. 64.8 Mistral Small 3.2
  34. 64.8 Nemotron 3 Nano 30B-A3B
  35. 64.6 Falcon3-10B-Instruct
  36. 64.6 ai21labs/Jamba-v0.1
  37. 64.5 TildeAI/TildeOpen-30b
  38. 64.5 bullerwins/Magistral-Small-2506-fp8
  39. 64.4 CohereForAI/aya-23-35B
  40. 64.3 Gemma 7B
  41. 64.2 BSC-LT/ALIA-40b
  42. 64.0 Gemma 3 4B PT
  43. 64.0 syvai/llama3-da-base
  44. 63.9 mistralai/Voxtral-Mini-3B-2507
  45. 63.8 meta-llama/Llama-2-13b-hf
  46. 63.7 utter-project/EuroLLM-9B
  47. 63.5 allenai/Llama-3.1-Tulu-3-8B-SFT
  48. 63.2 Qwen3-32B
  49. 63.2 Command R7B
  50. 63.0 allenai/Olmo-3.1-32B-Instruct-DPO
  51. 62.7 Falcon3-7B-Instruct
  52. 62.4 Mistral 7B v0.3
  53. 62.4 Granite-3.0-8B-Instruct
  54. 62.0 allenai/Llama-3.1-Tulu-3-8B-DPO
  55. 61.9 Granite-3.0-8B-Base
  56. 61.5 HuggingFaceTB/SmolLM3-3B-Base
  57. 61.5 Qwen3-14B
  58. 61.3 Voxtral Small
  59. 61.2 mii-llm/nesso-4B
  60. 61.1 deepcogito/cogito-v1-preview-qwen-32B
  61. 60.9 ibm-granite/granite-20b-code-base
  62. 60.9 Mistral 7B v0.1
  63. 60.7 Olmo 3.1 32B Instruct
  64. 60.6 senseable/WestLake-7B-v2
  65. 60.4 NovaSky-AI/Sky-T1-32B-Preview
  66. 60.1 occiglot/occiglot-7b-eu5
  67. 60.1 tiiuae/Falcon3-7B-Base
  68. 59.8 NousResearch/DeepHermes-3-Llama-3-8B-Preview
  69. 59.5 ibm-granite/granite-34b-code-instruct
  70. 59.4 LumiOpen/Llama-Poro-2-70B-SFT
  71. 59.2 baidu/ERNIE-4.5-21B-A3B-PT
  72. 59.0 mistralai/Mistral-Small-24B-Instruct-2501
  73. 58.9 Ministral 3 3B Reasoning
  74. 58.9 Phi-4-reasoning
  75. 58.6 norallm/normistral-11b-warm
  76. 58.5 Apertus 8B Instruct
  77. 58.3 MaLA-LM/emma-500-llama2-7b
  78. 58.2 utter-project/EuroLLM-9B-Instruct
  79. 58.1 kyutai/helium-1-2b
  80. 58.0 Qwen1.5-14B-Chat
  81. 57.7 Starling-LM-7B-beta
  82. 57.6 allenai/OLMo-2-1124-7B-Instruct
  83. 57.6 utter-project/EuroLLM-22B-Instruct-Preview
  84. 57.6 Mistral-7B-Instruct-v0.1
  85. 57.5 meta-llama/Llama-3.1-405B-Instruct-FP8
  86. 57.5 TrustLLMeu/baseline-7-8b_2-3t-tokens_llama
  87. 57.3 Llama-3.1-Tulu-3-8B
  88. 57.3 Qwen3-30B-A3B
  89. 57.2 arcee-ai/AFM-4.5B-Base
  90. 57.1 ibm-granite/granite-8b-code-base-4k
  91. 57.1 google/gemma-2-9b-it
  92. 56.6 Gemma 2 2B IT
  93. 56.6 ibm-granite/granite-3.0-3b-a800m-instruct
  94. 56.2 Cogito v1 Preview 70B
  95. 56.1 HuggingFaceTB/SmolLM3-3B
  96. 55.8 Aya Expanse 32B
  97. 55.7 google/gemma-2-27b-it
  98. 55.6 deepcogito/cogito-v1-preview-qwen-14B
  99. 55.5 Qwen3-235B-A22B
  100. 55.5 LumiOpen/Llama-Poro-2-70B-Instruct
  101. 55.4 152334H/miqu-1-70b-sf
  102. 55.4 Qwen3-8B
  103. 55.4 qwen3-4b-instruct-2507
  104. 55.3 Granite 4.0 H Small
  105. 55.1 Qwen/Qwen3-235B-A22B-Instruct-2507-FP8
  106. 55.0 deepcogito/cogito-v2-preview-llama-70B
  107. 54.9 ThatsGroes/gemma-2-27b-it-FP8-Dynamic
  108. 54.9 Qwen3-Next-80B-A3B-Instruct
  109. 54.8 Gemma 3 12B
  110. 54.7 allenai/Llama-3.1-Tulu-3.1-8B
  111. 54.5 ibm-granite/granite-4.0-h-micro
  112. 54.4 TrustLLMeu/baseline-7-8b_1t-tokens_llama
  113. 54.3 granite-4.0-micro
  114. 54.0 Gemma 3n E2B
  115. 54.0 seedboxai/KafkaLM-70B-German-V0.1
  116. 53.9 ibm-granite/granite-8b-code-instruct-4k
  117. 53.8 Reka Flash 3
  118. 53.8 Qwen/Qwen3-235B-A22B-Thinking-2507-FP8
  119. 53.6 deepseek-r1-0528-qwen3-8b
  120. 53.6 allenai/OLMo-2-1124-13B-Instruct
  121. 53.5 Qwen3-1.7B
  122. 53.5 Gemma 3 27B
  123. 53.4 Qwen3-4B
  124. 53.4 Qwen3-4B-Thinking-2507
  125. 53.3 google/gemma-2-2b
  126. 53.1 infly/OpenCoder-8B-Instruct
  127. 53.0 ollama_chat/gemma3:27b-it-fp16
  128. 53.0 common-pile/comma-v0.1-2t
  129. 52.7 Qwen/Qwen2.5-1.5B-Instruct
  130. 52.4 meta-llama/Llama-3.1-8B-Instruct
  131. 52.3 ibm-granite/granite-20b-code-instruct
  132. 51.8 meta-llama/Llama-2-13b-chat-hf
  133. 51.5 meta-llama/Llama-2-70b-chat-hf
  134. 51.5 Llama 4 Scout
  135. 50.8 LumiOpen/Viking-33B
  136. 50.8 nvidia-nemotron-nano-9b-v2
  137. 50.7 Falcon3-3B-Instruct
  138. 50.3 LiquidAI/LFM2-2.6B
  139. 50.0 TrustLLMeu/trustllm-v2-8b-midtrain@3c7af30bc43bb3c8607e0fab6e7f2a23dd64af49
  140. 49.9 ibm-granite/granite-3.0-3b-a800m-base
  141. 49.8 FacebookAI/xlm-roberta-large
  142. 49.6 gpt-oss-20b
  143. 49.0 ollama_chat/hf.co/google/gemma-3-4b-it-qat-q4_0-gguf
  144. 48.6 speakleash/Bielik-4.5B-v3.0-Instruct
  145. 48.6 allenai/Olmo-3-7B-Instruct
  146. 48.6 allenai/Olmo-3-7B-Think
  147. 48.5 DeepSeek-R1-Distill-Llama-70B
  148. 48.3 BSC-LT/salamandra-7b
  149. 48.2 allenai/Olmo-3-7B-Instruct-SFT
  150. 48.2 openGPT-X/Teuken-7B-instruct-commercial-v0.4
  151. 47.9 Mistral-7B-Instruct-v0.2
  152. 47.7 intfloat/multilingual-e5-large
  153. 47.6 Qwen3-Next-80B-A3B
  154. 47.4 DeepSeek-R1-Distill-Qwen-32B
  155. 46.8 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview
  156. 46.5 deepcogito/cogito-v1-preview-llama-8B
  157. 46.5 arcee-ai/Trinity-Nano-Preview
  158. 45.9 allenai/Olmo-3-7B-Instruct-DPO
  159. 45.9 QwQ-32B
  160. 45.8 ollama_chat/gemma3:4b-it-q4_K_M
  161. 45.4 meta-llama/Llama-2-7b-chat-hf
  162. 45.3 arcee-ai/Trinity-Nano-Base
  163. 45.3 meta-llama/Llama-3.2-3B-Instruct
  164. 45.2 speakleash/Bielik-1.5B-v3.0-Instruct
  165. 44.8 common-pile/comma-v0.1-1t
  166. 44.7 Reka Flash 3.1
  167. 44.0 Qwen/Qwen2.5-Math-7B
  168. 43.9 jhu-clsp/mmBERT-base
  169. 43.4 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f
  170. 43.4 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6
  171. 43.3 SmolLM2 1.7B Instruct
  172. 43.2 Gemma 3 1B PT
  173. 43.1 seedboxai/KafkaLM-13B-German-V0.1
  174. 43.0 occiglot/occiglot-7b-eu5-instruct
  175. 42.7 facebook/MobileLLM-R1-950M-base
  176. 42.3 tiiuae/Falcon3-3B-Base
  177. 41.8 TrustLLMeu/trustllm-v2-8b@d8cc924fa550fbf9474f5251e8e2f92d1c1bf894
  178. 41.4 utter-project/EuroMoE-2.6B-A0.6B-Preview
  179. 41.3 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660
  180. 41.2 yulan-team/YuLan-Mini-Instruct
  181. 41.1 Gemma 3 4B
  182. 40.6 EuroBERT/EuroBERT-610m
  183. 40.5 Qwen3-0.6B
  184. 39.0 SmolLM2-1.7B
  185. 38.5 Phi-4
  186. 36.9 utter-project/EuroLLM-22B-Instruct-2512
  187. 36.8 Gemma 3 1B
  188. 36.2 DeepSeek-R1-Distill-Qwen-7B
  189. 36.2 meta-llama/Llama-4-Scout-17B-16E
  190. 36.0 PleIAs/Pleias-3b-Preview
  191. 35.4 tencent/Hunyuan-7B-Instruct
  192. 35.2 tiiuae/falcon-11B
  193. 33.6 AI-Sweden-Models/ModernBERT-large
  194. 33.4 infly/OpenCoder-8B-Base
  195. 33.2 PleIAs/Pleias-1.2b-Preview
  196. 32.4 DeepSeek-R1-Distill-Qwen-14B
  197. 31.6 ibm-granite/granite-4.0-h-tiny
  198. 31.3 OpenLLM-France/Lucie-7B
  199. 31.1 ibm-granite/granite-3b-code-base-2k
  200. 30.2 GLM-4.5-Air
  201. 30.1 LFM2.5-1.2B-Instruct
  202. 29.7 timpal0l/Mistral-7B-v0.1-flashback-v2
  203. 29.1 speakleash/Bielik-11B-v2.6-Instruct
  204. 28.8 google/gemma-7b-it
  205. 28.4 tencent/Hunyuan-1.8B-Instruct
  206. 27.4 jhu-clsp/mmBERT-small
  207. 26.6 infly/OpenCoder-1.5B-Instruct
  208. 26.4 Llama 3.2 1B
  209. 26.2 deepseek-ai/DeepSeek-R1-Distill-Llama-8B
  210. 25.7 AFM-4.5B
  211. 25.0 LFM2.5-1.2B-Thinking
  212. 23.9 ibm-granite/granite-3b-code-instruct-2k
  213. 23.5 LiquidAI/LFM2.5-1.2B-Base
  214. 23.4 HPLT/hplt2c_deu_checkpoints
  215. 22.4 intfloat/multilingual-e5-base
  216. 21.9 LiquidAI/LFM2-700M
  217. 21.4 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct
  218. 21.2 deepcogito/cogito-v1-preview-llama-3B
  219. 20.9 Qwen1.5-14B
  220. 20.8 utter-project/EuroVLM-1.7B-Preview
  221. 20.6 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base
  222. 20.2 SmolLM2-360M
  223. 20.2 Gemma 3 270M
  224. 20.1 BSC-LT/salamandra-7b-instruct
  225. 19.7 LiquidAI/LFM2-350M
  226. 18.9 LFM2-1.2B
  227. 18.6 DeepSeek-R1-Distill-Qwen-1.5B
  228. 17.8 Phi-2
  229. 17.8 PleIAs/Pleias-350m-Preview
  230. 17.7 OLMo 2 32B Instruct
  231. 16.8 facebook/MobileLLM-R1-360M-base
  232. 16.8 HPLT/hplt2c_eng_checkpoints
  233. 15.9 baidu/ERNIE-4.5-0.3B-PT
  234. 14.9 arcee-ai/Trinity-Mini-Base
  235. 14.0 EuroBERT/EuroBERT-210m
  236. 14.0 answerdotai/ModernBERT-large
  237. 13.7 GLM-4.1V-9B-Thinking
  238. 13.5 Qwen/Qwen2.5-Math-1.5B
  239. 12.5 AI-Sweden-Models/bert-large-nordic-pile-1M-steps
  240. 12.3 Phi-4-mini-reasoning
  241. 12.1 PleIAs/Pleias-Pico
  242. 12.0 answerdotai/ModernBERT-base
  243. 11.5 meta-llama/Llama-3.2-1B-Instruct
  244. 10.8 HuggingFaceTB/SmolLM2-360M-Instruct
  245. 9.52 infly/OpenCoder-1.5B-Base
  246. 8.85 HPLT/hplt2c_nld_checkpoints
  247. 6.77 facebook/MobileLLM-R1-140M-base
  248. 6.55 PleIAs/Baguettotron@refs/pr/6
  249. 6.53 Nemotron 3 Nano 4B
  250. 5.87 EuropeanParliament/EUBERT
  251. 4.89 SmolLM2-135M
  252. 4.80 HuggingFaceTB/SmolLM2-135M-Instruct
  253. 4.55 AI-Sweden-Models/gpt-sw3-356m-instruct
  254. 4.21 AI-Sweden-Models/gpt-sw3-126m-instruct
  255. 4.20 AI-Sweden-Models/gpt-sw3-356m
  256. 3.96 Shekswess/trlm-135m
  257. 3.92 DiscoResearch/DiscoLM-70b
  258. 3.57 alea-institute/kl3m-003-3.7b
  259. 3.42 PleIAs/Monad
  260. 3.29 microsoft/mdeberta-v3-base
  261. 2.69 facebook/MobileLLM-R1-950M
  262. 2.21 AI-Sweden-Models/gpt-sw3-126m
  263. 0.77 distilbert/distilbert-base-multilingual-cased
  264. 0.72 Geotrend/distilbert-base-25lang-cased
  265. 0.16 facebook/MobileLLM-R1-360M
  266. 0.02 facebook/MobileLLM-R1-140M
  267. 0.0 fresh-electra-small
  268. 0.0 fresh-xlm-roberta-base
  269. 0.0 openai-community/gpt2
  270. 0.0 openai-community/gpt2-large
  271. 0.0 openai-community/gpt2-medium
  1. 1 allenai/Olmo-3-1125-32B 72.4%
  2. 2 Gemma 2 27B 70%
  3. 3 Llama 3.1-405B 69.4%
  4. 4 Llama 2-70B 69.2%
  5. 5 mistralai/Mistral-Small-3.1-24B-Base-2503 69%
  6. 6 allenai/OLMo-2-0325-32B 68.4%
  7. 7 Gemma 3 12B PT 68.4%
  8. 8 Gemma 3n E4B 68.4%
  9. 9 allenai/OLMo-2-1124-13B 68.3%
  10. 10 Yi-1.5-34B 68.2%
  11. 11 Llama 3.1-70B 68.1%
  12. 12 utter-project/EuroLLM-22B-Preview 68.1%
  13. 13 allenai/Olmo-3-1025-7B 68%
  14. 14 upstage/SOLAR-10.7B-v1.0 68%
  15. 15 Gemma 3 27B PT 68%
  16. 16 utter-project/EuroLLM-22B-2512 68%
  17. 17 Apertus 70B 67.6%
  18. 18 speakleash/Bielik-11B-v3-Base-20250730 67.5%
  19. 19 mistralai/Ministral-3-3B-Base-2512 67%
  20. 20 01-ai/Yi-1.5-9B 66.8%
  21. 21 allenai/OLMo-2-1124-7B 66.8%
  22. 22 tiiuae/Falcon3-10B-Base 66.8%
  23. 23 Llama 3.1-8B 66.5%
  24. 24 speakleash/Bielik-11B-v2.3-Instruct 66.3%
  25. 25 Llama 3-8B 65.8%
  26. 26 swiss-ai/Apertus-8B-2509 65.6%
  27. 27 ibm-granite/granite-34b-code-base 65.5%
  28. 28 mistralai/Magistral-Small-2509 65.5%
  29. 29 Mistral Small 3.1 65.4%
  30. 30 mistralai/Ministral-3-8B-Reasoning-2512 65.2%
  31. 31 Gemma 2 9B 65.2%
  32. 32 allenai/Olmo-3.1-32B-Instruct-SFT 64.9%
  33. 33 Mistral Small 3.2 64.8%
  34. 34 Nemotron 3 Nano 30B-A3B no-thinking 64.8%
  35. 35 Falcon3-10B-Instruct 64.6%
  36. 36 ai21labs/Jamba-v0.1 64.6%
  37. 37 TildeAI/TildeOpen-30b slow-tokenizer 64.5%
  38. 38 bullerwins/Magistral-Small-2506-fp8 64.5%
  39. 39 CohereForAI/aya-23-35B 64.4%
  40. 40 Gemma 7B 64.3%
  41. 41 BSC-LT/ALIA-40b 64.2%
  42. 42 Gemma 3 4B PT 64%
  43. 43 syvai/llama3-da-base 64%
  44. 44 mistralai/Voxtral-Mini-3B-2507 63.9%
  45. 45 meta-llama/Llama-2-13b-hf 63.8%
  46. 46 utter-project/EuroLLM-9B 63.7%
  47. 47 allenai/Llama-3.1-Tulu-3-8B-SFT 63.5%
  48. 48 Qwen3-32B no-thinking 63.2%
  49. 49 Command R7B 63.2%
  50. 50 allenai/Olmo-3.1-32B-Instruct-DPO 63%
  51. 51 Falcon3-7B-Instruct 62.7%
  52. 52 Mistral 7B v0.3 62.4%
  53. 53 Granite-3.0-8B-Instruct 62.4%
  54. 54 allenai/Llama-3.1-Tulu-3-8B-DPO 62%
  55. 55 Granite-3.0-8B-Base 61.9%
  56. 56 HuggingFaceTB/SmolLM3-3B-Base 61.5%
  57. 57 Qwen3-14B no-thinking 61.5%
  58. 58 Voxtral Small 61.3%
  59. 59 mii-llm/nesso-4B 61.2%
  60. 60 deepcogito/cogito-v1-preview-qwen-32B 61.1%
  61. 61 ibm-granite/granite-20b-code-base 60.9%
  62. 62 Mistral 7B v0.1 60.9%
  63. 63 Olmo 3.1 32B Instruct 60.7%
  64. 64 senseable/WestLake-7B-v2 60.6%
  65. 65 NovaSky-AI/Sky-T1-32B-Preview 60.4%
  66. 66 occiglot/occiglot-7b-eu5 60.1%
  67. 67 tiiuae/Falcon3-7B-Base 60.1%
  68. 68 NousResearch/DeepHermes-3-Llama-3-8B-Preview 59.8%
  69. 69 ibm-granite/granite-34b-code-instruct 59.5%
  70. 70 LumiOpen/Llama-Poro-2-70B-SFT 59.4%
  71. 71 baidu/ERNIE-4.5-21B-A3B-PT 59.2%
  72. 72 mistralai/Mistral-Small-24B-Instruct-2501 59%
  73. 73 Ministral 3 3B Reasoning 58.9%
  74. 74 Phi-4-reasoning 58.9%
  75. 75 norallm/normistral-11b-warm 58.6%
  76. 76 Apertus 8B Instruct 58.5%
  77. 77 MaLA-LM/emma-500-llama2-7b 58.3%
  78. 78 utter-project/EuroLLM-9B-Instruct 58.2%
  79. 79 kyutai/helium-1-2b 58.1%
  80. 80 Qwen1.5-14B-Chat 58%
  81. 81 Starling-LM-7B-beta 57.7%
  82. 82 allenai/OLMo-2-1124-7B-Instruct 57.6%
  83. 83 utter-project/EuroLLM-22B-Instruct-Preview 57.6%
  84. 84 Mistral-7B-Instruct-v0.1 57.6%
  85. 85 meta-llama/Llama-3.1-405B-Instruct-FP8 57.5%
  86. 86 TrustLLMeu/baseline-7-8b_2-3t-tokens_llama 57.5%
  87. 87 Llama-3.1-Tulu-3-8B 57.3%
  88. 88 Qwen3-30B-A3B no-thinking 57.3%
  89. 89 arcee-ai/AFM-4.5B-Base 57.2%
  90. 90 ibm-granite/granite-8b-code-base-4k 57.1%
  91. 91 google/gemma-2-9b-it 57.1%
  92. 92 Gemma 2 2B IT 56.6%
  93. 93 ibm-granite/granite-3.0-3b-a800m-instruct 56.6%
  94. 94 Cogito v1 Preview 70B 56.2%
  95. 95 HuggingFaceTB/SmolLM3-3B 56.1%
  96. 96 Aya Expanse 32B 55.8%
  97. 97 google/gemma-2-27b-it 55.7%
  98. 98 deepcogito/cogito-v1-preview-qwen-14B 55.6%
  99. 99 Qwen3-235B-A22B 55.5%
  100. 100 LumiOpen/Llama-Poro-2-70B-Instruct 55.5%
  101. 101 152334H/miqu-1-70b-sf 55.4%
  102. 102 Qwen3-8B no-thinking 55.4%
  103. 103 qwen3-4b-instruct-2507 55.4%
  104. 104 Granite 4.0 H Small 55.3%
  105. 105 Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 55.1%
  106. 106 deepcogito/cogito-v2-preview-llama-70B 55%
  107. 107 ThatsGroes/gemma-2-27b-it-FP8-Dynamic 54.9%
  108. 108 Qwen3-Next-80B-A3B-Instruct 54.9%
  109. 109 Gemma 3 12B 54.8%
  110. 110 allenai/Llama-3.1-Tulu-3.1-8B 54.7%
  111. 111 ibm-granite/granite-4.0-h-micro 54.5%
  112. 112 TrustLLMeu/baseline-7-8b_1t-tokens_llama 54.4%
  113. 113 granite-4.0-micro 54.3%
  114. 114 Gemma 3n E2B 54%
  115. 115 seedboxai/KafkaLM-70B-German-V0.1 54%
  116. 116 ibm-granite/granite-8b-code-instruct-4k 53.9%
  117. 117 Reka Flash 3 53.8%
  118. 118 Qwen/Qwen3-235B-A22B-Thinking-2507-FP8 53.8%
  119. 119 deepseek-r1-0528-qwen3-8b 53.6%
  120. 120 allenai/OLMo-2-1124-13B-Instruct 53.6%
  121. 121 Qwen3-1.7B 53.5%
  122. 122 Gemma 3 27B 53.5%
  123. 123 Qwen3-4B no-thinking 53.4%
  124. 124 Qwen3-4B-Thinking-2507 53.4%
  125. 125 google/gemma-2-2b 53.3%
  126. 126 infly/OpenCoder-8B-Instruct 53.1%
  127. 127 ollama_chat/gemma3:27b-it-fp16 53%
  128. 128 common-pile/comma-v0.1-2t 53%
  129. 129 Qwen/Qwen2.5-1.5B-Instruct 52.7%
  130. 130 meta-llama/Llama-3.1-8B-Instruct 52.4%
  131. 131 ibm-granite/granite-20b-code-instruct 52.3%
  132. 132 meta-llama/Llama-2-13b-chat-hf 51.8%
  133. 133 meta-llama/Llama-2-70b-chat-hf 51.5%
  134. 134 Llama 4 Scout 51.5%
  135. 135 LumiOpen/Viking-33B 50.8%
  136. 136 nvidia-nemotron-nano-9b-v2 50.8%
  137. 137 Falcon3-3B-Instruct 50.7%
  138. 138 LiquidAI/LFM2-2.6B 50.3%
  139. 139 TrustLLMeu/trustllm-v2-8b-midtrain@3c7af30bc43bb3c8607e0fab6e7f2a23dd64af49 50%
  140. 140 ibm-granite/granite-3.0-3b-a800m-base 49.9%
  141. 141 FacebookAI/xlm-roberta-large 49.8%
  142. 142 gpt-oss-20b medium 49.6%
  143. 143 ollama_chat/hf.co/google/gemma-3-4b-it-qat-q4_0-gguf 49%
  144. 144 speakleash/Bielik-4.5B-v3.0-Instruct 48.6%
  145. 145 allenai/Olmo-3-7B-Instruct 48.6%
  146. 146 allenai/Olmo-3-7B-Think 48.6%
  147. 147 DeepSeek-R1-Distill-Llama-70B 48.5%
  148. 148 BSC-LT/salamandra-7b 48.3%
  149. 149 allenai/Olmo-3-7B-Instruct-SFT 48.2%
  150. 150 openGPT-X/Teuken-7B-instruct-commercial-v0.4 48.2%
  151. 151 Mistral-7B-Instruct-v0.2 47.9%
  152. 152 intfloat/multilingual-e5-large 47.7%
  153. 153 Qwen3-Next-80B-A3B thinking 47.6%
  154. 154 DeepSeek-R1-Distill-Qwen-32B 47.4%
  155. 155 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview 46.8%
  156. 156 deepcogito/cogito-v1-preview-llama-8B 46.5%
  157. 157 arcee-ai/Trinity-Nano-Preview 46.5%
  158. 158 allenai/Olmo-3-7B-Instruct-DPO 45.9%
  159. 159 QwQ-32B 45.9%
  160. 160 ollama_chat/gemma3:4b-it-q4_K_M 45.8%
  161. 161 meta-llama/Llama-2-7b-chat-hf 45.4%
  162. 162 arcee-ai/Trinity-Nano-Base 45.3%
  163. 163 meta-llama/Llama-3.2-3B-Instruct 45.3%
  164. 164 speakleash/Bielik-1.5B-v3.0-Instruct 45.2%
  165. 165 common-pile/comma-v0.1-1t 44.8%
  166. 166 Reka Flash 3.1 44.7%
  167. 167 Qwen/Qwen2.5-Math-7B 44%
  168. 168 jhu-clsp/mmBERT-base 43.9%
  169. 169 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f zero-shot 43.4%
  170. 170 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 43.4%
  171. 171 SmolLM2 1.7B Instruct 43.3%
  172. 172 Gemma 3 1B PT 43.2%
  173. 173 seedboxai/KafkaLM-13B-German-V0.1 43.1%
  174. 174 occiglot/occiglot-7b-eu5-instruct 43%
  175. 175 facebook/MobileLLM-R1-950M-base 42.7%
  176. 176 tiiuae/Falcon3-3B-Base 42.3%
  177. 177 TrustLLMeu/trustllm-v2-8b@d8cc924fa550fbf9474f5251e8e2f92d1c1bf894 41.8%
  178. 178 utter-project/EuroMoE-2.6B-A0.6B-Preview 41.4%
  179. 179 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 41.3%
  180. 180 yulan-team/YuLan-Mini-Instruct 41.2%
  181. 181 Gemma 3 4B 41.1%
  182. 182 EuroBERT/EuroBERT-610m 40.6%
  183. 183 Qwen3-0.6B no-thinking 40.5%
  184. 184 SmolLM2-1.7B 39%
  185. 185 Phi-4 38.5%
  186. 186 utter-project/EuroLLM-22B-Instruct-2512 36.9%
  187. 187 Gemma 3 1B 36.8%
  188. 188 DeepSeek-R1-Distill-Qwen-7B 36.2%
  189. 189 meta-llama/Llama-4-Scout-17B-16E 36.2%
  190. 190 PleIAs/Pleias-3b-Preview 36%
  191. 191 tencent/Hunyuan-7B-Instruct 35.4%
  192. 192 tiiuae/falcon-11B 35.2%
  193. 193 AI-Sweden-Models/ModernBERT-large 33.6%
  194. 194 infly/OpenCoder-8B-Base 33.4%
  195. 195 PleIAs/Pleias-1.2b-Preview 33.2%
  196. 196 DeepSeek-R1-Distill-Qwen-14B 32.4%
  197. 197 ibm-granite/granite-4.0-h-tiny 31.6%
  198. 198 OpenLLM-France/Lucie-7B 31.3%
  199. 199 ibm-granite/granite-3b-code-base-2k 31.1%
  200. 200 GLM-4.5-Air no-thinking 30.2%
  201. 201 LFM2.5-1.2B-Instruct 30.1%
  202. 202 timpal0l/Mistral-7B-v0.1-flashback-v2 29.7%
  203. 203 speakleash/Bielik-11B-v2.6-Instruct 29.1%
  204. 204 google/gemma-7b-it 28.8%
  205. 205 tencent/Hunyuan-1.8B-Instruct 28.4%
  206. 206 jhu-clsp/mmBERT-small 27.4%
  207. 207 infly/OpenCoder-1.5B-Instruct 26.6%
  208. 208 Llama 3.2 1B 26.4%
  209. 209 deepseek-ai/DeepSeek-R1-Distill-Llama-8B 26.2%
  210. 210 AFM-4.5B 25.7%
  211. 211 LFM2.5-1.2B-Thinking 25%
  212. 212 ibm-granite/granite-3b-code-instruct-2k 23.9%
  213. 213 LiquidAI/LFM2.5-1.2B-Base 23.5%
  214. 214 HPLT/hplt2c_deu_checkpoints 23.4%
  215. 215 intfloat/multilingual-e5-base 22.4%
  216. 216 LiquidAI/LFM2-700M 21.9%
  217. 217 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct 21.4%
  218. 218 deepcogito/cogito-v1-preview-llama-3B 21.2%
  219. 219 Qwen1.5-14B 20.9%
  220. 220 utter-project/EuroVLM-1.7B-Preview 20.8%
  221. 221 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base 20.6%
  222. 222 SmolLM2-360M 20.2%
  223. 223 Gemma 3 270M 20.2%
  224. 224 BSC-LT/salamandra-7b-instruct 20.1%
  225. 225 LiquidAI/LFM2-350M 19.7%
  226. 226 LFM2-1.2B 18.9%
  227. 227 DeepSeek-R1-Distill-Qwen-1.5B 18.6%
  228. 228 Phi-2 17.8%
  229. 229 PleIAs/Pleias-350m-Preview 17.8%
  230. 230 OLMo 2 32B Instruct 17.7%
  231. 231 facebook/MobileLLM-R1-360M-base 16.8%
  232. 232 HPLT/hplt2c_eng_checkpoints 16.8%
  233. 233 baidu/ERNIE-4.5-0.3B-PT 15.9%
  234. 234 arcee-ai/Trinity-Mini-Base 14.9%
  235. 235 EuroBERT/EuroBERT-210m 14%
  236. 236 answerdotai/ModernBERT-large 14%
  237. 237 GLM-4.1V-9B-Thinking 13.7%
  238. 238 Qwen/Qwen2.5-Math-1.5B 13.5%
  239. 239 AI-Sweden-Models/bert-large-nordic-pile-1M-steps 12.5%
  240. 240 Phi-4-mini-reasoning 12.3%
  241. 241 PleIAs/Pleias-Pico 12.1%
  242. 242 answerdotai/ModernBERT-base 12%
  243. 243 meta-llama/Llama-3.2-1B-Instruct 11.5%
  244. 244 HuggingFaceTB/SmolLM2-360M-Instruct 10.8%
  245. 245 infly/OpenCoder-1.5B-Base 9.52%
  246. 246 HPLT/hplt2c_nld_checkpoints 8.85%
  247. 247 facebook/MobileLLM-R1-140M-base 6.77%
  248. 248 PleIAs/Baguettotron@refs/pr/6 no-thinking 6.55%
  249. 249 Nemotron 3 Nano 4B 6.53%
  250. 250 EuropeanParliament/EUBERT 5.87%
  251. 251 SmolLM2-135M 4.89%
  252. 252 HuggingFaceTB/SmolLM2-135M-Instruct 4.80%
  253. 253 AI-Sweden-Models/gpt-sw3-356m-instruct 4.55%
  254. 254 AI-Sweden-Models/gpt-sw3-126m-instruct 4.21%
  255. 255 AI-Sweden-Models/gpt-sw3-356m 4.20%
  256. 256 Shekswess/trlm-135m 3.96%
  257. 257 DiscoResearch/DiscoLM-70b 3.92%
  258. 258 alea-institute/kl3m-003-3.7b 3.57%
  259. 259 PleIAs/Monad 3.42%
  260. 260 microsoft/mdeberta-v3-base 3.29%
  261. 261 facebook/MobileLLM-R1-950M 2.69%
  262. 262 AI-Sweden-Models/gpt-sw3-126m 2.21%
  263. 263 distilbert/distilbert-base-multilingual-cased 0.77%
  264. 264 Geotrend/distilbert-base-25lang-cased 0.72%
  265. 265 facebook/MobileLLM-R1-360M 0.16%
  266. 266 facebook/MobileLLM-R1-140M 0.02%
  267. 267 fresh-electra-small 0%
  268. 268 fresh-xlm-roberta-base 0%
  269. 269 openai-community/gpt2 0%
  270. 270 openai-community/gpt2-large 0%
  271. 271 openai-community/gpt2-medium 0%

Source: EuroEval’s leaderboard

88 models. Scroll sideways for the rest.

  1. 70.0 Gemma 2 27B
  2. 69.4 Llama 3.1-405B
  3. 69.2 Llama 2-70B
  4. 68.4 Gemma 3 12B PT
  5. 68.4 Gemma 3n E4B
  6. 68.2 Yi-1.5-34B
  7. 68.1 Llama 3.1-70B
  8. 68.0 Gemma 3 27B PT
  9. 67.6 Apertus 70B
  10. 66.5 Llama 3.1-8B
  11. 65.8 Llama 3-8B
  12. 65.4 Mistral Small 3.1
  13. 65.2 Gemma 2 9B
  14. 64.8 Mistral Small 3.2
  15. 64.8 Nemotron 3 Nano 30B-A3B
  16. 64.6 Falcon3-10B-Instruct
  17. 64.3 Gemma 7B
  18. 64.0 Gemma 3 4B PT
  19. 63.2 Qwen3-32B
  20. 63.2 Command R7B
  21. 62.7 Falcon3-7B-Instruct
  22. 62.4 Mistral 7B v0.3
  23. 62.4 Granite-3.0-8B-Instruct
  24. 61.9 Granite-3.0-8B-Base
  25. 61.5 Qwen3-14B
  26. 61.3 Voxtral Small
  27. 60.9 Mistral 7B v0.1
  28. 60.7 Olmo 3.1 32B Instruct
  29. 58.9 Ministral 3 3B Reasoning
  30. 58.9 Phi-4-reasoning
  31. 58.5 Apertus 8B Instruct
  32. 58.0 Qwen1.5-14B-Chat
  33. 57.7 Starling-LM-7B-beta
  34. 57.6 Mistral-7B-Instruct-v0.1
  35. 57.3 Llama-3.1-Tulu-3-8B
  36. 57.3 Qwen3-30B-A3B
  37. 56.6 Gemma 2 2B IT
  38. 56.2 Cogito v1 Preview 70B
  39. 55.8 Aya Expanse 32B
  40. 55.5 Qwen3-235B-A22B
  41. 55.4 Qwen3-8B
  42. 55.4 qwen3-4b-instruct-2507
  43. 55.3 Granite 4.0 H Small
  44. 54.9 Qwen3-Next-80B-A3B-Instruct
  45. 54.8 Gemma 3 12B
  46. 54.3 granite-4.0-micro
  47. 54.0 Gemma 3n E2B
  48. 53.8 Reka Flash 3
  49. 53.6 deepseek-r1-0528-qwen3-8b
  50. 53.5 Qwen3-1.7B
  51. 53.5 Gemma 3 27B
  52. 53.4 Qwen3-4B
  53. 53.4 Qwen3-4B-Thinking-2507
  54. 51.5 Llama 4 Scout
  55. 50.8 nvidia-nemotron-nano-9b-v2
  56. 50.7 Falcon3-3B-Instruct
  57. 49.6 gpt-oss-20b
  58. 48.5 DeepSeek-R1-Distill-Llama-70B
  59. 47.9 Mistral-7B-Instruct-v0.2
  60. 47.6 Qwen3-Next-80B-A3B
  61. 47.4 DeepSeek-R1-Distill-Qwen-32B
  62. 45.9 QwQ-32B
  63. 44.7 Reka Flash 3.1
  64. 43.3 SmolLM2 1.7B Instruct
  65. 43.2 Gemma 3 1B PT
  66. 41.1 Gemma 3 4B
  67. 40.5 Qwen3-0.6B
  68. 39.0 SmolLM2-1.7B
  69. 38.5 Phi-4
  70. 36.8 Gemma 3 1B
  71. 36.2 DeepSeek-R1-Distill-Qwen-7B
  72. 32.4 DeepSeek-R1-Distill-Qwen-14B
  73. 30.2 GLM-4.5-Air
  74. 30.1 LFM2.5-1.2B-Instruct
  75. 26.4 Llama 3.2 1B
  76. 25.7 AFM-4.5B
  77. 25.0 LFM2.5-1.2B-Thinking
  78. 20.9 Qwen1.5-14B
  79. 20.2 SmolLM2-360M
  80. 20.2 Gemma 3 270M
  81. 18.9 LFM2-1.2B
  82. 18.6 DeepSeek-R1-Distill-Qwen-1.5B
  83. 17.8 Phi-2
  84. 17.7 OLMo 2 32B Instruct
  85. 13.7 GLM-4.1V-9B-Thinking
  86. 12.3 Phi-4-mini-reasoning
  87. 6.53 Nemotron 3 Nano 4B
  88. 4.89 SmolLM2-135M
  1. 1 Gemma 2 27B 70%
  2. 2 Llama 3.1-405B 69.4%
  3. 3 Llama 2-70B 69.2%
  4. 4 Gemma 3 12B PT 68.4%
  5. 5 Gemma 3n E4B 68.4%
  6. 6 Yi-1.5-34B 68.2%
  7. 7 Llama 3.1-70B 68.1%
  8. 8 Gemma 3 27B PT 68%
  9. 9 Apertus 70B 67.6%
  10. 10 Llama 3.1-8B 66.5%
  11. 11 Llama 3-8B 65.8%
  12. 12 Mistral Small 3.1 65.4%
  13. 13 Gemma 2 9B 65.2%
  14. 14 Mistral Small 3.2 64.8%
  15. 15 Nemotron 3 Nano 30B-A3B no-thinking 64.8%
  16. 16 Falcon3-10B-Instruct 64.6%
  17. 17 Gemma 7B 64.3%
  18. 18 Gemma 3 4B PT 64%
  19. 19 Qwen3-32B no-thinking 63.2%
  20. 20 Command R7B 63.2%
  21. 21 Falcon3-7B-Instruct 62.7%
  22. 22 Mistral 7B v0.3 62.4%
  23. 23 Granite-3.0-8B-Instruct 62.4%
  24. 24 Granite-3.0-8B-Base 61.9%
  25. 25 Qwen3-14B no-thinking 61.5%
  26. 26 Voxtral Small 61.3%
  27. 27 Mistral 7B v0.1 60.9%
  28. 28 Olmo 3.1 32B Instruct 60.7%
  29. 29 Ministral 3 3B Reasoning 58.9%
  30. 30 Phi-4-reasoning 58.9%
  31. 31 Apertus 8B Instruct 58.5%
  32. 32 Qwen1.5-14B-Chat 58%
  33. 33 Starling-LM-7B-beta 57.7%
  34. 34 Mistral-7B-Instruct-v0.1 57.6%
  35. 35 Llama-3.1-Tulu-3-8B 57.3%
  36. 36 Qwen3-30B-A3B no-thinking 57.3%
  37. 37 Gemma 2 2B IT 56.6%
  38. 38 Cogito v1 Preview 70B 56.2%
  39. 39 Aya Expanse 32B 55.8%
  40. 40 Qwen3-235B-A22B 55.5%
  41. 41 Qwen3-8B no-thinking 55.4%
  42. 42 qwen3-4b-instruct-2507 55.4%
  43. 43 Granite 4.0 H Small 55.3%
  44. 44 Qwen3-Next-80B-A3B-Instruct 54.9%
  45. 45 Gemma 3 12B 54.8%
  46. 46 granite-4.0-micro 54.3%
  47. 47 Gemma 3n E2B 54%
  48. 48 Reka Flash 3 53.8%
  49. 49 deepseek-r1-0528-qwen3-8b 53.6%
  50. 50 Qwen3-1.7B 53.5%
  51. 51 Gemma 3 27B 53.5%
  52. 52 Qwen3-4B no-thinking 53.4%
  53. 53 Qwen3-4B-Thinking-2507 53.4%
  54. 54 Llama 4 Scout 51.5%
  55. 55 nvidia-nemotron-nano-9b-v2 50.8%
  56. 56 Falcon3-3B-Instruct 50.7%
  57. 57 gpt-oss-20b medium 49.6%
  58. 58 DeepSeek-R1-Distill-Llama-70B 48.5%
  59. 59 Mistral-7B-Instruct-v0.2 47.9%
  60. 60 Qwen3-Next-80B-A3B thinking 47.6%
  61. 61 DeepSeek-R1-Distill-Qwen-32B 47.4%
  62. 62 QwQ-32B 45.9%
  63. 63 Reka Flash 3.1 44.7%
  64. 64 SmolLM2 1.7B Instruct 43.3%
  65. 65 Gemma 3 1B PT 43.2%
  66. 66 Gemma 3 4B 41.1%
  67. 67 Qwen3-0.6B no-thinking 40.5%
  68. 68 SmolLM2-1.7B 39%
  69. 69 Phi-4 38.5%
  70. 70 Gemma 3 1B 36.8%
  71. 71 DeepSeek-R1-Distill-Qwen-7B 36.2%
  72. 72 DeepSeek-R1-Distill-Qwen-14B 32.4%
  73. 73 GLM-4.5-Air no-thinking 30.2%
  74. 74 LFM2.5-1.2B-Instruct 30.1%
  75. 75 Llama 3.2 1B 26.4%
  76. 76 AFM-4.5B 25.7%
  77. 77 LFM2.5-1.2B-Thinking 25%
  78. 78 Qwen1.5-14B 20.9%
  79. 79 SmolLM2-360M 20.2%
  80. 80 Gemma 3 270M 20.2%
  81. 81 LFM2-1.2B 18.9%
  82. 82 DeepSeek-R1-Distill-Qwen-1.5B 18.6%
  83. 83 Phi-2 17.8%
  84. 84 OLMo 2 32B Instruct 17.7%
  85. 85 GLM-4.1V-9B-Thinking 13.7%
  86. 86 Phi-4-mini-reasoning 12.3%
  87. 87 Nemotron 3 Nano 4B 6.53%
  88. 88 SmolLM2-135M 4.89%

Source: EuroEval’s leaderboard

#EntryScore ±SettingDateLinkSource
1 allenai/Olmo-3-1125-32B open weights 72.4% ±0.9 hf.co Official leaderboard
2 Gemma 2 27B 70% ±1.6 Jun 2024 hf.co Official leaderboard
3 Llama 3.1-405B 69.4% ±1.4 Jul 2024 hf.co Official leaderboard
4 Llama 2-70B 69.2% ±1.3 Jul 2023 hf.co Official leaderboard
5 mistralai/Mistral-Small-3.1-24B-Base-2503 open weights 69% ±1.7 hf.co Official leaderboard
6 allenai/OLMo-2-0325-32B open weights 68.4% ±0.5 hf.co Official leaderboard
7 Gemma 3 12B PT 68.4% ±1.8 Mar 2025 hf.co Official leaderboard
8 Gemma 3n E4B 68.4% ±1.4 Jun 2025 hf.co Official leaderboard
9 allenai/OLMo-2-1124-13B open weights 68.3% ±0.4 hf.co Official leaderboard
10 Yi-1.5-34B 68.2% ±1.3 May 2024 hf.co Official leaderboard
11 Llama 3.1-70B 68.1% ±1.8 Jul 2024 hf.co Official leaderboard
12 utter-project/EuroLLM-22B-Preview open weights 68.1% ±1.1 hf.co Official leaderboard
13 allenai/Olmo-3-1025-7B open weights 68% ±0.4 hf.co Official leaderboard
14 upstage/SOLAR-10.7B-v1.0 open weights 68% ±1 hf.co Official leaderboard
15 Gemma 3 27B PT 68% ±1.8 Mar 2025 hf.co Official leaderboard
16 utter-project/EuroLLM-22B-2512 open weights 68% ±1.1 hf.co Official leaderboard
17 Apertus 70B 67.6% ±1.4 Sep 2025 hf.co Official leaderboard
18 speakleash/Bielik-11B-v3-Base-20250730 open weights 67.5% ±0.9 hf.co Official leaderboard
19 mistralai/Ministral-3-3B-Base-2512 open weights 67% ±1.7 hf.co Official leaderboard
20 01-ai/Yi-1.5-9B open weights 66.8% ±0.7 hf.co Official leaderboard
21 allenai/OLMo-2-1124-7B open weights 66.8% ±0.5 hf.co Official leaderboard
22 tiiuae/Falcon3-10B-Base open weights 66.8% ±1.4 hf.co Official leaderboard
23 Llama 3.1-8B 66.5% ±0.9 Jul 2024 hf.co Official leaderboard
24 speakleash/Bielik-11B-v2.3-Instruct open weights 66.3% ±0.8 hf.co Official leaderboard
25 Llama 3-8B 65.8% ±0.8 Apr 2024 hf.co Official leaderboard
26 swiss-ai/Apertus-8B-2509 open weights 65.6% ±1.4 hf.co Official leaderboard
27 ibm-granite/granite-34b-code-base open weights 65.5% ±0.7 hf.co Official leaderboard
28 mistralai/Magistral-Small-2509 open weights 65.5% ±1.6 hf.co Official leaderboard
29 Mistral Small 3.1 65.4% ±1.3 Mar 2025 hf.co Official leaderboard
30 mistralai/Ministral-3-8B-Reasoning-2512 open weights 65.2% ±1.9 hf.co Official leaderboard
31 Gemma 2 9B 65.2% ±2 Jun 2024 hf.co Official leaderboard
32 allenai/Olmo-3.1-32B-Instruct-SFT open weights 64.9% ±1.4 hf.co Official leaderboard
33 Mistral Small 3.2 64.8% ±1.6 Jun 2025 hf.co Official leaderboard
34 Nemotron 3 Nano 30B-A3B 64.8% ±0.8 no-thinking Dec 2025 hf.co Official leaderboard
35 Falcon3-10B-Instruct 64.6% ±1.3 Dec 2024 hf.co Official leaderboard
36 ai21labs/Jamba-v0.1 open weights 64.6% ±1.4 hf.co Official leaderboard
37 TildeAI/TildeOpen-30b open weights 64.5% ±1.6 slow-tokenizer hf.co Official leaderboard
38 bullerwins/Magistral-Small-2506-fp8 open weights 64.5% ±1.6 hf.co Official leaderboard
39 CohereForAI/aya-23-35B open weights 64.4% ±1.1 hf.co Official leaderboard
40 Gemma 7B 64.3% ±1.5 Feb 2024 hf.co Official leaderboard
41 BSC-LT/ALIA-40b open weights 64.2% ±1.7 hf.co Official leaderboard
42 Gemma 3 4B PT 64% ±1.4 Mar 2025 hf.co Official leaderboard
43 syvai/llama3-da-base closed weights 64% ±1.1 Official leaderboard
44 mistralai/Voxtral-Mini-3B-2507 open weights 63.9% ±1.9 hf.co Official leaderboard
45 meta-llama/Llama-2-13b-hf open weights 63.8% ±0.9 hf.co Official leaderboard
46 utter-project/EuroLLM-9B open weights 63.7% ±1.1 hf.co Official leaderboard
47 allenai/Llama-3.1-Tulu-3-8B-SFT open weights 63.5% ±1.1 hf.co Official leaderboard
48 Qwen3-32B 63.2% ±0.8 no-thinking Apr 2025 hf.co Official leaderboard
49 Command R7B 63.2% ±0.9 Dec 2024 hf.co Official leaderboard
50 allenai/Olmo-3.1-32B-Instruct-DPO open weights 63% ±0.6 hf.co Official leaderboard
51 Falcon3-7B-Instruct 62.7% ±0.6 Dec 2024 hf.co Official leaderboard
52 Mistral 7B v0.3 62.4% ±1 Sep 2023 hf.co Official leaderboard
53 Granite-3.0-8B-Instruct 62.4% ±1 Oct 2024 hf.co Official leaderboard
54 allenai/Llama-3.1-Tulu-3-8B-DPO open weights 62% ±1.1 hf.co Official leaderboard
55 Granite-3.0-8B-Base 61.9% ±1.2 Oct 2024 hf.co Official leaderboard
56 HuggingFaceTB/SmolLM3-3B-Base open weights 61.5% ±1.3 hf.co Official leaderboard
57 Qwen3-14B 61.5% ±1 no-thinking Apr 2025 hf.co Official leaderboard
58 Voxtral Small 61.3% ±1.6 Jul 2025 hf.co Official leaderboard
59 mii-llm/nesso-4B open weights 61.2% ±1.4 hf.co Official leaderboard
60 deepcogito/cogito-v1-preview-qwen-32B open weights 61.1% ±0.6 hf.co Official leaderboard
61 ibm-granite/granite-20b-code-base open weights 60.9% ±0.4 hf.co Official leaderboard
62 Mistral 7B v0.1 60.9% ±1.1 Sep 2023 hf.co Official leaderboard
63 Olmo 3.1 32B Instruct 60.7% ±0.9 Dec 2025 hf.co Official leaderboard
64 senseable/WestLake-7B-v2 open weights 60.6% ±1.6 hf.co Official leaderboard
65 NovaSky-AI/Sky-T1-32B-Preview open weights 60.4% ±0.8 hf.co Official leaderboard
66 occiglot/occiglot-7b-eu5 open weights 60.1% ±3 hf.co Official leaderboard
67 tiiuae/Falcon3-7B-Base open weights 60.1% ±1.2 hf.co Official leaderboard
68 NousResearch/DeepHermes-3-Llama-3-8B-Preview open weights 59.8% ±1.4 hf.co Official leaderboard
69 ibm-granite/granite-34b-code-instruct open weights 59.5% ±0.6 hf.co Official leaderboard
70 LumiOpen/Llama-Poro-2-70B-SFT open weights 59.4% ±1.6 hf.co Official leaderboard
71 baidu/ERNIE-4.5-21B-A3B-PT open weights 59.2% ±0.6 hf.co Official leaderboard
72 mistralai/Mistral-Small-24B-Instruct-2501 open weights 59% ±0.7 hf.co Official leaderboard
73 Ministral 3 3B Reasoning 58.9% ±0.8 Dec 2025 hf.co Official leaderboard
74 Phi-4-reasoning 58.9% ±1.6 Apr 2025 hf.co Official leaderboard
75 norallm/normistral-11b-warm open weights 58.6% ±0.9 hf.co Official leaderboard
76 Apertus 8B Instruct 58.5% ±1 Sep 2025 hf.co Official leaderboard
77 MaLA-LM/emma-500-llama2-7b open weights 58.3% ±0.7 hf.co Official leaderboard
78 utter-project/EuroLLM-9B-Instruct open weights 58.2% ±1.7 hf.co Official leaderboard
79 kyutai/helium-1-2b open weights 58.1% ±1 hf.co Official leaderboard
80 Qwen1.5-14B-Chat 58% ±0.9 Feb 2024 hf.co Official leaderboard
81 Starling-LM-7B-beta 57.7% ±1.6 Mar 2024 hf.co Official leaderboard
82 allenai/OLMo-2-1124-7B-Instruct open weights 57.6% ±0.6 hf.co Official leaderboard
83 utter-project/EuroLLM-22B-Instruct-Preview open weights 57.6% ±0.9 hf.co Official leaderboard
84 Mistral-7B-Instruct-v0.1 57.6% ±1.5 Sep 2023 hf.co Official leaderboard
85 meta-llama/Llama-3.1-405B-Instruct-FP8 open weights 57.5% ±0.8 hf.co Official leaderboard
86 TrustLLMeu/baseline-7-8b_2-3t-tokens_llama open weights 57.5% ±1 hf.co Official leaderboard
87 Llama-3.1-Tulu-3-8B 57.3% ±1.3 Nov 2024 hf.co Official leaderboard
88 Qwen3-30B-A3B 57.3% ±0.6 no-thinking Apr 2025 hf.co Official leaderboard
89 arcee-ai/AFM-4.5B-Base open weights 57.2% ±0.8 hf.co Official leaderboard
90 ibm-granite/granite-8b-code-base-4k open weights 57.1% ±0.6 hf.co Official leaderboard
91 google/gemma-2-9b-it open weights 57.1% ±0.9 hf.co Official leaderboard
92 Gemma 2 2B IT 56.6% ±0.9 Jul 2024 hf.co Official leaderboard
93 ibm-granite/granite-3.0-3b-a800m-instruct open weights 56.6% ±0.7 hf.co Official leaderboard
94 Cogito v1 Preview 70B 56.2% ±0.7 Apr 2025 hf.co Official leaderboard
95 HuggingFaceTB/SmolLM3-3B open weights 56.1% ±1 hf.co Official leaderboard
96 Aya Expanse 32B 55.8% ±0.7 Oct 2024 hf.co Official leaderboard
97 google/gemma-2-27b-it open weights 55.7% ±0.7 hf.co Official leaderboard
98 deepcogito/cogito-v1-preview-qwen-14B open weights 55.6% ±0.7 hf.co Official leaderboard
99 Qwen3-235B-A22B 55.5% ±0.4 Apr 2025 hf.co Official leaderboard
100 LumiOpen/Llama-Poro-2-70B-Instruct open weights 55.5% ±1.2 hf.co Official leaderboard
101 152334H/miqu-1-70b-sf open weights 55.4% ±0.7 hf.co Official leaderboard
102 Qwen3-8B 55.4% ±0.8 no-thinking Apr 2025 hf.co Official leaderboard
103 qwen3-4b-instruct-2507 55.4% ±0.6 Aug 2025 hf.co Official leaderboard
104 Granite 4.0 H Small 55.3% ±1.1 Oct 2025 hf.co Official leaderboard
105 Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 closed weights 55.1% ±0.8 hf.co Official leaderboard
106 deepcogito/cogito-v2-preview-llama-70B open weights 55% ±0.6 hf.co Official leaderboard
107 ThatsGroes/gemma-2-27b-it-FP8-Dynamic open weights 54.9% ±0.6 hf.co Official leaderboard
108 Qwen3-Next-80B-A3B-Instruct 54.9% ±0.5 Sep 2025 hf.co Official leaderboard
109 Gemma 3 12B 54.8% ±0.6 Mar 2025 hf.co Official leaderboard
110 allenai/Llama-3.1-Tulu-3.1-8B open weights 54.7% ±1 hf.co Official leaderboard
111 ibm-granite/granite-4.0-h-micro open weights 54.5% ±0.9 hf.co Official leaderboard
112 TrustLLMeu/baseline-7-8b_1t-tokens_llama open weights 54.4% ±0.9 hf.co Official leaderboard
113 granite-4.0-micro 54.3% ±1 Oct 2025 hf.co Official leaderboard
114 Gemma 3n E2B 54% ±0.9 Jun 2025 hf.co Official leaderboard
115 seedboxai/KafkaLM-70B-German-V0.1 open weights 54% ±1.4 hf.co Official leaderboard
116 ibm-granite/granite-8b-code-instruct-4k open weights 53.9% ±0.6 hf.co Official leaderboard
117 Reka Flash 3 53.8% ±0.5 Mar 2025 hf.co Official leaderboard
118 Qwen/Qwen3-235B-A22B-Thinking-2507-FP8 closed weights 53.8% ±0.2 hf.co Official leaderboard
119 deepseek-r1-0528-qwen3-8b 53.6% ±0.5 May 2025 hf.co Official leaderboard
120 allenai/OLMo-2-1124-13B-Instruct open weights 53.6% ±0.8 hf.co Official leaderboard
121 Qwen3-1.7B 53.5% ±1 Apr 2025 hf.co Official leaderboard
122 Gemma 3 27B 53.5% ±0.4 Mar 2025 hf.co Official leaderboard
123 Qwen3-4B 53.4% ±0.9 no-thinking Apr 2025 hf.co Official leaderboard
124 Qwen3-4B-Thinking-2507 53.4% ±0.5 Aug 2025 hf.co Official leaderboard
125 google/gemma-2-2b open weights 53.3% ±1.4 hf.co Official leaderboard
126 infly/OpenCoder-8B-Instruct open weights 53.1% ±0.5 hf.co Official leaderboard
127 ollama_chat/gemma3:27b-it-fp16 open weights 53% ±0.7 ollama.com Official leaderboard
128 common-pile/comma-v0.1-2t open weights 53% ±1.2 hf.co Official leaderboard
129 Qwen/Qwen2.5-1.5B-Instruct open weights 52.7% ±1.1 hf.co Official leaderboard
130 meta-llama/Llama-3.1-8B-Instruct open weights 52.4% ±0.8 hf.co Official leaderboard
131 ibm-granite/granite-20b-code-instruct open weights 52.3% ±0.5 hf.co Official leaderboard
132 meta-llama/Llama-2-13b-chat-hf open weights 51.8% ±0.6 hf.co Official leaderboard
133 meta-llama/Llama-2-70b-chat-hf open weights 51.5% ±0.5 hf.co Official leaderboard
134 Llama 4 Scout 51.5% ±0.5 Apr 2025 hf.co Official leaderboard
135 LumiOpen/Viking-33B open weights 50.8% ±0.9 hf.co Official leaderboard
136 nvidia-nemotron-nano-9b-v2 50.8% ±0.3 Aug 2025 hf.co Official leaderboard
137 Falcon3-3B-Instruct 50.7% ±0.7 Dec 2024 hf.co Official leaderboard
138 LiquidAI/LFM2-2.6B open weights 50.3% ±0.5 hf.co Official leaderboard
139 TrustLLMeu/trustllm-v2-8b-midtrain@3c7af30bc43bb3c8607e0fab6e7f2a23dd64af49 open weights 50% ±1.5 hf.co Official leaderboard
140 ibm-granite/granite-3.0-3b-a800m-base open weights 49.9% ±0.9 hf.co Official leaderboard
141 FacebookAI/xlm-roberta-large open weights 49.8% ±0.7 hf.co Official leaderboard
142 gpt-oss-20b 49.6% ±0.3 medium Aug 2025 hf.co Official leaderboard
143 ollama_chat/hf.co/google/gemma-3-4b-it-qat-q4_0-gguf open weights 49% ±1 ollama.com Official leaderboard
144 speakleash/Bielik-4.5B-v3.0-Instruct open weights 48.6% ±1.1 hf.co Official leaderboard
145 allenai/Olmo-3-7B-Instruct open weights 48.6% ±0.7 hf.co Official leaderboard
146 allenai/Olmo-3-7B-Think open weights 48.6% ±0.5 hf.co Official leaderboard
147 DeepSeek-R1-Distill-Llama-70B 48.5% ±0.5 Jan 2025 hf.co Official leaderboard
148 BSC-LT/salamandra-7b open weights 48.3% ±1.5 hf.co Official leaderboard
149 allenai/Olmo-3-7B-Instruct-SFT open weights 48.2% ±1.2 hf.co Official leaderboard
150 openGPT-X/Teuken-7B-instruct-commercial-v0.4 open weights 48.2% ±0.9 hf.co Official leaderboard
151 Mistral-7B-Instruct-v0.2 47.9% ±1 Dec 2023 hf.co Official leaderboard
152 intfloat/multilingual-e5-large open weights 47.7% ±0.9 hf.co Official leaderboard
153 Qwen3-Next-80B-A3B 47.6% ±0.3 thinking Sep 2025 hf.co Official leaderboard
154 DeepSeek-R1-Distill-Qwen-32B 47.4% ±0.7 Jan 2025 hf.co Official leaderboard
155 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview open weights 46.8% ±0.6 hf.co Official leaderboard
156 deepcogito/cogito-v1-preview-llama-8B open weights 46.5% ±2.5 hf.co Official leaderboard
157 arcee-ai/Trinity-Nano-Preview open weights 46.5% ±2.4 hf.co Official leaderboard
158 allenai/Olmo-3-7B-Instruct-DPO open weights 45.9% ±1 hf.co Official leaderboard
159 QwQ-32B 45.9% ±4.7 Mar 2025 hf.co Official leaderboard
160 ollama_chat/gemma3:4b-it-q4_K_M open weights 45.8% ±0.8 ollama.com Official leaderboard
161 meta-llama/Llama-2-7b-chat-hf open weights 45.4% ±0.7 hf.co Official leaderboard
162 arcee-ai/Trinity-Nano-Base open weights 45.3% ±1.7 hf.co Official leaderboard
163 meta-llama/Llama-3.2-3B-Instruct open weights 45.3% ±0.9 hf.co Official leaderboard
164 speakleash/Bielik-1.5B-v3.0-Instruct open weights 45.2% ±1.5 hf.co Official leaderboard
165 common-pile/comma-v0.1-1t open weights 44.8% ±1.3 hf.co Official leaderboard
166 Reka Flash 3.1 44.7% ±0.6 May 2025 hf.co Official leaderboard
167 Qwen/Qwen2.5-Math-7B open weights 44% ±2.9 hf.co Official leaderboard
168 jhu-clsp/mmBERT-base open weights 43.9% ±0.7 hf.co Official leaderboard
169 TrustLLMeu/trustllm-v2-8b-sft-8k@2aadff8653393e41abf583222164e279461a619f open weights 43.4% ±0.2 zero-shot hf.co Official leaderboard
170 TrustLLMeu/trustllm-v2-8b-sft-mixb2-exp@f33c260efa853437a55d780ce5b910944b1395f6 open weights 43.4% ±1.8 hf.co Official leaderboard
171 SmolLM2 1.7B Instruct 43.3% ±0.8 Oct 2024 hf.co Official leaderboard
172 Gemma 3 1B PT 43.2% ±0.6 Mar 2025 hf.co Official leaderboard
173 seedboxai/KafkaLM-13B-German-V0.1 open weights 43.1% ±1.8 hf.co Official leaderboard
174 occiglot/occiglot-7b-eu5-instruct open weights 43% ±1.1 hf.co Official leaderboard
175 facebook/MobileLLM-R1-950M-base open weights 42.7% ±0.9 hf.co Official leaderboard
176 tiiuae/Falcon3-3B-Base open weights 42.3% ±1.3 hf.co Official leaderboard
177 TrustLLMeu/trustllm-v2-8b@d8cc924fa550fbf9474f5251e8e2f92d1c1bf894 open weights 41.8% ±2.6 hf.co Official leaderboard
178 utter-project/EuroMoE-2.6B-A0.6B-Preview open weights 41.4% ±0.8 hf.co Official leaderboard
179 TrustLLMeu/trustllm-v2-8b-sft@f442f651172c1a97143738fc30c3a6342d1e8660 open weights 41.3% ±0.9 hf.co Official leaderboard
180 yulan-team/YuLan-Mini-Instruct open weights 41.2% ±0.9 hf.co Official leaderboard
181 Gemma 3 4B 41.1% ±2.3 Mar 2025 hf.co Official leaderboard
182 EuroBERT/EuroBERT-610m open weights 40.6% ±1.1 hf.co Official leaderboard
183 Qwen3-0.6B 40.5% ±0.6 no-thinking Apr 2025 hf.co Official leaderboard
184 SmolLM2-1.7B 39% ±1.5 Oct 2024 hf.co Official leaderboard
185 Phi-4 38.5% ±1.6 Dec 2024 hf.co Official leaderboard
186 utter-project/EuroLLM-22B-Instruct-2512 open weights 36.9% ±0.9 hf.co Official leaderboard
187 Gemma 3 1B 36.8% ±0.4 Mar 2025 ollama.com Official leaderboard
188 DeepSeek-R1-Distill-Qwen-7B 36.2% ±0.3 Jan 2025 hf.co Official leaderboard
189 meta-llama/Llama-4-Scout-17B-16E open weights 36.2% ±1.9 hf.co Official leaderboard
190 PleIAs/Pleias-3b-Preview open weights 36% ±1.4 hf.co Official leaderboard
191 tencent/Hunyuan-7B-Instruct open weights 35.4% ±0.4 hf.co Official leaderboard
192 tiiuae/falcon-11B open weights 35.2% ±1.8 hf.co Official leaderboard
193 AI-Sweden-Models/ModernBERT-large open weights 33.6% ±1.1 hf.co Official leaderboard
194 infly/OpenCoder-8B-Base open weights 33.4% ±0.8 hf.co Official leaderboard
195 PleIAs/Pleias-1.2b-Preview open weights 33.2% ±1.2 hf.co Official leaderboard
196 DeepSeek-R1-Distill-Qwen-14B 32.4% ±1.2 Jan 2025 hf.co Official leaderboard
197 ibm-granite/granite-4.0-h-tiny open weights 31.6% ±1.8 hf.co Official leaderboard
198 OpenLLM-France/Lucie-7B open weights 31.3% ±1.1 hf.co Official leaderboard
199 ibm-granite/granite-3b-code-base-2k open weights 31.1% ±1.8 hf.co Official leaderboard
200 GLM-4.5-Air 30.2% ±1.2 no-thinking Jul 2025 hf.co Official leaderboard
201 LFM2.5-1.2B-Instruct 30.1% ±0.5 Jan 2026 hf.co Official leaderboard
202 timpal0l/Mistral-7B-v0.1-flashback-v2 open weights 29.7% ±2.8 hf.co Official leaderboard
203 speakleash/Bielik-11B-v2.6-Instruct open weights 29.1% ±1.4 hf.co Official leaderboard
204 google/gemma-7b-it open weights 28.8% ±0.9 hf.co Official leaderboard
205 tencent/Hunyuan-1.8B-Instruct open weights 28.4% ±0.5 hf.co Official leaderboard
206 jhu-clsp/mmBERT-small open weights 27.4% ±2.2 hf.co Official leaderboard
207 infly/OpenCoder-1.5B-Instruct open weights 26.6% ±0.7 hf.co Official leaderboard
208 Llama 3.2 1B 26.4% ±1.7 Sep 2024 hf.co Official leaderboard
209 deepseek-ai/DeepSeek-R1-Distill-Llama-8B open weights 26.2% ±0.3 hf.co Official leaderboard
210 AFM-4.5B 25.7% ±0.7 Jul 2025 hf.co Official leaderboard
211 LFM2.5-1.2B-Thinking 25% ±0.4 Jan 2026 hf.co Official leaderboard
212 ibm-granite/granite-3b-code-instruct-2k open weights 23.9% ±2.1 hf.co Official leaderboard
213 LiquidAI/LFM2.5-1.2B-Base open weights 23.5% ±0.8 hf.co Official leaderboard
214 HPLT/hplt2c_deu_checkpoints open weights 23.4% ±1.2 hf.co Official leaderboard
215 intfloat/multilingual-e5-base open weights 22.4% ±2.2 hf.co Official leaderboard
216 LiquidAI/LFM2-700M open weights 21.9% ±0.4 hf.co Official leaderboard
217 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct open weights 21.4% ±0.6 hf.co Official leaderboard
218 deepcogito/cogito-v1-preview-llama-3B open weights 21.2% ±1.3 hf.co Official leaderboard
219 Qwen1.5-14B 20.9% ±0.2 Feb 2024 hf.co Official leaderboard
220 utter-project/EuroVLM-1.7B-Preview open weights 20.8% ±0.8 hf.co Official leaderboard
221 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base open weights 20.6% ±0.7 hf.co Official leaderboard
222 SmolLM2-360M 20.2% ±0.5 Oct 2024 hf.co Official leaderboard
223 Gemma 3 270M 20.2% ±0.8 Aug 2025 hf.co Official leaderboard
224 BSC-LT/salamandra-7b-instruct open weights 20.1% ±0.6 hf.co Official leaderboard
225 LiquidAI/LFM2-350M open weights 19.7% ±0.8 hf.co Official leaderboard
226 LFM2-1.2B 18.9% ±0.7 Jul 2025 hf.co Official leaderboard
227 DeepSeek-R1-Distill-Qwen-1.5B 18.6% ±0.3 Jan 2025 hf.co Official leaderboard
228 Phi-2 17.8% ±1.4 Dec 2023 hf.co Official leaderboard
229 PleIAs/Pleias-350m-Preview open weights 17.8% ±0.9 hf.co Official leaderboard
230 OLMo 2 32B Instruct 17.7% ±0.8 Mar 2025 hf.co Official leaderboard
231 facebook/MobileLLM-R1-360M-base open weights 16.8% ±0.6 hf.co Official leaderboard
232 HPLT/hplt2c_eng_checkpoints open weights 16.8% ±0.8 hf.co Official leaderboard
233 baidu/ERNIE-4.5-0.3B-PT open weights 15.9% ±1.3 hf.co Official leaderboard
234 arcee-ai/Trinity-Mini-Base open weights 14.9% ±1.6 hf.co Official leaderboard
235 EuroBERT/EuroBERT-210m open weights 14% ±2.5 hf.co Official leaderboard
236 answerdotai/ModernBERT-large open weights 14% ±1.2 hf.co Official leaderboard
237 GLM-4.1V-9B-Thinking 13.7% ±0.4 Jul 2025 hf.co Official leaderboard
238 Qwen/Qwen2.5-Math-1.5B open weights 13.5% ±1.1 hf.co Official leaderboard
239 AI-Sweden-Models/bert-large-nordic-pile-1M-steps open weights 12.5% ±2 hf.co Official leaderboard
240 Phi-4-mini-reasoning 12.3% ±0.2 Apr 2025 hf.co Official leaderboard
241 PleIAs/Pleias-Pico open weights 12.1% ±0.8 hf.co Official leaderboard
242 answerdotai/ModernBERT-base open weights 12% ±1 hf.co Official leaderboard
243 meta-llama/Llama-3.2-1B-Instruct open weights 11.5% ±1 hf.co Official leaderboard
244 HuggingFaceTB/SmolLM2-360M-Instruct open weights 10.8% ±0.4 hf.co Official leaderboard
245 infly/OpenCoder-1.5B-Base open weights 9.52% ±0.6 hf.co Official leaderboard
246 HPLT/hplt2c_nld_checkpoints open weights 8.85% ±0.8 hf.co Official leaderboard
247 facebook/MobileLLM-R1-140M-base open weights 6.77% ±0.8 hf.co Official leaderboard
248 PleIAs/Baguettotron@refs/pr/6 open weights 6.55% ±0.3 no-thinking hf.co Official leaderboard
249 Nemotron 3 Nano 4B 6.53% ±0.9 Mar 2026 hf.co Official leaderboard
250 EuropeanParliament/EUBERT open weights 5.87% ±0.8 hf.co Official leaderboard
251 SmolLM2-135M 4.89% ±0.3 Oct 2024 hf.co Official leaderboard
252 HuggingFaceTB/SmolLM2-135M-Instruct open weights 4.80% ±0.4 hf.co Official leaderboard
253 AI-Sweden-Models/gpt-sw3-356m-instruct open weights 4.55% ±0.3 hf.co Official leaderboard
254 AI-Sweden-Models/gpt-sw3-126m-instruct open weights 4.21% ±0.3 hf.co Official leaderboard
255 AI-Sweden-Models/gpt-sw3-356m open weights 4.20% ±0.4 hf.co Official leaderboard
256 Shekswess/trlm-135m open weights 3.96% ±0.6 hf.co Official leaderboard
257 DiscoResearch/DiscoLM-70b open weights 3.92% ±0.1 hf.co Official leaderboard
258 alea-institute/kl3m-003-3.7b open weights 3.57% ±0.6 hf.co Official leaderboard
259 PleIAs/Monad open weights 3.42% ±0.1 hf.co Official leaderboard
260 microsoft/mdeberta-v3-base open weights 3.29% ±2.2 hf.co Official leaderboard
261 facebook/MobileLLM-R1-950M open weights 2.69% ±0.3 hf.co Official leaderboard
262 AI-Sweden-Models/gpt-sw3-126m open weights 2.21% ±0.4 hf.co Official leaderboard
263 distilbert/distilbert-base-multilingual-cased open weights 0.77% ±0.4 hf.co Official leaderboard
264 Geotrend/distilbert-base-25lang-cased open weights 0.72% ±0.4 hf.co Official leaderboard
265 facebook/MobileLLM-R1-360M open weights 0.16% ±5.9 hf.co Official leaderboard
266 facebook/MobileLLM-R1-140M open weights 0.02% ±0.5 hf.co Official leaderboard
267 fresh-electra-small open weights 0% Official leaderboard
268 fresh-xlm-roberta-base open weights 0% Official leaderboard
269 openai-community/gpt2 open weights 0% hf.co Official leaderboard
270 openai-community/gpt2-large open weights 0% hf.co Official leaderboard
271 openai-community/gpt2-medium open weights 0% hf.co Official leaderboard
#ModelScore ±SettingDateLinkSource
1 Gemma 2 27B 70% ±1.6 Jun 2024 hf.co Official leaderboard
2 Llama 3.1-405B 69.4% ±1.4 Jul 2024 hf.co Official leaderboard
3 Llama 2-70B 69.2% ±1.3 Jul 2023 hf.co Official leaderboard
4 Gemma 3 12B PT 68.4% ±1.8 Mar 2025 hf.co Official leaderboard
5 Gemma 3n E4B 68.4% ±1.4 Jun 2025 hf.co Official leaderboard
6 Yi-1.5-34B 68.2% ±1.3 May 2024 hf.co Official leaderboard
7 Llama 3.1-70B 68.1% ±1.8 Jul 2024 hf.co Official leaderboard
8 Gemma 3 27B PT 68% ±1.8 Mar 2025 hf.co Official leaderboard
9 Apertus 70B 67.6% ±1.4 Sep 2025 hf.co Official leaderboard
10 Llama 3.1-8B 66.5% ±0.9 Jul 2024 hf.co Official leaderboard
11 Llama 3-8B 65.8% ±0.8 Apr 2024 hf.co Official leaderboard
12 Mistral Small 3.1 65.4% ±1.3 Mar 2025 hf.co Official leaderboard
13 Gemma 2 9B 65.2% ±2 Jun 2024 hf.co Official leaderboard
14 Mistral Small 3.2 64.8% ±1.6 Jun 2025 hf.co Official leaderboard
15 Nemotron 3 Nano 30B-A3B 64.8% ±0.8 no-thinking Dec 2025 hf.co Official leaderboard
16 Falcon3-10B-Instruct 64.6% ±1.3 Dec 2024 hf.co Official leaderboard
17 Gemma 7B 64.3% ±1.5 Feb 2024 hf.co Official leaderboard
18 Gemma 3 4B PT 64% ±1.4 Mar 2025 hf.co Official leaderboard
19 Qwen3-32B 63.2% ±0.8 no-thinking Apr 2025 hf.co Official leaderboard
20 Command R7B 63.2% ±0.9 Dec 2024 hf.co Official leaderboard
21 Falcon3-7B-Instruct 62.7% ±0.6 Dec 2024 hf.co Official leaderboard
22 Mistral 7B v0.3 62.4% ±1 Sep 2023 hf.co Official leaderboard
23 Granite-3.0-8B-Instruct 62.4% ±1 Oct 2024 hf.co Official leaderboard
24 Granite-3.0-8B-Base 61.9% ±1.2 Oct 2024 hf.co Official leaderboard
25 Qwen3-14B 61.5% ±1 no-thinking Apr 2025 hf.co Official leaderboard
26 Voxtral Small 61.3% ±1.6 Jul 2025 hf.co Official leaderboard
27 Mistral 7B v0.1 60.9% ±1.1 Sep 2023 hf.co Official leaderboard
28 Olmo 3.1 32B Instruct 60.7% ±0.9 Dec 2025 hf.co Official leaderboard
29 Ministral 3 3B Reasoning 58.9% ±0.8 Dec 2025 hf.co Official leaderboard
30 Phi-4-reasoning 58.9% ±1.6 Apr 2025 hf.co Official leaderboard
31 Apertus 8B Instruct 58.5% ±1 Sep 2025 hf.co Official leaderboard
32 Qwen1.5-14B-Chat 58% ±0.9 Feb 2024 hf.co Official leaderboard
33 Starling-LM-7B-beta 57.7% ±1.6 Mar 2024 hf.co Official leaderboard
34 Mistral-7B-Instruct-v0.1 57.6% ±1.5 Sep 2023 hf.co Official leaderboard
35 Llama-3.1-Tulu-3-8B 57.3% ±1.3 Nov 2024 hf.co Official leaderboard
36 Qwen3-30B-A3B 57.3% ±0.6 no-thinking Apr 2025 hf.co Official leaderboard
37 Gemma 2 2B IT 56.6% ±0.9 Jul 2024 hf.co Official leaderboard
38 Cogito v1 Preview 70B 56.2% ±0.7 Apr 2025 hf.co Official leaderboard
39 Aya Expanse 32B 55.8% ±0.7 Oct 2024 hf.co Official leaderboard
40 Qwen3-235B-A22B 55.5% ±0.4 Apr 2025 hf.co Official leaderboard
41 Qwen3-8B 55.4% ±0.8 no-thinking Apr 2025 hf.co Official leaderboard
42 qwen3-4b-instruct-2507 55.4% ±0.6 Aug 2025 hf.co Official leaderboard
43 Granite 4.0 H Small 55.3% ±1.1 Oct 2025 hf.co Official leaderboard
44 Qwen3-Next-80B-A3B-Instruct 54.9% ±0.5 Sep 2025 hf.co Official leaderboard
45 Gemma 3 12B 54.8% ±0.6 Mar 2025 hf.co Official leaderboard
46 granite-4.0-micro 54.3% ±1 Oct 2025 hf.co Official leaderboard
47 Gemma 3n E2B 54% ±0.9 Jun 2025 hf.co Official leaderboard
48 Reka Flash 3 53.8% ±0.5 Mar 2025 hf.co Official leaderboard
49 deepseek-r1-0528-qwen3-8b 53.6% ±0.5 May 2025 hf.co Official leaderboard
50 Qwen3-1.7B 53.5% ±1 Apr 2025 hf.co Official leaderboard
51 Gemma 3 27B 53.5% ±0.4 Mar 2025 hf.co Official leaderboard
52 Qwen3-4B 53.4% ±0.9 no-thinking Apr 2025 hf.co Official leaderboard
53 Qwen3-4B-Thinking-2507 53.4% ±0.5 Aug 2025 hf.co Official leaderboard
54 Llama 4 Scout 51.5% ±0.5 Apr 2025 hf.co Official leaderboard
55 nvidia-nemotron-nano-9b-v2 50.8% ±0.3 Aug 2025 hf.co Official leaderboard
56 Falcon3-3B-Instruct 50.7% ±0.7 Dec 2024 hf.co Official leaderboard
57 gpt-oss-20b 49.6% ±0.3 medium Aug 2025 hf.co Official leaderboard
58 DeepSeek-R1-Distill-Llama-70B 48.5% ±0.5 Jan 2025 hf.co Official leaderboard
59 Mistral-7B-Instruct-v0.2 47.9% ±1 Dec 2023 hf.co Official leaderboard
60 Qwen3-Next-80B-A3B 47.6% ±0.3 thinking Sep 2025 hf.co Official leaderboard
61 DeepSeek-R1-Distill-Qwen-32B 47.4% ±0.7 Jan 2025 hf.co Official leaderboard
62 QwQ-32B 45.9% ±4.7 Mar 2025 hf.co Official leaderboard
63 Reka Flash 3.1 44.7% ±0.6 May 2025 hf.co Official leaderboard
64 SmolLM2 1.7B Instruct 43.3% ±0.8 Oct 2024 hf.co Official leaderboard
65 Gemma 3 1B PT 43.2% ±0.6 Mar 2025 hf.co Official leaderboard
66 Gemma 3 4B 41.1% ±2.3 Mar 2025 hf.co Official leaderboard
67 Qwen3-0.6B 40.5% ±0.6 no-thinking Apr 2025 hf.co Official leaderboard
68 SmolLM2-1.7B 39% ±1.5 Oct 2024 hf.co Official leaderboard
69 Phi-4 38.5% ±1.6 Dec 2024 hf.co Official leaderboard
70 Gemma 3 1B 36.8% ±0.4 Mar 2025 ollama.com Official leaderboard
71 DeepSeek-R1-Distill-Qwen-7B 36.2% ±0.3 Jan 2025 hf.co Official leaderboard
72 DeepSeek-R1-Distill-Qwen-14B 32.4% ±1.2 Jan 2025 hf.co Official leaderboard
73 GLM-4.5-Air 30.2% ±1.2 no-thinking Jul 2025 hf.co Official leaderboard
74 LFM2.5-1.2B-Instruct 30.1% ±0.5 Jan 2026 hf.co Official leaderboard
75 Llama 3.2 1B 26.4% ±1.7 Sep 2024 hf.co Official leaderboard
76 AFM-4.5B 25.7% ±0.7 Jul 2025 hf.co Official leaderboard
77 LFM2.5-1.2B-Thinking 25% ±0.4 Jan 2026 hf.co Official leaderboard
78 Qwen1.5-14B 20.9% ±0.2 Feb 2024 hf.co Official leaderboard
79 SmolLM2-360M 20.2% ±0.5 Oct 2024 hf.co Official leaderboard
80 Gemma 3 270M 20.2% ±0.8 Aug 2025 hf.co Official leaderboard
81 LFM2-1.2B 18.9% ±0.7 Jul 2025 hf.co Official leaderboard
82 DeepSeek-R1-Distill-Qwen-1.5B 18.6% ±0.3 Jan 2025 hf.co Official leaderboard
83 Phi-2 17.8% ±1.4 Dec 2023 hf.co Official leaderboard
84 OLMo 2 32B Instruct 17.7% ±0.8 Mar 2025 hf.co Official leaderboard
85 GLM-4.1V-9B-Thinking 13.7% ±0.4 Jul 2025 hf.co Official leaderboard
86 Phi-4-mini-reasoning 12.3% ±0.2 Apr 2025 hf.co Official leaderboard
87 Nemotron 3 Nano 4B 6.53% ±0.9 Mar 2026 hf.co Official leaderboard
88 SmolLM2-135M 4.89% ±0.3 Oct 2024 hf.co Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Qwen3-14B
no-thinking
61.5%
Qwen3-8B
no-thinking
55.4%
Qwen3-1.7B
no-thinking
53.5%
Qwen3-4B
no-thinking
53.4%
nvidia-nemotron-nano-9b-v2
no-thinking
50.8%
gpt-oss-20b
low
49.6%
Qwen3-0.6B
no-thinking
40.5%
GLM-4.5-Air
thinking no-thinking
30.2%

Source: EuroEval’s leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Gemma 2 27B 70%
Llama 3.1-405B 69.4%
Llama 2-70B 69.2%
Gemma 3 12B PT 68.4%
Gemma 3n E4B 68.4%
Yi-1.5-34B 68.2%
Llama 3.1-70B 68.1%
Gemma 3 27B PT 68%
Apertus 70B 67.6%
Llama 3.1-8B 66.5%
Llama 3-8B 65.8%
Mistral Small 3.1 65.4%
Gemma 2 9B 65.2%
Mistral Small 3.2 64.8%
Nemotron 3 Nano 30B-A3B no-thinking 64.8%
Falcon3-10B-Instruct 64.6%
Gemma 7B 64.3%
Gemma 3 4B PT 64%
Qwen3-32B no-thinking 63.2%
Command R7B 63.2%
Falcon3-7B-Instruct 62.7%
Mistral 7B v0.3 62.4%
Granite-3.0-8B-Instruct 62.4%
Granite-3.0-8B-Base 61.9%
Qwen3-14B no-thinking 61.5%
Voxtral Small 61.3%
Mistral 7B v0.1 60.9%
Olmo 3.1 32B Instruct 60.7%
Ministral 3 3B Reasoning 58.9%
Phi-4-reasoning 58.9%
Apertus 8B Instruct 58.5%
Qwen1.5-14B-Chat 58%
Starling-LM-7B-beta 57.7%
Mistral-7B-Instruct-v0.1 57.6%
Llama-3.1-Tulu-3-8B 57.3%
Qwen3-30B-A3B no-thinking 57.3%
Gemma 2 2B IT 56.6%
Qwen3-14B 56.5%
Cogito v1 Preview 70B 56.2%
Apertus 70B 55.9%
Aya Expanse 32B 55.8%
Qwen3-235B-A22B 55.5%
Qwen3-8B no-thinking 55.4%
qwen3-4b-instruct-2507 55.4%
Granite 4.0 H Small 55.3%
Qwen3-Next-80B-A3B-Instruct 54.9%
Gemma 3 12B 54.8%
granite-4.0-micro 54.3%
Gemma 3n E2B 54%
Reka Flash 3 53.8%
deepseek-r1-0528-qwen3-8b 53.6%
Qwen3-1.7B 53.5%
Gemma 3 27B 53.5%
Qwen3-4B no-thinking 53.4%
Qwen3-4B-Thinking-2507 53.4%
Qwen3-1.7B no-thinking 53.4%
Qwen3-4B 53.4%
Llama 4 Scout 51.5%
Qwen3-8B 51.2%
nvidia-nemotron-nano-9b-v2 50.8%
Falcon3-3B-Instruct 50.7%
Gemma 3n E4B 50.2%
gpt-oss-20b medium 49.6%
gpt-oss-20b low 48.8%
DeepSeek-R1-Distill-Llama-70B 48.5%
Mistral-7B-Instruct-v0.2 47.9%
Qwen3-Next-80B-A3B thinking 47.6%
DeepSeek-R1-Distill-Qwen-32B 47.4%
gpt-oss-20b high 45.9%
QwQ-32B 45.9%
Reka Flash 3.1 44.7%
SmolLM2 1.7B Instruct 43.3%
Gemma 3 1B PT 43.2%
Gemma 3 4B 41.1%
Qwen3-0.6B no-thinking 40.5%
Qwen3-0.6B 39.8%
SmolLM2-1.7B 39%
Phi-4 38.5%
Gemma 3 1B 36.8%
DeepSeek-R1-Distill-Qwen-7B 36.2%
DeepSeek-R1-Distill-Qwen-14B 32.4%
GLM-4.5-Air no-thinking 30.2%
LFM2.5-1.2B-Instruct 30.1%
Llama 3.2 1B 26.4%
AFM-4.5B 25.7%
LFM2.5-1.2B-Thinking 25%
GLM-4.5-Air thinking 23.4%
Qwen1.5-14B 20.9%
Gemma 3 1B 20.4%
SmolLM2-360M 20.2%
Gemma 3 270M 20.2%
LFM2-1.2B 18.9%
DeepSeek-R1-Distill-Qwen-1.5B 18.6%
Phi-2 17.8%
OLMo 2 32B Instruct 17.7%
GLM-4.1V-9B-Thinking 13.7%
Phi-4-mini-reasoning 12.3%
Gemma 3 270M 11.2%
Nemotron 3 Nano 4B 6.53%
SmolLM2-135M 4.89%
nvidia-nemotron-nano-9b-v2 no-thinking 1.32%

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Qwen3-14B
no-thinking
61.5%
Qwen3-8B
no-thinking
55.4%
Qwen3-1.7B
no-thinking
53.5%
Qwen3-4B
no-thinking
53.4%
nvidia-nemotron-nano-9b-v2
no-thinking
50.8%
gpt-oss-20b
low
49.6%
Qwen3-0.6B
no-thinking
40.5%
GLM-4.5-Air
thinking no-thinking
30.2%

Source: EuroEval’s leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Gemma 2 27B 70%
Llama 3.1-405B 69.4%
Llama 2-70B 69.2%
Gemma 3 12B PT 68.4%
Gemma 3n E4B 68.4%
Yi-1.5-34B 68.2%
Llama 3.1-70B 68.1%
Gemma 3 27B PT 68%
Apertus 70B 67.6%
Llama 3.1-8B 66.5%
Llama 3-8B 65.8%
Mistral Small 3.1 65.4%
Gemma 2 9B 65.2%
Mistral Small 3.2 64.8%
Nemotron 3 Nano 30B-A3B no-thinking 64.8%
Falcon3-10B-Instruct 64.6%
Gemma 7B 64.3%
Gemma 3 4B PT 64%
Qwen3-32B no-thinking 63.2%
Command R7B 63.2%
Falcon3-7B-Instruct 62.7%
Mistral 7B v0.3 62.4%
Granite-3.0-8B-Instruct 62.4%
Granite-3.0-8B-Base 61.9%
Qwen3-14B no-thinking 61.5%
Voxtral Small 61.3%
Mistral 7B v0.1 60.9%
Olmo 3.1 32B Instruct 60.7%
Ministral 3 3B Reasoning 58.9%
Phi-4-reasoning 58.9%
Apertus 8B Instruct 58.5%
Qwen1.5-14B-Chat 58%
Starling-LM-7B-beta 57.7%
Mistral-7B-Instruct-v0.1 57.6%
Llama-3.1-Tulu-3-8B 57.3%
Qwen3-30B-A3B no-thinking 57.3%
Gemma 2 2B IT 56.6%
Qwen3-14B 56.5%
Cogito v1 Preview 70B 56.2%
Apertus 70B 55.9%
Aya Expanse 32B 55.8%
Qwen3-235B-A22B 55.5%
Qwen3-8B no-thinking 55.4%
qwen3-4b-instruct-2507 55.4%
Granite 4.0 H Small 55.3%
Qwen3-Next-80B-A3B-Instruct 54.9%
Gemma 3 12B 54.8%
granite-4.0-micro 54.3%
Gemma 3n E2B 54%
Reka Flash 3 53.8%
deepseek-r1-0528-qwen3-8b 53.6%
Qwen3-1.7B 53.5%
Gemma 3 27B 53.5%
Qwen3-4B no-thinking 53.4%
Qwen3-4B-Thinking-2507 53.4%
Qwen3-1.7B no-thinking 53.4%
Qwen3-4B 53.4%
Llama 4 Scout 51.5%
Qwen3-8B 51.2%
nvidia-nemotron-nano-9b-v2 50.8%
Falcon3-3B-Instruct 50.7%
Gemma 3n E4B 50.2%
gpt-oss-20b medium 49.6%
gpt-oss-20b low 48.8%
DeepSeek-R1-Distill-Llama-70B 48.5%
Mistral-7B-Instruct-v0.2 47.9%
Qwen3-Next-80B-A3B thinking 47.6%
DeepSeek-R1-Distill-Qwen-32B 47.4%
gpt-oss-20b high 45.9%
QwQ-32B 45.9%
Reka Flash 3.1 44.7%
SmolLM2 1.7B Instruct 43.3%
Gemma 3 1B PT 43.2%
Gemma 3 4B 41.1%
Qwen3-0.6B no-thinking 40.5%
Qwen3-0.6B 39.8%
SmolLM2-1.7B 39%
Phi-4 38.5%
Gemma 3 1B 36.8%
DeepSeek-R1-Distill-Qwen-7B 36.2%
DeepSeek-R1-Distill-Qwen-14B 32.4%
GLM-4.5-Air no-thinking 30.2%
LFM2.5-1.2B-Instruct 30.1%
Llama 3.2 1B 26.4%
AFM-4.5B 25.7%
LFM2.5-1.2B-Thinking 25%
GLM-4.5-Air thinking 23.4%
Qwen1.5-14B 20.9%
Gemma 3 1B 20.4%
SmolLM2-360M 20.2%
Gemma 3 270M 20.2%
LFM2-1.2B 18.9%
DeepSeek-R1-Distill-Qwen-1.5B 18.6%
Phi-2 17.8%
OLMo 2 32B Instruct 17.7%
GLM-4.1V-9B-Thinking 13.7%
Phi-4-mini-reasoning 12.3%
Gemma 3 270M 11.2%
Nemotron 3 Nano 4B 6.53%
SmolLM2-135M 4.89%
nvidia-nemotron-nano-9b-v2 no-thinking 1.32%

2 models have held number one on EuroEval’s leaderboard since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Meta Google DeepMind
EuroEval leaderboard Not tested since Mar 2026

Source: EuroEval’s leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On EuroEval’s leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Gemma 2 27B Llama 2-70B 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Gemma 2 27B

Source: EuroEval’s leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Multilingual

All 545 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%

Data audit

9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 8 months ago, in March 2026, for Nemotron 3 Nano 4B. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (271 entries).
  • Pass Models scored 88 models we track, plus 183 other entries on its leaderboard.
  • Warning Newest score The newest came 8 months ago, in March 2026, for Nemotron 3 Nano 4B.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B (#131).
  • Pass Headroom The best is 70%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on EuroEval: 101 rows for models we track and 183 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check