Benchmark Almanac

EuroEval ERRNews

Llama 3.3 70B Instruct 32.8 leads since August 2025
Not in model cards

Models summarize Estonian radio news transcripts. EuroEval adds stories from TalTech’s training split to make a 2,048-story test split and keeps 256 stories for validation.

EuroEval reports scores for 207 distinct model names across test and validation runs. Its public script rebuilds the splits from the source archive.

Made by
EuroEval
Released
August 2025
Kind
Reading in many languages
Level
School
Scored
Checked automatically
Answers
Open-ended
Tasks
All public
Measures
Capability
Language
Estonian
In and out
Text → Text
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
Strongest model tested
GPT-6.2 Astra #1 on the capability index
Last model added
November 2026
Scores refreshed
November 20th, 2026
199
Entries on its own leaderboard, 76 models we track

Llama 3.3 70B Instruct leads with 32.8 Llama 3.3 70B Instruct leads with 32.8

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

199 entries. Scroll sideways for the rest.

  1. 32.8 Llama 3.3 70B Instruct
  2. 32.7 deepcogito/cogito-v2-preview-llama-70B
  3. 32.3 Gemma 3 27B
  4. 32.2 mistralai/Mistral-Small-3.1-24B-Instruct-2503
  5. 32.2 Gemma 3 12B
  6. 32.1 GPT-6 Astra
  7. 32.1 GPT-6 Sol
  8. 31.9 mistralai/Mistral-Small-3.2-24B-Instruct-2506
  9. 31.9 GPT-6.1 Sol
  10. 31.9 deepcogito/cogito-v1-preview-qwen-32B
  11. 31.8 Qwen/Qwen3-32B#no-thinking
  12. 31.7 GPT-6.2 Astra
  13. 31.7 Qwen2.5-72B
  14. 31.5 GPT-5.6 Sol
  15. 31.5 Qwen/Qwen3-14B#no-thinking
  16. 31.5 Gemma 3n E4B
  17. 31.5 GPT-6.2 Sol
  18. 31.4 deepcogito/cogito-v1-preview-llama-8B
  19. 31.3 deepcogito/cogito-v1-preview-qwen-14B
  20. 31.3 GPT-4.1
  21. 31.2 NbAiLab/borealis2-31b-preview
  22. 31.2 GPT-6 Luna
  23. 31.0 Gemma 4 31B IT
  24. 31.0 Qwen/Qwen3-8B#no-thinking
  25. 31.0 gemini/gemini-3-flash-preview#no-thinking
  26. 30.9 Mistral Medium 3.5
  27. 30.9 deepseek/deepseek-flash#no-thinking
  28. 30.9 utter-project/EuroLLM-22B-Instruct-2512
  29. 30.9 GPT-5.6 Terra
  30. 30.8 Apertus 1.5 70B
  31. 30.8 speakleash/Bielik-Minitron-7B-v3.0-Instruct
  32. 30.8 Voxtral Small
  33. 30.8 Gemma 3 4B
  34. 30.7 Qwen/Qwen3-4B#no-thinking
  35. 30.7 speakleash/Bielik-11B-v3.0-Instruct
  36. 30.6 GLM-5.3-Flash
  37. 30.6 Qwen/Qwen3.5-35B-A3B-FP8
  38. 30.6 gemini/gemini-2.5-flash-lite#no-thinking
  39. 30.5 mii-llm/nesso-4B
  40. 30.5 Qwen3-4B-Thinking-2507
  41. 30.5 meta-llama/Llama-3.1-8B-Instruct
  42. 30.5 speakleash/Bielik-11B-v3.1-Instruct
  43. 30.5 Gemini 3 Pro
  44. 30.4 NbAiLab/borealis2-26b-a4b-preview
  45. 30.3 Gemma 4 26B A4B
  46. 30.3 Gemini 3.7 Flash
  47. 30.3 GPT-5.6 Luna
  48. 30.3 amalia-llm/AMALIA-9B-0626-DPO
  49. 30.3 Qwen 3.8 27B
  50. 30.2 utter-project/EuroLLM-9B-Instruct-2512
  51. 30.2 Apertus 1.5 8B
  52. 30.2 deepcogito/cogito-v1-preview-llama-3B
  53. 30.1 mistralai/Mistral-Large-3-675B-Instruct-2512
  54. 30.1 claude-sonnet-4-5-20250929#no-thinking
  55. 30.1 Gemma 4 12B
  56. 30.1 openai/gpt-5.4-mini-2026-03-17#high
  57. 30.0 openai/gpt-5.4-mini-2026-03-17#medium
  58. 30.0 mistralai/Mistral-Small-4-119B-2603
  59. 29.8 Gemma 4 E4B
  60. 29.8 Grok 4.1 Fast
  61. 29.8 deepseek/deepseek-flash#low
  62. 29.7 amalia-llm/AMALIA-9B-0626-SFT
  63. 29.7 Gemini 3.1 Flash-Lite
  64. 29.6 Qwen/Qwen3.6-27B-FP8
  65. 29.6 openai/gpt-5.4-mini-2026-03-17#none
  66. 29.6 Qwen/Qwen3-1.7B#no-thinking
  67. 29.6 mistralai/Ministral-3-14B-Instruct-2512
  68. 29.4 openai/gpt-5.4-mini-2026-03-17#low
  69. 29.4 Gemma 4 E2B
  70. 29.4 gemini/gemini-3-flash-preview#thinking
  71. 29.4 Gemini 2.5 Flash-Lite
  72. 29.4 Ministral 3 8B Instruct
  73. 29.3 Claude Sonnet 4.6
  74. 29.2 GPT-5 mini
  75. 29.1 claude-sonnet-4-5-20250929#thinking
  76. 29.1 speakleash/Bielik-11B-v2.3-Instruct
  77. 29.1 Ministral 3 3B Reasoning
  78. 29.0 meta-llama/Llama-3.2-3B-Instruct
  79. 28.9 Qwen3.5-2B
  80. 28.9 NbAiLab/borealis2-e4b-preview
  81. 28.9 openai/gpt-5.4-2026-03-05#none
  82. 28.9 EXAONE 4.0 32B
  83. 28.9 granite-4.0-micro
  84. 28.9 Aleph-Alpha/Kolibri-1
  85. 28.8 Claude Haiku 4.5
  86. 28.8 GPT-5.2
  87. 28.7 Ministral 3 3B Instruct
  88. 28.7 Gemini 3.6 Flash
  89. 28.6 ibm-granite/granite-4.0-h-micro
  90. 28.6 openGPT-X/Teuken-7B-instruct-v0.6
  91. 28.5 Gemini 3.5 Flash-Lite
  92. 28.5 Olmo 3.1 32B Instruct
  93. 28.3 NbAiLab/borealis2-e2b-preview
  94. 28.1 allenai/Olmo-3.1-32B-Instruct-DPO
  95. 28.0 glm-4.7-flash
  96. 28.0 Qwen3.5-35B-A3B-Base
  97. 28.0 mistralai/Mistral-Small-3.1-24B-Base-2503
  98. 28.0 swiss-ai/Apertus-v1.1-4B-Instruct
  99. 27.8 Qwen3.5-9B
  100. 27.7 LiquidAI/LFM2.5-8B-A1B-Base
  101. 27.4 Gemma 3 270M
  102. 27.4 HuggingFaceTB/SmolLM3-3B#no-thinking
  103. 27.4 openai/gpt-5.4-nano-2026-03-17#high
  104. 27.3 GadflyII/GLM-4.7-Flash-NVFP4
  105. 27.2 OpenLLM-France/Luciole-23B-Instruct-1.1
  106. 27.1 Aya Expanse 8B
  107. 27.1 openai/gpt-5.4-nano-2026-03-17#medium
  108. 27.0 BSC-LT/salamandra-7b-instruct
  109. 27.0 openai/gpt-5.4-nano-2026-03-17#none
  110. 26.9 openai/gpt-5.4-nano-2026-03-17#low
  111. 26.9 yulan-team/YuLan-Mini-Instruct
  112. 26.6 mistralai/Ministral-3-8B-Reasoning-2512
  113. 26.4 domyn/Domyn-Small-v1.0
  114. 26.3 Llama 3.1-70B
  115. 26.2 gpt-oss-20b
  116. 26.1 gpt-oss-120b
  117. 26.1 swiss-ai/Apertus-v1.1-1.5B-Instruct
  118. 26.0 Gemma 3 1B
  119. 25.9 allenai/Llama-3.1-Tulu-3-8B-SFT
  120. 25.9 GPT-5 nano
  121. 25.8 mistralai/Ministral-3-14B-Base-2512
  122. 25.8 allenai/Olmo-3-1125-32B
  123. 25.8 OpenLLM-France/Luciole-8B-Instruct-1.1
  124. 25.6 Llama 3.1-8B
  125. 25.3 allenai/Olmo-3-7B-Instruct-DPO
  126. 25.2 mistralai/Ministral-3-8B-Base-2512
  127. 25.2 allenai/Olmo-3-7B-Instruct
  128. 24.9 Qwen/Qwen3-0.6B#no-thinking
  129. 24.5 occiglot/occiglot-7b-eu5-instruct
  130. 24.5 Qwen3.5-0.8B
  131. 24.4 Gemma 3 27B PT
  132. 24.3 utter-project/EuroLLM-22B-2512
  133. 24.1 utter-project/EuroLLM-1.7B-Instruct
  134. 24.1 LFM2-1.2B
  135. 24.1 swiss-ai/Apertus-v1.1-4B
  136. 24.0 LiquidAI/LFM2-700M
  137. 24.0 Gemma 3 12B PT
  138. 23.9 swiss-ai/Apertus-v1.1-1.5B
  139. 23.6 danish-foundation-models/DFM-Mimir
  140. 23.5 utter-project/EuroLLM-9B
  141. 23.5 LiquidAI/LFM2-350M
  142. 23.4 swiss-ai/Apertus-v1.1-0.5B-Instruct
  143. 23.1 Qwen3-0.6B
  144. 22.6 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16
  145. 22.2 Gemma 3 4B PT
  146. 22.1 openeurollm/prelude-long
  147. 21.9 Qwen3.5-9B-Base
  148. 21.9 OpenLLM-France/Luciole-1B-Instruct-1.1
  149. 21.9 allenai/Olmo-3-1025-7B
  150. 21.5 openeurollm/prelude-base
  151. 21.4 Qwen3.5-4B
  152. 21.2 openeurollm/oellm-9b-128k-theta32m-v3
  153. 21.1 Olmo 3.1 32B Think
  154. 21.1 HuggingFaceTB/SmolLM3-3B-Base
  155. 21.1 Qwen3.5-4B-Base
  156. 21.0 speakleash/Bielik-1.5B-v3.0-Instruct
  157. 21.0 Llama 3.2 1B
  158. 20.9 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
  159. 20.7 BSC-LT/salamandra-7b
  160. 20.6 danish-foundation-models/DFM-Mimir-v1.5
  161. 20.3 Gemma 3 1B PT
  162. 20.2 AI-Sweden-Models/Llama-3-8B-instruct
  163. 20.1 Qwen3.5-2B-Base
  164. 19.9 occiglot/occiglot-7b-eu5
  165. 19.9 swiss-ai/Apertus-v1.1-0.5B
  166. 19.9 utter-project/EuroLLM-1.7B
  167. 19.6 utter-project/EuroMoE-2.6B-A0.6B-Preview
  168. 19.5 OpenLLM-France/Luciole-8B-Base
  169. 19.1 OpenLLM-France/Luciole-23B-Base
  170. 19.1 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8
  171. 19.1 meta-llama/Llama-3.2-1B-Instruct
  172. 18.8 allenai/Olmo-3-7B-Instruct-SFT
  173. 18.6 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview
  174. 18.5 Qwen3.5-0.8B-Base
  175. 18.4 allenai/Olmo-3.1-32B-Instruct-SFT
  176. 18.0 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
  177. 17.1 PleIAs/Pleias-1.2b-Preview
  178. 17.1 OpenLLM-France/Luciole-1B-Base
  179. 17.0 PleIAs/Pleias-3b-Preview
  180. 16.8 HuggingFaceTB/SmolLM2-360M-Instruct
  181. 16.6 PleIAs/Pleias-350m-Preview
  182. 16.4 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct
  183. 16.3 CohereLabs/aya-23-8B
  184. 16.1 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8
  185. 15.5 Nemotron-Cascade 2 30B-A3B
  186. 14.0 HuggingFaceTB/SmolLM2-135M-Instruct
  187. 13.6 SmolLM2-360M
  188. 13.6 SmolLM2-135M
  189. 12.7 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base
  190. 12.4 TildeAI/TildeOpen-30b
  191. 10.6 AIDC-AI/Marco-Mini-Global-Base
  192. 5.38 PleIAs/Monad
  193. 4.30 LFM2.5-8B-A1B
  194. 4.06 inclusionAI/Ling-mini-2.0
  195. 2.13 openbmb/MiniCPM5-1B
  196. 1.94 Ministral 3 14B
  197. 1.16 Muse-research/Muse-3B
  198. 0.62 inclusionAI/Ling-mini-base-2.0
  199. 0.00 openGPT-X/Teuken-7B-base-v0.6
  1. 1 Llama 3.3 70B Instruct 32.8
  2. 2 deepcogito/cogito-v2-preview-llama-70B 32.7
  3. 3 Gemma 3 27B 32.3
  4. 4 mistralai/Mistral-Small-3.1-24B-Instruct-2503 32.2
  5. 5 Gemma 3 12B 32.2
  6. 6 GPT-6 Astra zero-shot, val 32.1
  7. 7 GPT-6 Sol zero-shot, val 32.1
  8. 8 mistralai/Mistral-Small-3.2-24B-Instruct-2506 31.9
  9. 9 GPT-6.1 Sol zero-shot, val 31.9
  10. 10 deepcogito/cogito-v1-preview-qwen-32B 31.9
  11. 11 Qwen/Qwen3-32B#no-thinking 31.8
  12. 12 GPT-6.2 Astra zero-shot, val 31.7
  13. 13 Qwen2.5-72B 31.7
  14. 14 GPT-5.6 Sol zero-shot, val 31.5
  15. 15 Qwen/Qwen3-14B#no-thinking 31.5
  16. 16 Gemma 3n E4B 31.5
  17. 17 GPT-6.2 Sol zero-shot, val 31.5
  18. 18 deepcogito/cogito-v1-preview-llama-8B 31.4
  19. 19 deepcogito/cogito-v1-preview-qwen-14B 31.3
  20. 20 GPT-4.1 zero-shot, val 31.3
  21. 21 NbAiLab/borealis2-31b-preview val 31.2
  22. 22 GPT-6 Luna zero-shot, val 31.2
  23. 23 Gemma 4 31B IT val 31.0
  24. 24 Qwen/Qwen3-8B#no-thinking 31.0
  25. 25 gemini/gemini-3-flash-preview#no-thinking zero-shot, val 31.0
  26. 26 Mistral Medium 3.5 val 30.9
  27. 27 deepseek/deepseek-flash#no-thinking val 30.9
  28. 28 utter-project/EuroLLM-22B-Instruct-2512 30.9
  29. 29 GPT-5.6 Terra zero-shot, val 30.9
  30. 30 Apertus 1.5 70B val 30.8
  31. 31 speakleash/Bielik-Minitron-7B-v3.0-Instruct val 30.8
  32. 32 Voxtral Small 30.8
  33. 33 Gemma 3 4B 30.8
  34. 34 Qwen/Qwen3-4B#no-thinking 30.7
  35. 35 speakleash/Bielik-11B-v3.0-Instruct val 30.7
  36. 36 GLM-5.3-Flash val 30.6
  37. 37 Qwen/Qwen3.5-35B-A3B-FP8 val 30.6
  38. 38 gemini/gemini-2.5-flash-lite#no-thinking zero-shot, val 30.6
  39. 39 mii-llm/nesso-4B 30.5
  40. 40 Qwen3-4B-Thinking-2507 30.5
  41. 41 meta-llama/Llama-3.1-8B-Instruct 30.5
  42. 42 speakleash/Bielik-11B-v3.1-Instruct val 30.5
  43. 43 Gemini 3 Pro zero-shot, val 30.5
  44. 44 NbAiLab/borealis2-26b-a4b-preview zero-shot, val 30.4
  45. 45 Gemma 4 26B A4B val 30.3
  46. 46 Gemini 3.7 Flash zero-shot, val 30.3
  47. 47 GPT-5.6 Luna zero-shot, val 30.3
  48. 48 amalia-llm/AMALIA-9B-0626-DPO val 30.3
  49. 49 Qwen 3.8 27B val 30.3
  50. 50 utter-project/EuroLLM-9B-Instruct-2512 val 30.2
  51. 51 Apertus 1.5 8B zero-shot, val 30.2
  52. 52 deepcogito/cogito-v1-preview-llama-3B 30.2
  53. 53 mistralai/Mistral-Large-3-675B-Instruct-2512 val 30.1
  54. 54 claude-sonnet-4-5-20250929#no-thinking zero-shot, val 30.1
  55. 55 Gemma 4 12B val 30.1
  56. 56 openai/gpt-5.4-mini-2026-03-17#high zero-shot, val 30.1
  57. 57 openai/gpt-5.4-mini-2026-03-17#medium zero-shot, val 30.0
  58. 58 mistralai/Mistral-Small-4-119B-2603 val 30.0
  59. 59 Gemma 4 E4B val 29.8
  60. 60 Grok 4.1 Fast zero-shot, val 29.8
  61. 61 deepseek/deepseek-flash#low zero-shot, val 29.8
  62. 62 amalia-llm/AMALIA-9B-0626-SFT val 29.7
  63. 63 Gemini 3.1 Flash-Lite zero-shot, val 29.7
  64. 64 Qwen/Qwen3.6-27B-FP8 val 29.6
  65. 65 openai/gpt-5.4-mini-2026-03-17#none zero-shot, val 29.6
  66. 66 Qwen/Qwen3-1.7B#no-thinking 29.6
  67. 67 mistralai/Ministral-3-14B-Instruct-2512 val 29.6
  68. 68 openai/gpt-5.4-mini-2026-03-17#low zero-shot, val 29.4
  69. 69 Gemma 4 E2B val 29.4
  70. 70 gemini/gemini-3-flash-preview#thinking zero-shot, val 29.4
  71. 71 Gemini 2.5 Flash-Lite Sep 2025, zero-shot, val 29.4
  72. 72 Ministral 3 8B Instruct val 29.4
  73. 73 Claude Sonnet 4.6 zero-shot, val 29.3
  74. 74 GPT-5 mini zero-shot, val 29.2
  75. 75 claude-sonnet-4-5-20250929#thinking zero-shot, val 29.1
  76. 76 speakleash/Bielik-11B-v2.3-Instruct val 29.1
  77. 77 Ministral 3 3B Reasoning 29.1
  78. 78 meta-llama/Llama-3.2-3B-Instruct 29.0
  79. 79 Qwen3.5-2B val 28.9
  80. 80 NbAiLab/borealis2-e4b-preview zero-shot, val 28.9
  81. 81 openai/gpt-5.4-2026-03-05#none zero-shot, val 28.9
  82. 82 EXAONE 4.0 32B val 28.9
  83. 83 granite-4.0-micro 28.9
  84. 84 Aleph-Alpha/Kolibri-1 zero-shot, val 28.9
  85. 85 Claude Haiku 4.5 zero-shot, val 28.8
  86. 86 GPT-5.2 zero-shot, val 28.8
  87. 87 Ministral 3 3B Instruct val 28.7
  88. 88 Gemini 3.6 Flash zero-shot, val 28.7
  89. 89 ibm-granite/granite-4.0-h-micro 28.6
  90. 90 openGPT-X/Teuken-7B-instruct-v0.6 val 28.6
  91. 91 Gemini 3.5 Flash-Lite zero-shot, val 28.5
  92. 92 Olmo 3.1 32B Instruct 28.5
  93. 93 NbAiLab/borealis2-e2b-preview zero-shot, val 28.3
  94. 94 allenai/Olmo-3.1-32B-Instruct-DPO 28.1
  95. 95 glm-4.7-flash val 28.0
  96. 96 Qwen3.5-35B-A3B-Base val 28.0
  97. 97 mistralai/Mistral-Small-3.1-24B-Base-2503 28.0
  98. 98 swiss-ai/Apertus-v1.1-4B-Instruct val 28.0
  99. 99 Qwen3.5-9B val 27.8
  100. 100 LiquidAI/LFM2.5-8B-A1B-Base val 27.7
  101. 101 Gemma 3 270M 27.4
  102. 102 HuggingFaceTB/SmolLM3-3B#no-thinking 27.4
  103. 103 openai/gpt-5.4-nano-2026-03-17#high zero-shot, val 27.4
  104. 104 GadflyII/GLM-4.7-Flash-NVFP4 val 27.3
  105. 105 OpenLLM-France/Luciole-23B-Instruct-1.1 val 27.2
  106. 106 Aya Expanse 8B val 27.1
  107. 107 openai/gpt-5.4-nano-2026-03-17#medium zero-shot, val 27.1
  108. 108 BSC-LT/salamandra-7b-instruct 27.0
  109. 109 openai/gpt-5.4-nano-2026-03-17#none zero-shot, val 27.0
  110. 110 openai/gpt-5.4-nano-2026-03-17#low zero-shot, val 26.9
  111. 111 yulan-team/YuLan-Mini-Instruct 26.9
  112. 112 mistralai/Ministral-3-8B-Reasoning-2512 26.6
  113. 113 domyn/Domyn-Small-v1.0 val 26.4
  114. 114 Llama 3.1-70B 26.3
  115. 115 gpt-oss-20b medium 26.2
  116. 116 gpt-oss-120b val 26.1
  117. 117 swiss-ai/Apertus-v1.1-1.5B-Instruct val 26.1
  118. 118 Gemma 3 1B 26.0
  119. 119 allenai/Llama-3.1-Tulu-3-8B-SFT 25.9
  120. 120 GPT-5 nano zero-shot, val 25.9
  121. 121 mistralai/Ministral-3-14B-Base-2512 val 25.8
  122. 122 allenai/Olmo-3-1125-32B 25.8
  123. 123 OpenLLM-France/Luciole-8B-Instruct-1.1 val 25.8
  124. 124 Llama 3.1-8B 25.6
  125. 125 allenai/Olmo-3-7B-Instruct-DPO 25.3
  126. 126 mistralai/Ministral-3-8B-Base-2512 val 25.2
  127. 127 allenai/Olmo-3-7B-Instruct 25.2
  128. 128 Qwen/Qwen3-0.6B#no-thinking 24.9
  129. 129 occiglot/occiglot-7b-eu5-instruct 24.5
  130. 130 Qwen3.5-0.8B val 24.5
  131. 131 Gemma 3 27B PT 24.4
  132. 132 utter-project/EuroLLM-22B-2512 24.3
  133. 133 utter-project/EuroLLM-1.7B-Instruct val 24.1
  134. 134 LFM2-1.2B 24.1
  135. 135 swiss-ai/Apertus-v1.1-4B val 24.1
  136. 136 LiquidAI/LFM2-700M 24.0
  137. 137 Gemma 3 12B PT 24.0
  138. 138 swiss-ai/Apertus-v1.1-1.5B val 23.9
  139. 139 danish-foundation-models/DFM-Mimir val 23.6
  140. 140 utter-project/EuroLLM-9B 23.5
  141. 141 LiquidAI/LFM2-350M 23.5
  142. 142 swiss-ai/Apertus-v1.1-0.5B-Instruct val 23.4
  143. 143 Qwen3-0.6B 23.1
  144. 144 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 val 22.6
  145. 145 Gemma 3 4B PT 22.2
  146. 146 openeurollm/prelude-long val 22.1
  147. 147 Qwen3.5-9B-Base val 21.9
  148. 148 OpenLLM-France/Luciole-1B-Instruct-1.1 val 21.9
  149. 149 allenai/Olmo-3-1025-7B 21.9
  150. 150 openeurollm/prelude-base val 21.5
  151. 151 Qwen3.5-4B val 21.4
  152. 152 openeurollm/oellm-9b-128k-theta32m-v3 val 21.2
  153. 153 Olmo 3.1 32B Think val 21.1
  154. 154 HuggingFaceTB/SmolLM3-3B-Base 21.1
  155. 155 Qwen3.5-4B-Base val 21.1
  156. 156 speakleash/Bielik-1.5B-v3.0-Instruct 21.0
  157. 157 Llama 3.2 1B 21.0
  158. 158 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 val 20.9
  159. 159 BSC-LT/salamandra-7b 20.7
  160. 160 danish-foundation-models/DFM-Mimir-v1.5 val 20.6
  161. 161 Gemma 3 1B PT 20.3
  162. 162 AI-Sweden-Models/Llama-3-8B-instruct val 20.2
  163. 163 Qwen3.5-2B-Base val 20.1
  164. 164 occiglot/occiglot-7b-eu5 19.9
  165. 165 swiss-ai/Apertus-v1.1-0.5B val 19.9
  166. 166 utter-project/EuroLLM-1.7B val 19.9
  167. 167 utter-project/EuroMoE-2.6B-A0.6B-Preview 19.6
  168. 168 OpenLLM-France/Luciole-8B-Base val 19.5
  169. 169 OpenLLM-France/Luciole-23B-Base val 19.1
  170. 170 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 val 19.1
  171. 171 meta-llama/Llama-3.2-1B-Instruct 19.1
  172. 172 allenai/Olmo-3-7B-Instruct-SFT 18.8
  173. 173 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview 18.6
  174. 174 Qwen3.5-0.8B-Base val 18.5
  175. 175 allenai/Olmo-3.1-32B-Instruct-SFT 18.4
  176. 176 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 18.0
  177. 177 PleIAs/Pleias-1.2b-Preview 17.1
  178. 178 OpenLLM-France/Luciole-1B-Base val 17.1
  179. 179 PleIAs/Pleias-3b-Preview 17.0
  180. 180 HuggingFaceTB/SmolLM2-360M-Instruct 16.8
  181. 181 PleIAs/Pleias-350m-Preview 16.6
  182. 182 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct 16.4
  183. 183 CohereLabs/aya-23-8B val 16.3
  184. 184 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 val 16.1
  185. 185 Nemotron-Cascade 2 30B-A3B val 15.5
  186. 186 HuggingFaceTB/SmolLM2-135M-Instruct 14.0
  187. 187 SmolLM2-360M 13.6
  188. 188 SmolLM2-135M 13.6
  189. 189 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base 12.7
  190. 190 TildeAI/TildeOpen-30b val 12.4
  191. 191 AIDC-AI/Marco-Mini-Global-Base val 10.6
  192. 192 PleIAs/Monad 5.38
  193. 193 LFM2.5-8B-A1B val 4.30
  194. 194 inclusionAI/Ling-mini-2.0 val 4.06
  195. 195 openbmb/MiniCPM5-1B val 2.13
  196. 196 Ministral 3 14B val 1.94
  197. 197 Muse-research/Muse-3B val 1.16
  198. 198 inclusionAI/Ling-mini-base-2.0 val 0.62
  199. 199 openGPT-X/Teuken-7B-base-v0.6 val 0.00

Source: Official EuroEval ERRNews leaderboard

76 models. Scroll sideways for the rest.

  1. 32.8 Llama 3.3 70B Instruct
  2. 32.3 Gemma 3 27B
  3. 32.2 Gemma 3 12B
  4. 32.1 GPT-6 Astra
  5. 32.1 GPT-6 Sol
  6. 31.9 GPT-6.1 Sol
  7. 31.7 GPT-6.2 Astra
  8. 31.7 Qwen2.5-72B
  9. 31.5 GPT-5.6 Sol
  10. 31.5 Gemma 3n E4B
  11. 31.5 GPT-6.2 Sol
  12. 31.3 GPT-4.1
  13. 31.2 GPT-6 Luna
  14. 31.0 Gemma 4 31B IT
  15. 30.9 Mistral Medium 3.5
  16. 30.9 GPT-5.6 Terra
  17. 30.8 Apertus 1.5 70B
  18. 30.8 Voxtral Small
  19. 30.8 Gemma 3 4B
  20. 30.6 GLM-5.3-Flash
  21. 30.5 Qwen3-4B-Thinking-2507
  22. 30.5 Gemini 3 Pro
  23. 30.3 Gemma 4 26B A4B
  24. 30.3 Gemini 3.7 Flash
  25. 30.3 GPT-5.6 Luna
  26. 30.3 Qwen 3.8 27B
  27. 30.2 Apertus 1.5 8B
  28. 30.1 Gemma 4 12B
  29. 29.8 Gemma 4 E4B
  30. 29.8 Grok 4.1 Fast
  31. 29.7 Gemini 3.1 Flash-Lite
  32. 29.4 Gemma 4 E2B
  33. 29.4 Gemini 2.5 Flash-Lite
  34. 29.4 Ministral 3 8B Instruct
  35. 29.3 Claude Sonnet 4.6
  36. 29.2 GPT-5 mini
  37. 29.1 Ministral 3 3B Reasoning
  38. 28.9 Qwen3.5-2B
  39. 28.9 EXAONE 4.0 32B
  40. 28.9 granite-4.0-micro
  41. 28.8 Claude Haiku 4.5
  42. 28.8 GPT-5.2
  43. 28.7 Ministral 3 3B Instruct
  44. 28.7 Gemini 3.6 Flash
  45. 28.5 Gemini 3.5 Flash-Lite
  46. 28.5 Olmo 3.1 32B Instruct
  47. 28.0 glm-4.7-flash
  48. 28.0 Qwen3.5-35B-A3B-Base
  49. 27.8 Qwen3.5-9B
  50. 27.4 Gemma 3 270M
  51. 27.1 Aya Expanse 8B
  52. 26.3 Llama 3.1-70B
  53. 26.2 gpt-oss-20b
  54. 26.1 gpt-oss-120b
  55. 26.0 Gemma 3 1B
  56. 25.9 GPT-5 nano
  57. 25.6 Llama 3.1-8B
  58. 24.5 Qwen3.5-0.8B
  59. 24.4 Gemma 3 27B PT
  60. 24.1 LFM2-1.2B
  61. 24.0 Gemma 3 12B PT
  62. 23.1 Qwen3-0.6B
  63. 22.2 Gemma 3 4B PT
  64. 21.9 Qwen3.5-9B-Base
  65. 21.4 Qwen3.5-4B
  66. 21.1 Olmo 3.1 32B Think
  67. 21.1 Qwen3.5-4B-Base
  68. 21.0 Llama 3.2 1B
  69. 20.3 Gemma 3 1B PT
  70. 20.1 Qwen3.5-2B-Base
  71. 18.5 Qwen3.5-0.8B-Base
  72. 15.5 Nemotron-Cascade 2 30B-A3B
  73. 13.6 SmolLM2-360M
  74. 13.6 SmolLM2-135M
  75. 4.30 LFM2.5-8B-A1B
  76. 1.94 Ministral 3 14B
  1. 1 Llama 3.3 70B Instruct 32.8
  2. 2 Gemma 3 27B 32.3
  3. 3 Gemma 3 12B 32.2
  4. 4 GPT-6 Astra zero-shot, val 32.1
  5. 5 GPT-6 Sol zero-shot, val 32.1
  6. 6 GPT-6.1 Sol zero-shot, val 31.9
  7. 7 GPT-6.2 Astra zero-shot, val 31.7
  8. 8 Qwen2.5-72B 31.7
  9. 9 GPT-5.6 Sol zero-shot, val 31.5
  10. 10 Gemma 3n E4B 31.5
  11. 11 GPT-6.2 Sol zero-shot, val 31.5
  12. 12 GPT-4.1 zero-shot, val 31.3
  13. 13 GPT-6 Luna zero-shot, val 31.2
  14. 14 Gemma 4 31B IT val 31.0
  15. 15 Mistral Medium 3.5 val 30.9
  16. 16 GPT-5.6 Terra zero-shot, val 30.9
  17. 17 Apertus 1.5 70B val 30.8
  18. 18 Voxtral Small 30.8
  19. 19 Gemma 3 4B 30.8
  20. 20 GLM-5.3-Flash val 30.6
  21. 21 Qwen3-4B-Thinking-2507 30.5
  22. 22 Gemini 3 Pro zero-shot, val 30.5
  23. 23 Gemma 4 26B A4B val 30.3
  24. 24 Gemini 3.7 Flash zero-shot, val 30.3
  25. 25 GPT-5.6 Luna zero-shot, val 30.3
  26. 26 Qwen 3.8 27B val 30.3
  27. 27 Apertus 1.5 8B zero-shot, val 30.2
  28. 28 Gemma 4 12B val 30.1
  29. 29 Gemma 4 E4B val 29.8
  30. 30 Grok 4.1 Fast zero-shot, val 29.8
  31. 31 Gemini 3.1 Flash-Lite zero-shot, val 29.7
  32. 32 Gemma 4 E2B val 29.4
  33. 33 Gemini 2.5 Flash-Lite Sep 2025, zero-shot, val 29.4
  34. 34 Ministral 3 8B Instruct val 29.4
  35. 35 Claude Sonnet 4.6 zero-shot, val 29.3
  36. 36 GPT-5 mini zero-shot, val 29.2
  37. 37 Ministral 3 3B Reasoning 29.1
  38. 38 Qwen3.5-2B val 28.9
  39. 39 EXAONE 4.0 32B val 28.9
  40. 40 granite-4.0-micro 28.9
  41. 41 Claude Haiku 4.5 zero-shot, val 28.8
  42. 42 GPT-5.2 zero-shot, val 28.8
  43. 43 Ministral 3 3B Instruct val 28.7
  44. 44 Gemini 3.6 Flash zero-shot, val 28.7
  45. 45 Gemini 3.5 Flash-Lite zero-shot, val 28.5
  46. 46 Olmo 3.1 32B Instruct 28.5
  47. 47 glm-4.7-flash val 28.0
  48. 48 Qwen3.5-35B-A3B-Base val 28.0
  49. 49 Qwen3.5-9B val 27.8
  50. 50 Gemma 3 270M 27.4
  51. 51 Aya Expanse 8B val 27.1
  52. 52 Llama 3.1-70B 26.3
  53. 53 gpt-oss-20b medium 26.2
  54. 54 gpt-oss-120b val 26.1
  55. 55 Gemma 3 1B 26.0
  56. 56 GPT-5 nano zero-shot, val 25.9
  57. 57 Llama 3.1-8B 25.6
  58. 58 Qwen3.5-0.8B val 24.5
  59. 59 Gemma 3 27B PT 24.4
  60. 60 LFM2-1.2B 24.1
  61. 61 Gemma 3 12B PT 24.0
  62. 62 Qwen3-0.6B 23.1
  63. 63 Gemma 3 4B PT 22.2
  64. 64 Qwen3.5-9B-Base val 21.9
  65. 65 Qwen3.5-4B val 21.4
  66. 66 Olmo 3.1 32B Think val 21.1
  67. 67 Qwen3.5-4B-Base val 21.1
  68. 68 Llama 3.2 1B 21.0
  69. 69 Gemma 3 1B PT 20.3
  70. 70 Qwen3.5-2B-Base val 20.1
  71. 71 Qwen3.5-0.8B-Base val 18.5
  72. 72 Nemotron-Cascade 2 30B-A3B val 15.5
  73. 73 SmolLM2-360M 13.6
  74. 74 SmolLM2-135M 13.6
  75. 75 LFM2.5-8B-A1B val 4.30
  76. 76 Ministral 3 14B val 1.94

Source: Official EuroEval ERRNews leaderboard

#EntryScore ±SettingDateLinkSource
1 Llama 3.3 70B Instruct 32.8 ±0.1 Dec 2024 hf.co Official leaderboard
2 deepcogito/cogito-v2-preview-llama-70B open weights 32.7 ±0.1 hf.co Official leaderboard
3 Gemma 3 27B 32.3 ±0.1 Mar 2025 hf.co Official leaderboard
4 mistralai/Mistral-Small-3.1-24B-Instruct-2503 open weights 32.2 ±0.1 hf.co Official leaderboard
5 Gemma 3 12B 32.2 ±0.1 Mar 2025 hf.co Official leaderboard
6 GPT-6 Astra 32.1 ±0.2 zero-shot, val Sep 2026 platform.openai.com Official leaderboard
7 GPT-6 Sol 32.1 ±0.2 zero-shot, val Sep 2026 platform.openai.com Official leaderboard
8 mistralai/Mistral-Small-3.2-24B-Instruct-2506 open weights 31.9 ±0.2 hf.co Official leaderboard
9 GPT-6.1 Sol 31.9 ±0.4 zero-shot, val Sep 2026 platform.openai.com Official leaderboard
10 deepcogito/cogito-v1-preview-qwen-32B open weights 31.9 ±0.1 hf.co Official leaderboard
11 Qwen/Qwen3-32B#no-thinking closed weights 31.8 ±0.1 hf.co Official leaderboard
12 GPT-6.2 Astra 31.7 ±0.2 zero-shot, val Nov 2026 platform.openai.com Official leaderboard
13 Qwen2.5-72B 31.7 ±0.1 Sep 2024 hf.co Official leaderboard
14 GPT-5.6 Sol 31.5 ±0.2 zero-shot, val Jul 2026 platform.openai.com Official leaderboard
15 Qwen/Qwen3-14B#no-thinking closed weights 31.5 ±0.1 hf.co Official leaderboard
16 Gemma 3n E4B 31.5 ±0.1 Jun 2025 hf.co Official leaderboard
17 GPT-6.2 Sol 31.5 ±0.2 zero-shot, val Nov 2026 platform.openai.com Official leaderboard
18 deepcogito/cogito-v1-preview-llama-8B open weights 31.4 ±0.2 hf.co Official leaderboard
19 deepcogito/cogito-v1-preview-qwen-14B open weights 31.3 ±0.1 hf.co Official leaderboard
20 GPT-4.1 31.3 ±0.2 zero-shot, val Apr 2025 platform.openai.com Official leaderboard
21 NbAiLab/borealis2-31b-preview open weights 31.2 ±0.3 val hf.co Official leaderboard
22 GPT-6 Luna 31.2 ±0.2 zero-shot, val Sep 2026 platform.openai.com Official leaderboard
23 Gemma 4 31B IT 31.0 ±0.2 val Apr 2026 hf.co Official leaderboard
24 Qwen/Qwen3-8B#no-thinking closed weights 31.0 ±0.1 hf.co Official leaderboard
25 gemini/gemini-3-flash-preview#no-thinking closed weights 31.0 ±0.2 zero-shot, val ai.google.dev Official leaderboard
26 Mistral Medium 3.5 30.9 ±0.2 val Apr 2026 hf.co Official leaderboard
27 deepseek/deepseek-flash#no-thinking closed weights 30.9 ±0.4 val api-docs.deepseek.com Official leaderboard
28 utter-project/EuroLLM-22B-Instruct-2512 open weights 30.9 ±1.1 hf.co Official leaderboard
29 GPT-5.6 Terra 30.9 ±0.2 zero-shot, val Jul 2026 platform.openai.com Official leaderboard
30 Apertus 1.5 70B 30.8 ±0.2 val Jul 2026 hf.co Official leaderboard
31 speakleash/Bielik-Minitron-7B-v3.0-Instruct open weights 30.8 val hf.co Official leaderboard
32 Voxtral Small 30.8 ±0.5 Jul 2025 hf.co Official leaderboard
33 Gemma 3 4B 30.8 ±0.1 Mar 2025 hf.co Official leaderboard
34 Qwen/Qwen3-4B#no-thinking closed weights 30.7 ±0.1 hf.co Official leaderboard
35 speakleash/Bielik-11B-v3.0-Instruct open weights 30.7 val hf.co Official leaderboard
36 GLM-5.3-Flash 30.6 ±0.9 val Aug 2026 hf.co Official leaderboard
37 Qwen/Qwen3.5-35B-A3B-FP8 open weights 30.6 ±0.3 val hf.co Official leaderboard
38 gemini/gemini-2.5-flash-lite#no-thinking closed weights 30.6 ±0.2 zero-shot, val ai.google.dev Official leaderboard
39 mii-llm/nesso-4B open weights 30.5 ±0.2 hf.co Official leaderboard
40 Qwen3-4B-Thinking-2507 30.5 ±0.1 Aug 2025 hf.co Official leaderboard
41 meta-llama/Llama-3.1-8B-Instruct open weights 30.5 ±0.5 hf.co Official leaderboard
42 speakleash/Bielik-11B-v3.1-Instruct open weights 30.5 val hf.co Official leaderboard
43 Gemini 3 Pro 30.5 ±0.2 zero-shot, val Nov 2025 ai.google.dev Official leaderboard
44 NbAiLab/borealis2-26b-a4b-preview open weights 30.4 ±0.2 zero-shot, val hf.co Official leaderboard
45 Gemma 4 26B A4B 30.3 ±0.3 val Apr 2026 hf.co Official leaderboard
46 Gemini 3.7 Flash 30.3 ±0.2 zero-shot, val Aug 2026 ai.google.dev Official leaderboard
47 GPT-5.6 Luna 30.3 ±0.1 zero-shot, val Jul 2026 platform.openai.com Official leaderboard
48 amalia-llm/AMALIA-9B-0626-DPO open weights 30.3 ±0.2 val hf.co Official leaderboard
49 Qwen 3.8 27B 30.3 ±0.3 val Aug 2026 hf.co Official leaderboard
50 utter-project/EuroLLM-9B-Instruct-2512 open weights 30.2 ±0.2 val hf.co Official leaderboard
51 Apertus 1.5 8B 30.2 ±0.2 zero-shot, val Jul 2026 hf.co Official leaderboard
52 deepcogito/cogito-v1-preview-llama-3B open weights 30.2 ±0.1 hf.co Official leaderboard
53 mistralai/Mistral-Large-3-675B-Instruct-2512 open weights 30.1 ±0.2 val hf.co Official leaderboard
54 claude-sonnet-4-5-20250929#no-thinking closed weights 30.1 ±0.2 zero-shot, val docs.anthropic.com Official leaderboard
55 Gemma 4 12B 30.1 ±0.3 val Jun 2026 hf.co Official leaderboard
56 openai/gpt-5.4-mini-2026-03-17#high closed weights 30.1 ±0.2 zero-shot, val platform.openai.com Official leaderboard
57 openai/gpt-5.4-mini-2026-03-17#medium closed weights 30.0 ±0.2 zero-shot, val platform.openai.com Official leaderboard
58 mistralai/Mistral-Small-4-119B-2603 open weights 30.0 ±0.2 val hf.co Official leaderboard
59 Gemma 4 E4B 29.8 ±0.2 val Mar 2026 hf.co Official leaderboard
60 Grok 4.1 Fast 29.8 ±0.2 zero-shot, val Nov 2025 docs.x.ai Official leaderboard
61 deepseek/deepseek-flash#low closed weights 29.8 ±0.2 zero-shot, val api-docs.deepseek.com Official leaderboard
62 amalia-llm/AMALIA-9B-0626-SFT open weights 29.7 ±0.3 val hf.co Official leaderboard
63 Gemini 3.1 Flash-Lite 29.7 ±0.2 zero-shot, val Mar 2026 ai.google.dev Official leaderboard
64 Qwen/Qwen3.6-27B-FP8 open weights 29.6 ±0.3 val hf.co Official leaderboard
65 openai/gpt-5.4-mini-2026-03-17#none closed weights 29.6 ±0.2 zero-shot, val platform.openai.com Official leaderboard
66 Qwen/Qwen3-1.7B#no-thinking closed weights 29.6 ±0.1 hf.co Official leaderboard
67 mistralai/Ministral-3-14B-Instruct-2512 open weights 29.6 ±0.2 val hf.co Official leaderboard
68 openai/gpt-5.4-mini-2026-03-17#low closed weights 29.4 ±0.2 zero-shot, val platform.openai.com Official leaderboard
69 Gemma 4 E2B 29.4 ±0.2 val Mar 2026 hf.co Official leaderboard
70 gemini/gemini-3-flash-preview#thinking closed weights 29.4 ±0.2 zero-shot, val ai.google.dev Official leaderboard
71 Gemini 2.5 Flash-Lite Sep 2025 29.4 ±0.2 zero-shot, val Sep 2025 ai.google.dev Official leaderboard
72 Ministral 3 8B Instruct 29.4 ±0.2 val Dec 2025 hf.co Official leaderboard
73 Claude Sonnet 4.6 29.3 ±0.2 zero-shot, val Feb 2026 docs.anthropic.com Official leaderboard
74 GPT-5 mini 29.2 ±0.2 zero-shot, val Aug 2025 platform.openai.com Official leaderboard
75 claude-sonnet-4-5-20250929#thinking closed weights 29.1 ±0.2 zero-shot, val docs.anthropic.com Official leaderboard
76 speakleash/Bielik-11B-v2.3-Instruct open weights 29.1 ±0.4 val hf.co Official leaderboard
77 Ministral 3 3B Reasoning 29.1 ±0.1 Dec 2025 hf.co Official leaderboard
78 meta-llama/Llama-3.2-3B-Instruct open weights 29.0 ±0.1 hf.co Official leaderboard
79 Qwen3.5-2B 28.9 ±0.3 val Feb 2026 hf.co Official leaderboard
80 NbAiLab/borealis2-e4b-preview open weights 28.9 ±0.2 zero-shot, val hf.co Official leaderboard
81 openai/gpt-5.4-2026-03-05#none closed weights 28.9 ±0.2 zero-shot, val platform.openai.com Official leaderboard
82 EXAONE 4.0 32B 28.9 ±0.2 val Jul 2025 hf.co Official leaderboard
83 granite-4.0-micro 28.9 ±0.7 Oct 2025 hf.co Official leaderboard
84 Aleph-Alpha/Kolibri-1 open weights 28.9 ±0.1 zero-shot, val hf.co Official leaderboard
85 Claude Haiku 4.5 28.8 ±0.2 zero-shot, val Oct 2025 docs.anthropic.com Official leaderboard
86 GPT-5.2 28.8 ±0.2 zero-shot, val Dec 2025 platform.openai.com Official leaderboard
87 Ministral 3 3B Instruct 28.7 ±0.2 val Dec 2025 hf.co Official leaderboard
88 Gemini 3.6 Flash 28.7 ±0.2 zero-shot, val Jul 2026 ai.google.dev Official leaderboard
89 ibm-granite/granite-4.0-h-micro open weights 28.6 ±0.5 hf.co Official leaderboard
90 openGPT-X/Teuken-7B-instruct-v0.6 open weights 28.6 ±0.2 val hf.co Official leaderboard
91 Gemini 3.5 Flash-Lite 28.5 ±0.2 zero-shot, val Jul 2026 ai.google.dev Official leaderboard
92 Olmo 3.1 32B Instruct 28.5 ±0.2 Dec 2025 hf.co Official leaderboard
93 NbAiLab/borealis2-e2b-preview open weights 28.3 ±0.1 zero-shot, val hf.co Official leaderboard
94 allenai/Olmo-3.1-32B-Instruct-DPO open weights 28.1 ±0.3 hf.co Official leaderboard
95 glm-4.7-flash 28.0 ±0.2 val Jan 2026 hf.co Official leaderboard
96 Qwen3.5-35B-A3B-Base 28.0 ±0.4 val Feb 2026 hf.co Official leaderboard
97 mistralai/Mistral-Small-3.1-24B-Base-2503 open weights 28.0 ±1.4 hf.co Official leaderboard
98 swiss-ai/Apertus-v1.1-4B-Instruct open weights 28.0 ±0.9 val hf.co Official leaderboard
99 Qwen3.5-9B 27.8 ±0.7 val Feb 2026 hf.co Official leaderboard
100 LiquidAI/LFM2.5-8B-A1B-Base open weights 27.7 ±0.3 val hf.co Official leaderboard
101 Gemma 3 270M 27.4 ±1.5 Aug 2025 hf.co Official leaderboard
102 HuggingFaceTB/SmolLM3-3B#no-thinking open weights 27.4 ±0.2 hf.co Official leaderboard
103 openai/gpt-5.4-nano-2026-03-17#high closed weights 27.4 ±0.2 zero-shot, val platform.openai.com Official leaderboard
104 GadflyII/GLM-4.7-Flash-NVFP4 open weights 27.3 ±0.4 val hf.co Official leaderboard
105 OpenLLM-France/Luciole-23B-Instruct-1.1 open weights 27.2 ±0.2 val hf.co Official leaderboard
106 Aya Expanse 8B 27.1 ±0.7 val Oct 2024 hf.co Official leaderboard
107 openai/gpt-5.4-nano-2026-03-17#medium closed weights 27.1 ±0.2 zero-shot, val platform.openai.com Official leaderboard
108 BSC-LT/salamandra-7b-instruct open weights 27.0 ±0.5 hf.co Official leaderboard
109 openai/gpt-5.4-nano-2026-03-17#none open weights 27.0 ±0.2 zero-shot, val platform.openai.com Official leaderboard
110 openai/gpt-5.4-nano-2026-03-17#low closed weights 26.9 ±0.2 zero-shot, val platform.openai.com Official leaderboard
111 yulan-team/YuLan-Mini-Instruct open weights 26.9 ±0.4 hf.co Official leaderboard
112 mistralai/Ministral-3-8B-Reasoning-2512 open weights 26.6 ±0.8 hf.co Official leaderboard
113 domyn/Domyn-Small-v1.0 open weights 26.4 ±0.2 val hf.co Official leaderboard
114 Llama 3.1-70B 26.3 ±1.5 Jul 2024 hf.co Official leaderboard
115 gpt-oss-20b 26.2 ±0.1 medium Aug 2025 hf.co Official leaderboard
116 gpt-oss-120b 26.1 ±0.3 val Aug 2025 hf.co Official leaderboard
117 swiss-ai/Apertus-v1.1-1.5B-Instruct open weights 26.1 ±0.9 val hf.co Official leaderboard
118 Gemma 3 1B 26.0 ±0.3 Mar 2025 hf.co Official leaderboard
119 allenai/Llama-3.1-Tulu-3-8B-SFT open weights 25.9 ±1.6 hf.co Official leaderboard
120 GPT-5 nano 25.9 ±0.2 zero-shot, val Aug 2025 platform.openai.com Official leaderboard
121 mistralai/Ministral-3-14B-Base-2512 open weights 25.8 ±1.5 val hf.co Official leaderboard
122 allenai/Olmo-3-1125-32B open weights 25.8 ±1 hf.co Official leaderboard
123 OpenLLM-France/Luciole-8B-Instruct-1.1 open weights 25.8 ±0.2 val hf.co Official leaderboard
124 Llama 3.1-8B 25.6 ±0.9 Jul 2024 hf.co Official leaderboard
125 allenai/Olmo-3-7B-Instruct-DPO open weights 25.3 ±0.9 hf.co Official leaderboard
126 mistralai/Ministral-3-8B-Base-2512 open weights 25.2 ±1.5 val hf.co Official leaderboard
127 allenai/Olmo-3-7B-Instruct open weights 25.2 ±1.3 hf.co Official leaderboard
128 Qwen/Qwen3-0.6B#no-thinking closed weights 24.9 ±0.8 hf.co Official leaderboard
129 occiglot/occiglot-7b-eu5-instruct open weights 24.5 ±1.1 hf.co Official leaderboard
130 Qwen3.5-0.8B 24.5 ±0.3 val Feb 2026 hf.co Official leaderboard
131 Gemma 3 27B PT 24.4 ±1.4 Mar 2025 hf.co Official leaderboard
132 utter-project/EuroLLM-22B-2512 open weights 24.3 ±1.4 hf.co Official leaderboard
133 utter-project/EuroLLM-1.7B-Instruct open weights 24.1 ±1 val hf.co Official leaderboard
134 LFM2-1.2B 24.1 ±0.4 Jul 2025 hf.co Official leaderboard
135 swiss-ai/Apertus-v1.1-4B open weights 24.1 ±1.2 val hf.co Official leaderboard
136 LiquidAI/LFM2-700M open weights 24.0 ±0.4 hf.co Official leaderboard
137 Gemma 3 12B PT 24.0 ±1.3 Mar 2025 hf.co Official leaderboard
138 swiss-ai/Apertus-v1.1-1.5B open weights 23.9 ±0.5 val hf.co Official leaderboard
139 danish-foundation-models/DFM-Mimir open weights 23.6 ±0.7 val hf.co Official leaderboard
140 utter-project/EuroLLM-9B open weights 23.5 ±1.7 hf.co Official leaderboard
141 LiquidAI/LFM2-350M open weights 23.5 ±0.4 hf.co Official leaderboard
142 swiss-ai/Apertus-v1.1-0.5B-Instruct open weights 23.4 ±1.6 val hf.co Official leaderboard
143 Qwen3-0.6B 23.1 ±0.4 Apr 2025 hf.co Official leaderboard
144 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 open weights 22.6 ±1.6 val hf.co Official leaderboard
145 Gemma 3 4B PT 22.2 ±1.2 Mar 2025 hf.co Official leaderboard
146 openeurollm/prelude-long open weights 22.1 ±1.2 val hf.co Official leaderboard
147 Qwen3.5-9B-Base 21.9 ±1.1 val Feb 2026 hf.co Official leaderboard
148 OpenLLM-France/Luciole-1B-Instruct-1.1 open weights 21.9 ±0.1 val hf.co Official leaderboard
149 allenai/Olmo-3-1025-7B open weights 21.9 ±1.1 hf.co Official leaderboard
150 openeurollm/prelude-base open weights 21.5 ±1.2 val hf.co Official leaderboard
151 Qwen3.5-4B 21.4 ±0.4 val Feb 2026 hf.co Official leaderboard
152 openeurollm/oellm-9b-128k-theta32m-v3 open weights 21.2 ±1.4 val hf.co Official leaderboard
153 Olmo 3.1 32B Think 21.1 ±1.3 val Dec 2025 hf.co Official leaderboard
154 HuggingFaceTB/SmolLM3-3B-Base open weights 21.1 ±0.9 hf.co Official leaderboard
155 Qwen3.5-4B-Base 21.1 ±1.2 val Feb 2026 hf.co Official leaderboard
156 speakleash/Bielik-1.5B-v3.0-Instruct open weights 21.0 ±1.8 hf.co Official leaderboard
157 Llama 3.2 1B 21.0 ±0.7 Sep 2024 hf.co Official leaderboard
158 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 open weights 20.9 ±0.2 val hf.co Official leaderboard
159 BSC-LT/salamandra-7b open weights 20.7 ±1.1 hf.co Official leaderboard
160 danish-foundation-models/DFM-Mimir-v1.5 open weights 20.6 ±1 val hf.co Official leaderboard
161 Gemma 3 1B PT 20.3 ±0.7 Mar 2025 hf.co Official leaderboard
162 AI-Sweden-Models/Llama-3-8B-instruct open weights 20.2 ±0.4 val hf.co Official leaderboard
163 Qwen3.5-2B-Base 20.1 ±0.8 val Feb 2026 hf.co Official leaderboard
164 occiglot/occiglot-7b-eu5 open weights 19.9 ±0.9 hf.co Official leaderboard
165 swiss-ai/Apertus-v1.1-0.5B open weights 19.9 ±0.7 val hf.co Official leaderboard
166 utter-project/EuroLLM-1.7B open weights 19.9 ±1.2 val hf.co Official leaderboard
167 utter-project/EuroMoE-2.6B-A0.6B-Preview open weights 19.6 ±0.7 hf.co Official leaderboard
168 OpenLLM-France/Luciole-8B-Base open weights 19.5 ±1.3 val hf.co Official leaderboard
169 OpenLLM-France/Luciole-23B-Base open weights 19.1 ±1.1 val hf.co Official leaderboard
170 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 open weights 19.1 ±0.4 val hf.co Official leaderboard
171 meta-llama/Llama-3.2-1B-Instruct open weights 19.1 ±0.8 hf.co Official leaderboard
172 allenai/Olmo-3-7B-Instruct-SFT open weights 18.8 ±0.9 hf.co Official leaderboard
173 utter-project/EuroMoE-2.6B-A0.6B-Instruct-Preview open weights 18.6 ±2.7 hf.co Official leaderboard
174 Qwen3.5-0.8B-Base 18.5 ±0.7 val Feb 2026 hf.co Official leaderboard
175 allenai/Olmo-3.1-32B-Instruct-SFT open weights 18.4 ±1 hf.co Official leaderboard
176 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 open weights 18.0 ±0.2 hf.co Official leaderboard
177 PleIAs/Pleias-1.2b-Preview open weights 17.1 ±0.9 hf.co Official leaderboard
178 OpenLLM-France/Luciole-1B-Base open weights 17.1 ±1.2 val hf.co Official leaderboard
179 PleIAs/Pleias-3b-Preview open weights 17.0 ±1 hf.co Official leaderboard
180 HuggingFaceTB/SmolLM2-360M-Instruct open weights 16.8 ±1.1 hf.co Official leaderboard
181 PleIAs/Pleias-350m-Preview open weights 16.6 ±1.3 hf.co Official leaderboard
182 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Instruct open weights 16.4 ±1.1 hf.co Official leaderboard
183 CohereLabs/aya-23-8B open weights 16.3 ±1 val hf.co Official leaderboard
184 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 open weights 16.1 ±0.4 val hf.co Official leaderboard
185 Nemotron-Cascade 2 30B-A3B 15.5 ±0.5 val Mar 2026 hf.co Official leaderboard
186 HuggingFaceTB/SmolLM2-135M-Instruct open weights 14.0 ±1.2 hf.co Official leaderboard
187 SmolLM2-360M 13.6 ±0.6 Oct 2024 hf.co Official leaderboard
188 SmolLM2-135M 13.6 ±1.2 Oct 2024 hf.co Official leaderboard
189 tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base open weights 12.7 ±1.1 hf.co Official leaderboard
190 TildeAI/TildeOpen-30b open weights 12.4 ±0.9 val hf.co Official leaderboard
191 AIDC-AI/Marco-Mini-Global-Base open weights 10.6 ±0.5 val hf.co Official leaderboard
192 PleIAs/Monad open weights 5.38 ±0.1 hf.co Official leaderboard
193 LFM2.5-8B-A1B 4.30 ±0.5 val May 2026 hf.co Official leaderboard
194 inclusionAI/Ling-mini-2.0 open weights 4.06 ±0.2 val hf.co Official leaderboard
195 openbmb/MiniCPM5-1B open weights 2.13 ±0.2 val hf.co Official leaderboard
196 Ministral 3 14B 1.94 ±1.1 val Dec 2025 hf.co Official leaderboard
197 Muse-research/Muse-3B open weights 1.16 ±0.2 val hf.co Official leaderboard
198 inclusionAI/Ling-mini-base-2.0 open weights 0.62 ±0.1 val hf.co Official leaderboard
199 openGPT-X/Teuken-7B-base-v0.6 open weights 0.00 val hf.co Official leaderboard
#ModelScore ±SettingDateLinkSource
1 Llama 3.3 70B Instruct 32.8 ±0.1 Dec 2024 hf.co Official leaderboard
2 Gemma 3 27B 32.3 ±0.1 Mar 2025 hf.co Official leaderboard
3 Gemma 3 12B 32.2 ±0.1 Mar 2025 hf.co Official leaderboard
4 GPT-6 Astra 32.1 ±0.2 zero-shot, val Sep 2026 platform.openai.com Official leaderboard
5 GPT-6 Sol 32.1 ±0.2 zero-shot, val Sep 2026 platform.openai.com Official leaderboard
6 GPT-6.1 Sol 31.9 ±0.4 zero-shot, val Sep 2026 platform.openai.com Official leaderboard
7 GPT-6.2 Astra 31.7 ±0.2 zero-shot, val Nov 2026 platform.openai.com Official leaderboard
8 Qwen2.5-72B 31.7 ±0.1 Sep 2024 hf.co Official leaderboard
9 GPT-5.6 Sol 31.5 ±0.2 zero-shot, val Jul 2026 platform.openai.com Official leaderboard
10 Gemma 3n E4B 31.5 ±0.1 Jun 2025 hf.co Official leaderboard
11 GPT-6.2 Sol 31.5 ±0.2 zero-shot, val Nov 2026 platform.openai.com Official leaderboard
12 GPT-4.1 31.3 ±0.2 zero-shot, val Apr 2025 platform.openai.com Official leaderboard
13 GPT-6 Luna 31.2 ±0.2 zero-shot, val Sep 2026 platform.openai.com Official leaderboard
14 Gemma 4 31B IT 31.0 ±0.2 val Apr 2026 hf.co Official leaderboard
15 Mistral Medium 3.5 30.9 ±0.2 val Apr 2026 hf.co Official leaderboard
16 GPT-5.6 Terra 30.9 ±0.2 zero-shot, val Jul 2026 platform.openai.com Official leaderboard
17 Apertus 1.5 70B 30.8 ±0.2 val Jul 2026 hf.co Official leaderboard
18 Voxtral Small 30.8 ±0.5 Jul 2025 hf.co Official leaderboard
19 Gemma 3 4B 30.8 ±0.1 Mar 2025 hf.co Official leaderboard
20 GLM-5.3-Flash 30.6 ±0.9 val Aug 2026 hf.co Official leaderboard
21 Qwen3-4B-Thinking-2507 30.5 ±0.1 Aug 2025 hf.co Official leaderboard
22 Gemini 3 Pro 30.5 ±0.2 zero-shot, val Nov 2025 ai.google.dev Official leaderboard
23 Gemma 4 26B A4B 30.3 ±0.3 val Apr 2026 hf.co Official leaderboard
24 Gemini 3.7 Flash 30.3 ±0.2 zero-shot, val Aug 2026 ai.google.dev Official leaderboard
25 GPT-5.6 Luna 30.3 ±0.1 zero-shot, val Jul 2026 platform.openai.com Official leaderboard
26 Qwen 3.8 27B 30.3 ±0.3 val Aug 2026 hf.co Official leaderboard
27 Apertus 1.5 8B 30.2 ±0.2 zero-shot, val Jul 2026 hf.co Official leaderboard
28 Gemma 4 12B 30.1 ±0.3 val Jun 2026 hf.co Official leaderboard
29 Gemma 4 E4B 29.8 ±0.2 val Mar 2026 hf.co Official leaderboard
30 Grok 4.1 Fast 29.8 ±0.2 zero-shot, val Nov 2025 docs.x.ai Official leaderboard
31 Gemini 3.1 Flash-Lite 29.7 ±0.2 zero-shot, val Mar 2026 ai.google.dev Official leaderboard
32 Gemma 4 E2B 29.4 ±0.2 val Mar 2026 hf.co Official leaderboard
33 Gemini 2.5 Flash-Lite Sep 2025 29.4 ±0.2 zero-shot, val Sep 2025 ai.google.dev Official leaderboard
34 Ministral 3 8B Instruct 29.4 ±0.2 val Dec 2025 hf.co Official leaderboard
35 Claude Sonnet 4.6 29.3 ±0.2 zero-shot, val Feb 2026 docs.anthropic.com Official leaderboard
36 GPT-5 mini 29.2 ±0.2 zero-shot, val Aug 2025 platform.openai.com Official leaderboard
37 Ministral 3 3B Reasoning 29.1 ±0.1 Dec 2025 hf.co Official leaderboard
38 Qwen3.5-2B 28.9 ±0.3 val Feb 2026 hf.co Official leaderboard
39 EXAONE 4.0 32B 28.9 ±0.2 val Jul 2025 hf.co Official leaderboard
40 granite-4.0-micro 28.9 ±0.7 Oct 2025 hf.co Official leaderboard
41 Claude Haiku 4.5 28.8 ±0.2 zero-shot, val Oct 2025 docs.anthropic.com Official leaderboard
42 GPT-5.2 28.8 ±0.2 zero-shot, val Dec 2025 platform.openai.com Official leaderboard
43 Ministral 3 3B Instruct 28.7 ±0.2 val Dec 2025 hf.co Official leaderboard
44 Gemini 3.6 Flash 28.7 ±0.2 zero-shot, val Jul 2026 ai.google.dev Official leaderboard
45 Gemini 3.5 Flash-Lite 28.5 ±0.2 zero-shot, val Jul 2026 ai.google.dev Official leaderboard
46 Olmo 3.1 32B Instruct 28.5 ±0.2 Dec 2025 hf.co Official leaderboard
47 glm-4.7-flash 28.0 ±0.2 val Jan 2026 hf.co Official leaderboard
48 Qwen3.5-35B-A3B-Base 28.0 ±0.4 val Feb 2026 hf.co Official leaderboard
49 Qwen3.5-9B 27.8 ±0.7 val Feb 2026 hf.co Official leaderboard
50 Gemma 3 270M 27.4 ±1.5 Aug 2025 hf.co Official leaderboard
51 Aya Expanse 8B 27.1 ±0.7 val Oct 2024 hf.co Official leaderboard
52 Llama 3.1-70B 26.3 ±1.5 Jul 2024 hf.co Official leaderboard
53 gpt-oss-20b 26.2 ±0.1 medium Aug 2025 hf.co Official leaderboard
54 gpt-oss-120b 26.1 ±0.3 val Aug 2025 hf.co Official leaderboard
55 Gemma 3 1B 26.0 ±0.3 Mar 2025 hf.co Official leaderboard
56 GPT-5 nano 25.9 ±0.2 zero-shot, val Aug 2025 platform.openai.com Official leaderboard
57 Llama 3.1-8B 25.6 ±0.9 Jul 2024 hf.co Official leaderboard
58 Qwen3.5-0.8B 24.5 ±0.3 val Feb 2026 hf.co Official leaderboard
59 Gemma 3 27B PT 24.4 ±1.4 Mar 2025 hf.co Official leaderboard
60 LFM2-1.2B 24.1 ±0.4 Jul 2025 hf.co Official leaderboard
61 Gemma 3 12B PT 24.0 ±1.3 Mar 2025 hf.co Official leaderboard
62 Qwen3-0.6B 23.1 ±0.4 Apr 2025 hf.co Official leaderboard
63 Gemma 3 4B PT 22.2 ±1.2 Mar 2025 hf.co Official leaderboard
64 Qwen3.5-9B-Base 21.9 ±1.1 val Feb 2026 hf.co Official leaderboard
65 Qwen3.5-4B 21.4 ±0.4 val Feb 2026 hf.co Official leaderboard
66 Olmo 3.1 32B Think 21.1 ±1.3 val Dec 2025 hf.co Official leaderboard
67 Qwen3.5-4B-Base 21.1 ±1.2 val Feb 2026 hf.co Official leaderboard
68 Llama 3.2 1B 21.0 ±0.7 Sep 2024 hf.co Official leaderboard
69 Gemma 3 1B PT 20.3 ±0.7 Mar 2025 hf.co Official leaderboard
70 Qwen3.5-2B-Base 20.1 ±0.8 val Feb 2026 hf.co Official leaderboard
71 Qwen3.5-0.8B-Base 18.5 ±0.7 val Feb 2026 hf.co Official leaderboard
72 Nemotron-Cascade 2 30B-A3B 15.5 ±0.5 val Mar 2026 hf.co Official leaderboard
73 SmolLM2-360M 13.6 ±0.6 Oct 2024 hf.co Official leaderboard
74 SmolLM2-135M 13.6 ±1.2 Oct 2024 hf.co Official leaderboard
75 LFM2.5-8B-A1B 4.30 ±0.5 val May 2026 hf.co Official leaderboard
76 Ministral 3 14B 1.94 ±1.1 val Dec 2025 hf.co Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Mistral Medium 3.5
zero-shot, …
30.9
Apertus 1.5 70B
zero-shot, …
30.8
GLM-5.3-Flash
zero-shot, … val
30.6
Apertus 1.5 8B
val
30.2
Ministral 3 3B Instruct
zero-shot, …
28.7
gpt-oss-20b
low high
26.2

Source: Official EuroEval ERRNews leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Llama 3.3 70B Instruct 32.8
Gemma 3 27B 32.3
Gemma 3 12B 32.2
GPT-6 Astra zero-shot, val 32.1
GPT-6 Sol zero-shot, val 32.1
GPT-6.1 Sol zero-shot, val 31.9
GPT-6.2 Astra zero-shot, val 31.7
Qwen2.5-72B 31.7
GPT-5.6 Sol zero-shot, val 31.5
Gemma 3n E4B 31.5
GPT-6.2 Sol zero-shot, val 31.5
GPT-4.1 zero-shot, val 31.3
GPT-6 Luna zero-shot, val 31.2
Gemma 4 31B IT val 31.0
Mistral Medium 3.5 val 30.9
GPT-5.6 Terra zero-shot, val 30.9
Apertus 1.5 70B val 30.8
Voxtral Small 30.8
Gemma 3 4B 30.8
GLM-5.3-Flash val 30.6
Apertus 1.5 70B zero-shot, val 30.5
Qwen3-4B-Thinking-2507 30.5
Gemini 3 Pro zero-shot, val 30.5
Mistral Medium 3.5 zero-shot, val 30.4
Gemma 4 26B A4B val 30.3
Gemini 3.7 Flash zero-shot, val 30.3
GPT-5.6 Luna zero-shot, val 30.3
Qwen 3.8 27B val 30.3
Apertus 1.5 8B zero-shot, val 30.2
Gemma 4 12B val 30.1
Apertus 1.5 8B val 30.0
Gemma 4 E4B val 29.8
Grok 4.1 Fast zero-shot, val 29.8
Gemini 3.1 Flash-Lite zero-shot, val 29.7
Gemma 4 E2B val 29.4
Gemini 2.5 Flash-Lite Sep 2025 zero-shot, val 29.4
Ministral 3 8B Instruct val 29.4
Claude Sonnet 4.6 zero-shot, val 29.3
GPT-5 mini zero-shot, val 29.2
Ministral 3 3B Reasoning 29.1
Qwen3.5-2B val 28.9
EXAONE 4.0 32B val 28.9
granite-4.0-micro 28.9
Claude Haiku 4.5 zero-shot, val 28.8
GPT-5.2 zero-shot, val 28.8
Ministral 3 3B Instruct val 28.7
Gemini 3.6 Flash zero-shot, val 28.7
Gemini 3.5 Flash-Lite zero-shot, val 28.5
Olmo 3.1 32B Instruct 28.5
Ministral 3 3B Instruct zero-shot, val 28.3
glm-4.7-flash val 28.0
Qwen3.5-35B-A3B-Base val 28.0
GLM-5.3-Flash zero-shot, val 27.9
Qwen3.5-9B val 27.8
Gemma 3 270M 27.4
Aya Expanse 8B val 27.1
Llama 3.1-70B 26.3
gpt-oss-20b medium 26.2
gpt-oss-120b val 26.1
Gemma 3 1B 26.0
GPT-5 nano zero-shot, val 25.9
Llama 3.1-8B 25.6
gpt-oss-20b low 25.3
Qwen3.5-0.8B val 24.5
Gemma 3 27B PT 24.4
LFM2-1.2B 24.1
Gemma 3 12B PT 24.0
Qwen3-0.6B 23.1
Gemma 3 4B PT 22.2
Qwen3.5-9B-Base val 21.9
Qwen3.5-4B val 21.4
Gemma 4 12B val 21.2
Olmo 3.1 32B Think val 21.1
Qwen3.5-4B-Base val 21.1
Llama 3.2 1B 21.0
Gemma 3 1B PT 20.3
Qwen3.5-2B-Base val 20.1
Gemma 4 26B A4B val 19.5
Gemma 4 31B IT val 19.2
Qwen3.5-0.8B-Base val 18.5
gpt-oss-20b high 18.0
Gemma 3 270M 17.9
Gemma 4 E4B val 16.1
Gemma 4 E2B val 15.8
Nemotron-Cascade 2 30B-A3B val 15.5
SmolLM2-360M 13.6
SmolLM2-135M 13.6
LFM2.5-8B-A1B val 4.30
Ministral 3 14B val 1.94

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Mistral Medium 3.5
zero-shot, …
30.9
Apertus 1.5 70B
zero-shot, …
30.8
GLM-5.3-Flash
zero-shot, … val
30.6
Apertus 1.5 8B
val
30.2
Ministral 3 3B Instruct
zero-shot, …
28.7
gpt-oss-20b
low high
26.2

Source: Official EuroEval ERRNews leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Llama 3.3 70B Instruct 32.8
Gemma 3 27B 32.3
Gemma 3 12B 32.2
GPT-6 Astra zero-shot, val 32.1
GPT-6 Sol zero-shot, val 32.1
GPT-6.1 Sol zero-shot, val 31.9
GPT-6.2 Astra zero-shot, val 31.7
Qwen2.5-72B 31.7
GPT-5.6 Sol zero-shot, val 31.5
Gemma 3n E4B 31.5
GPT-6.2 Sol zero-shot, val 31.5
GPT-4.1 zero-shot, val 31.3
GPT-6 Luna zero-shot, val 31.2
Gemma 4 31B IT val 31.0
Mistral Medium 3.5 val 30.9
GPT-5.6 Terra zero-shot, val 30.9
Apertus 1.5 70B val 30.8
Voxtral Small 30.8
Gemma 3 4B 30.8
GLM-5.3-Flash val 30.6
Apertus 1.5 70B zero-shot, val 30.5
Qwen3-4B-Thinking-2507 30.5
Gemini 3 Pro zero-shot, val 30.5
Mistral Medium 3.5 zero-shot, val 30.4
Gemma 4 26B A4B val 30.3
Gemini 3.7 Flash zero-shot, val 30.3
GPT-5.6 Luna zero-shot, val 30.3
Qwen 3.8 27B val 30.3
Apertus 1.5 8B zero-shot, val 30.2
Gemma 4 12B val 30.1
Apertus 1.5 8B val 30.0
Gemma 4 E4B val 29.8
Grok 4.1 Fast zero-shot, val 29.8
Gemini 3.1 Flash-Lite zero-shot, val 29.7
Gemma 4 E2B val 29.4
Gemini 2.5 Flash-Lite Sep 2025 zero-shot, val 29.4
Ministral 3 8B Instruct val 29.4
Claude Sonnet 4.6 zero-shot, val 29.3
GPT-5 mini zero-shot, val 29.2
Ministral 3 3B Reasoning 29.1
Qwen3.5-2B val 28.9
EXAONE 4.0 32B val 28.9
granite-4.0-micro 28.9
Claude Haiku 4.5 zero-shot, val 28.8
GPT-5.2 zero-shot, val 28.8
Ministral 3 3B Instruct val 28.7
Gemini 3.6 Flash zero-shot, val 28.7
Gemini 3.5 Flash-Lite zero-shot, val 28.5
Olmo 3.1 32B Instruct 28.5
Ministral 3 3B Instruct zero-shot, val 28.3
glm-4.7-flash val 28.0
Qwen3.5-35B-A3B-Base val 28.0
GLM-5.3-Flash zero-shot, val 27.9
Qwen3.5-9B val 27.8
Gemma 3 270M 27.4
Aya Expanse 8B val 27.1
Llama 3.1-70B 26.3
gpt-oss-20b medium 26.2
gpt-oss-120b val 26.1
Gemma 3 1B 26.0
GPT-5 nano zero-shot, val 25.9
Llama 3.1-8B 25.6
gpt-oss-20b low 25.3
Qwen3.5-0.8B val 24.5
Gemma 3 27B PT 24.4
LFM2-1.2B 24.1
Gemma 3 12B PT 24.0
Qwen3-0.6B 23.1
Gemma 3 4B PT 22.2
Qwen3.5-9B-Base val 21.9
Qwen3.5-4B val 21.4
Gemma 4 12B val 21.2
Olmo 3.1 32B Think val 21.1
Qwen3.5-4B-Base val 21.1
Llama 3.2 1B 21.0
Gemma 3 1B PT 20.3
Qwen3.5-2B-Base val 20.1
Gemma 4 26B A4B val 19.5
Gemma 4 31B IT val 19.2
Qwen3.5-0.8B-Base val 18.5
gpt-oss-20b high 18.0
Gemma 3 270M 17.9
Gemma 4 E4B val 16.1
Gemma 4 E2B val 15.8
Nemotron-Cascade 2 30B-A3B val 15.5
SmolLM2-360M 13.6
SmolLM2-135M 13.6
LFM2.5-8B-A1B val 4.30
Ministral 3 14B val 1.94

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Meta
Own leaderboard 1 leader

Source: Official EuroEval ERRNews leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
0.00 10.0 20.0 30.0 40.0 2025 2026 2027 0.00 10.0 20.0 30.0 40.0 2025 2026 2027

Source: Official EuroEval ERRNews leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Multilingual

All 545 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (199 entries).
  • Pass Models scored 76 models we track, plus 123 other entries on its leaderboard.
  • Pass Newest score The newest is from November 2026, for GPT-6.2 Astra.
  • Pass Frontier models 3 of the top 10 have a score, led by GPT-6.2 Astra (#1).
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on euroeval.com: 89 rows for models we track and 123 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check