Benchmark Almanac

MERA

Claude Opus 4.6 0.86 leads since February 2026
Not in model cards

Models complete Russian-language tasks with several answer formats. The hub port grades short answers from Part 1 of the Russian Unified State Exam.

The 2024 MERA paper compares 19 models. Mistral led overall at 40.0, and ruGPT-3.5 led the original USE task at 2.5 on the 100 point scale.

Released
November 2023
Kind
Knowledge in many languages
Level
University
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Measures
Capability
Language
Russian
In and out
Text → Text
Interaction
Single turn
Items from
Exams and contests
Run by
A host
Strongest model tested
GPT-5.4 #28 on the capability index
Last model added
April 2026
Scores refreshed
November 20th, 2026
209
Entries on its own leaderboard, 102 models we track

Claude Opus 4.6 leads with 0.86 among models released by April 2026 Claude Opus 4.6 leads with 0.86 among models released by April 2026

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

The newest model with a score here came out in April 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

209 entries. Scroll sideways for the rest.

  1. 0.86 Claude Opus 4.6
  2. 0.83 BerryLM-XL
  3. 0.82 Cotype Pro 3
  4. 0.82 BerryLM-L-v2-early-ckpt
  5. 0.82 GPT-5.4
  6. 0.81 BerryLM-v2-reasoning-budget-low
  7. 0.80 GLM-5.1
  8. 0.80 Qwen3-235B-A22B-Thinking
  9. 0.79 Cotype Light 3
  10. 0.79 Qwen 3.6 35B-A3B
  11. 0.79 Qwen3.6 27B
  12. 0.78 Qwen3.5-27B
  13. 0.78 BerryLM-L
  14. 0.77 Qwen3.5-122B-A10B
  15. 0.76 Claude Opus 4.5
  16. 0.75 Qwen3-30B-A3B-Thinking
  17. 0.75 Kimi K2.5
  18. 0.74 BerryLM-MT
  19. 0.74 BerryLM
  20. 0.73 Qwen3-32B
  21. 0.73 Qwen3.5-35B-A3B
  22. 0.71 GigaChat-3.1-Ultra
  23. 0.71 Qwen3-Next-80B-A3B
  24. 0.71 Gemma 4 31B IT
  25. 0.71 Qwen3-4B-Thinking-2507
  26. 0.71 GPT-5.2
  27. 0.69 MiniMax-M2.7
  28. 0.69 Qwen3-14B
  29. 0.69 DeepSeek-V4-Pro
  30. 0.68 Claude 3.7 Sonnet
  31. 0.68 Gemini 2.0 Flash
  32. 0.68 Qwen3.5-9B
  33. 0.68 DeepSeek-V3
  34. 0.68 DeepSeek-V4-Flash
  35. 0.68 Gemini 1.5 Pro
  36. 0.67 Cotype Pro 2.5
  37. 0.67 Qwen3-4B
  38. 0.67 GigaChat 2 Max
  39. 0.66 Qwen2.5-Max
  40. 0.66 T-pro-it-2.0
  41. 0.65 GigaChat 2 Pro
  42. 0.65 Qwen3-8B
  43. 0.64 GPT-4o
  44. 0.64 Gemini 2.0 Flash-Lite
  45. 0.64 Meta-Llama-3.1-405B-Instruct
  46. 0.63 Cotype (preview)
  47. 0.63 T-pro-it-1.0
  48. 0.63 Gemini 1.5 Flash
  49. 0.62 Zero-Mistral-24B
  50. 0.62 Qwen2.5-72B
  51. 0.62 A-vibe
  52. 0.62 DeepSeek-R1-Distill-Qwen-32B
  53. 0.62 gpt-oss-20b
  54. 0.61 Nova Pro 1.0
  55. 0.61 RuadaptQwen-32B-instruct
  56. 0.61 Llama 4 Maverick
  57. 0.60 Gemma 4 26B A4B
  58. 0.60 Qwen2.5-32B-Instruct
  59. 0.60 Supernova-N1-32B-Preview
  60. 0.60 Meta-Llama-3.1-70B-Instruct
  61. 0.59 Mixtral-8x22B-Instruct-v0.1
  62. 0.59 Mistral-Large-Instruct-2407
  63. 0.59 GigaChat Max
  64. 0.59 Meta-Llama-3-70B-Instruct
  65. 0.58 gemma-2-27b-it
  66. 0.57 Llama 3.1-70B
  67. 0.57 Qwen2-72B
  68. 0.57 GPT-4o mini
  69. 0.57 Gemma 3 27B
  70. 0.57 Qwen3.5-4B
  71. 0.56 Llama 3-70B
  72. 0.56 Meno-Lite-0.1
  73. 0.56 Llama 3.3 70B Instruct
  74. 0.55 T-lite-it-1.0
  75. 0.55 Qwen-Turbo
  76. 0.55 Claude 3.5 Haiku
  77. 0.54 GigaChat 2
  78. 0.54 Gemini 1.5 Flash 8B
  79. 0.54 RuadaptQwen2.5-7B-Lite-v1
  80. 0.53 Qwen2.5 14B Instruct
  81. 0.53 Mistral Small 3.1 24B Instruct
  82. 0.53 Llama-3.1-Nemotron-70B-Instruct
  83. 0.53 Llama 4 Scout
  84. 0.52 gemma-2-9b-it
  85. 0.52 Zero-Mistral-Small-24B-Instruct-2501
  86. 0.52 Gemma 2 27B
  87. 0.52 Qwen2-57B-A14B-Instruct
  88. 0.52 Mixtral-8x7B-Instruct-v0.1
  89. 0.51 GigaChat-20B-A3B
  90. 0.51 GigaChat Pro
  91. 0.51 T-lite-0.1
  92. 0.51 phi-3-medium 14B
  93. 0.51 Phi-3-medium-128k-instruct
  94. 0.51 Qwen2.5-Math-72B-Instruct
  95. 0.51 Qwen2.5-Math-72B
  96. 0.50 GigaChat-3.1-Lightning
  97. 0.50 Phi-3.5-MoE
  98. 0.50 GigaChat
  99. 0.49 Qwen2-57B-A14B
  100. 0.49 Qwen2.5-7B-Instruct
  101. 0.49 Mixtral 8x7B
  102. 0.49 Qwen3-30B-A3B-Instruct
  103. 0.49 Meta-Llama-3.1-8B-Instruct
  104. 0.49 Qwen1.5-32B-Chat
  105. 0.48 Qwen1.5-32B
  106. 0.48 Gemma 4 E4B
  107. 0.48 Yi-1.5-34B-32K
  108. 0.48 Meta-Llama-3-8B-Instruct
  109. 0.47 Qwen2-7B-Instruct
  110. 0.47 Olmo-3.1-32B-Instruct-DPO
  111. 0.47 Yi-1.5-34B
  112. 0.47 qwen3-4b-instruct-2507
  113. 0.47 phi-3-small 7.4B
  114. 0.46 Qwen2.5-Coder-7B-Instruct
  115. 0.46 Olmo-3.1-32B-Instruct-SFT
  116. 0.46 Qwen2.5-7B
  117. 0.46 Command R (03-2024)
  118. 0.46 Qwen2-7B
  119. 0.45 Gemma 2 9B
  120. 0.45 ruadapt llama3-8B-instruct lep ft
  121. 0.45 Llama 3.1-8B
  122. 0.45 DeepSeek-R1-Distill-Qwen-14B
  123. 0.45 Llama 3-8B
  124. 0.44 Yi-1.5-34B-Chat-16K
  125. 0.44 Qwen2.5-3B-Instruct
  126. 0.44 Yi-34B
  127. 0.44 GPT-3.5 Turbo
  128. 0.44 Yi-1.5-34B-Chat
  129. 0.43 Mistral-7B-Instruct-v0.3
  130. 0.43 Olmo 3.1 32B Instruct
  131. 0.43 Yi-1.5-9B
  132. 0.43 Yi-1.5-9B-32K
  133. 0.43 Yi-34B-200K
  134. 0.42 Yi-34B-Chat
  135. 0.42 Watari-7b-v1
  136. 0.42 Mistral NeMo
  137. 0.41 ruadapt_qwen2.5_3B_ext_u48_instruct_v4
  138. 0.41 Qwen2.5-3B
  139. 0.41 Mistral-7B-Instruct-v0.2
  140. 0.40 Mistral 7B v0.1
  141. 0.40 Gemma 4 E2B
  142. 0.40 Yi-1.5-9B-Chat-16K
  143. 0.40 phi-3-mini 3.8B
  144. 0.40 Yi-Coder-9B-Chat
  145. 0.40 Mistral 7B v0.3
  146. 0.40 Phi-3-mini-128k-instruct
  147. 0.39 Qwen3-0.6B
  148. 0.39 Yi-1.5-9B-Chat
  149. 0.39 Gemma 2 2B IT
  150. 0.38 Gamayun-1.5B
  151. 0.38 Llama-3.2-3B-Instruct
  152. 0.37 Yi-Coder-9B
  153. 0.37 Qwen1.5-7B
  154. 0.37 LFM2-24B-A2B
  155. 0.37 Mistral-7B-Instruct-v0.1
  156. 0.37 Qwen1.5-7B-Chat
  157. 0.36 Meno-Tiny-0.1
  158. 0.36 Llama 3.2 3B
  159. 0.36 Qwen2.5 1.5B Instruct
  160. 0.36 Yi-1.5-6B
  161. 0.35 Qwen2.5-1.5B-Instruct
  162. 0.35 LFM-3B
  163. 0.35 Qwen3.5-2B
  164. 0.35 Qwen2.5-Math-7B-Instruct
  165. 0.34 Llama-2-13b-hf
  166. 0.34 Qwen2-Math-7B-Instruct
  167. 0.34 Olmo-3-7B-Instruct
  168. 0.34 Qwen2-1.5B-Instruct
  169. 0.34 Qwen3-1.7B
  170. 0.34 Qwen1.5-MoE-A2.7B-Chat
  171. 0.33 Yi-1.5-6B-Chat
  172. 0.33 Qwen2-1.5B
  173. 0.32 Qwen1.5-4B
  174. 0.32 Yi 6B
  175. 0.32 Qwen2.5-Coder (1.5B)
  176. 0.31 gemma-2-2b
  177. 0.31 Qwen1.5-4B-Chat
  178. 0.31 Qwen2.5-Math-1.5B
  179. 0.31 Qwen2.5-Coder-1.5B-Instruct
  180. 0.31 Yi-6B-200K
  181. 0.30 Llama-2-7b-chat-hf
  182. 0.30 Qwen3.5-0.8B
  183. 0.30 Llama-2-7b-hf
  184. 0.30 Llama-3.2-1B-Instruct
  185. 0.29 Qwen2.5-0.5B-Instruct
  186. 0.28 Qwen2.5-Math-1.5B-Instruct
  187. 0.28 LFM2-2.6B
  188. 0.28 LFM2-8B-A1B
  189. 0.27 Qwen1.5-1.8B
  190. 0.26 Qwen1.5-1.8B-Chat
  191. 0.26 Qwen2-0.5B-Instruct
  192. 0.25 Llama-2-13b-chat-hf
  193. 0.25 Llama 3.2 1B
  194. 0.25 Qwen2-0.5B
  195. 0.24 LFM2-700M
  196. 0.23 Yi-6B-Chat
  197. 0.22 Megra 1.1 709M (SFT-4096)
  198. 0.22 Qwen1.5-0.5B
  199. 0.22 Qwen1.5-0.5B-Chat
  200. 0.21 LFM2-350M
  201. 0.21 ruGPT-3.5
  202. 0.21 ruGPT3-medium
  203. 0.20 ruGPT3-large
  204. 0.20 ruGPT-3-small
  205. 0.20 LFM2.5-1.2B-Instruct
  206. 0.20 ruGPT-3 XL 1.3B
  207. 0.19 ruT5-base
  208. 0.19 Qwen2-Math-1.5B-Instruct
  209. 0.17 ruGPT-3-medium
  1. 1 Claude Opus 4.6 generative-scoring, multi-turn, chat template, system prompt 0.86
  2. 2 BerryLM-XL generative-scoring, multi-turn, chat template, system prompt 0.83
  3. 3 Cotype Pro 3 generative-scoring, multi-turn, chat template, system prompt 0.82
  4. 4 BerryLM-L-v2-early-ckpt generative-scoring, multi-turn, system prompt 0.82
  5. 5 GPT-5.4 generative-scoring, multi-turn, chat template, system prompt 0.82
  6. 6 BerryLM-v2-reasoning-budget-low generative-scoring, multi-turn, chat template, system prompt 0.81
  7. 7 GLM-5.1 generative-scoring, multi-turn, chat template, system prompt 0.80
  8. 8 Qwen3-235B-A22B-Thinking Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.80
  9. 9 Cotype Light 3 generative-scoring, multi-turn, chat template, system prompt 0.79
  10. 10 Qwen 3.6 35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.79
  11. 11 Qwen3.6 27B generative-scoring, multi-turn, chat template, system prompt 0.79
  12. 12 Qwen3.5-27B generative-scoring, multi-turn, chat template, system prompt 0.78
  13. 13 BerryLM-L generative-scoring, multi-turn, chat template, system prompt 0.78
  14. 14 Qwen3.5-122B-A10B generative-scoring, multi-turn, chat template, system prompt 0.77
  15. 15 Claude Opus 4.5 generative-scoring, multi-turn, chat template, system prompt 0.76
  16. 16 Qwen3-30B-A3B-Thinking Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.75
  17. 17 Kimi K2.5 generative-scoring, multi-turn, chat template, system prompt 0.75
  18. 18 BerryLM-MT generative-scoring, multi-turn, chat template, system prompt 0.74
  19. 19 BerryLM generative-scoring, chat template, system prompt 0.74
  20. 20 Qwen3-32B generative-scoring, multi-turn, chat template, system prompt 0.73
  21. 21 Qwen3.5-35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.73
  22. 22 GigaChat-3.1-Ultra generative-scoring, multi-turn, chat template, system prompt 0.71
  23. 23 Qwen3-Next-80B-A3B generative-scoring, multi-turn, chat template, system prompt 0.71
  24. 24 Gemma 4 31B IT generative-scoring, multi-turn, chat template, system prompt 0.71
  25. 25 Qwen3-4B-Thinking-2507 generative-scoring, multi-turn, chat template, system prompt 0.71
  26. 26 GPT-5.2 generative-scoring, multi-turn, chat template, system prompt 0.71
  27. 27 MiniMax-M2.7 generative-scoring, multi-turn 0.69
  28. 28 Qwen3-14B generative-scoring, multi-turn, chat template, system prompt 0.69
  29. 29 DeepSeek-V4-Pro generative-scoring, multi-turn, chat template, system prompt 0.69
  30. 30 Claude 3.7 Sonnet generative-scoring, multi-turn, chat template, system prompt 0.68
  31. 31 Gemini 2.0 Flash Feb 2025, generative-scoring, multi-turn, chat template, system prompt 0.68
  32. 32 Qwen3.5-9B generative-scoring, multi-turn, chat template, system prompt 0.68
  33. 33 DeepSeek-V3 Mar 2025, generative-scoring, multi-turn, chat template, system prompt 0.68
  34. 34 DeepSeek-V4-Flash generative-scoring, multi-turn, chat template, system prompt 0.68
  35. 35 Gemini 1.5 Pro Sep 2024, generative-scoring, multi-turn, chat template, system prompt 0.68
  36. 36 Cotype Pro 2.5 generative-scoring, multi-turn, chat template, system prompt 0.67
  37. 37 Qwen3-4B generative-scoring, multi-turn, chat template, system prompt 0.67
  38. 38 GigaChat 2 Max generative-scoring, multi-turn, chat template, system prompt 0.67
  39. 39 Qwen2.5-Max generative-scoring, multi-turn, chat template, system prompt 0.66
  40. 40 T-pro-it-2.0 generative-scoring, multi-turn, chat template, system prompt 0.66
  41. 41 GigaChat 2 Pro generative-scoring, multi-turn, chat template, system prompt 0.65
  42. 42 Qwen3-8B generative-scoring, multi-turn, chat template, system prompt 0.65
  43. 43 GPT-4o Mar 2025, generative-scoring, multi-turn, chat template, system prompt 0.64
  44. 44 Gemini 2.0 Flash-Lite generative-scoring, multi-turn, chat template, system prompt 0.64
  45. 45 Meta-Llama-3.1-405B-Instruct log-likelihood 0.64
  46. 46 Cotype (preview) generative-scoring, multi-turn, chat template, system prompt 0.63
  47. 47 T-pro-it-1.0 generative-scoring, multi-turn, chat template, system prompt 0.63
  48. 48 Gemini 1.5 Flash Sep 2024, generative-scoring, multi-turn, chat template, system prompt 0.63
  49. 49 Zero-Mistral-24B generative-scoring, multi-turn, chat template, system prompt 0.62
  50. 50 Qwen2.5-72B log-likelihood, system prompt 0.62
  51. 51 A-vibe generative-scoring, multi-turn, chat template, system prompt 0.62
  52. 52 DeepSeek-R1-Distill-Qwen-32B generative-scoring, chat template, system prompt 0.62
  53. 53 gpt-oss-20b generative-scoring, multi-turn, chat template, system prompt 0.62
  54. 54 Nova Pro 1.0 generative-scoring, multi-turn, chat template, system prompt 0.61
  55. 55 RuadaptQwen-32B-instruct generative-scoring, multi-turn, chat template, system prompt 0.61
  56. 56 Llama 4 Maverick generative-scoring, multi-turn, chat template, system prompt 0.61
  57. 57 Gemma 4 26B A4B generative-scoring, multi-turn, chat template, system prompt 0.60
  58. 58 Qwen2.5-32B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.60
  59. 59 Supernova-N1-32B-Preview generative-scoring, multi-turn, chat template, system prompt 0.60
  60. 60 Meta-Llama-3.1-70B-Instruct log-likelihood 0.60
  61. 61 Mixtral-8x22B-Instruct-v0.1 log-likelihood 0.59
  62. 62 Mistral-Large-Instruct-2407 log-likelihood, chat template, system prompt 0.59
  63. 63 GigaChat Max generative-scoring, multi-turn, chat template, system prompt 0.59
  64. 64 Meta-Llama-3-70B-Instruct log-likelihood 0.59
  65. 65 gemma-2-27b-it log-likelihood 0.58
  66. 66 Llama 3.1-70B log-likelihood 0.57
  67. 67 Qwen2-72B generative-scoring, multi-turn, chat template, system prompt 0.57
  68. 68 GPT-4o mini generative-scoring, multi-turn, chat template, system prompt 0.57
  69. 69 Gemma 3 27B generative-scoring, multi-turn, chat template, system prompt 0.57
  70. 70 Qwen3.5-4B generative-scoring, multi-turn, chat template, system prompt 0.57
  71. 71 Llama 3-70B generative-scoring 0.56
  72. 72 Meno-Lite-0.1 generative-scoring, multi-turn, chat template, system prompt 0.56
  73. 73 Llama 3.3 70B Instruct generative-scoring, multi-turn, chat template, system prompt 0.56
  74. 74 T-lite-it-1.0 generative-scoring, multi-turn, chat template, system prompt 0.55
  75. 75 Qwen-Turbo generative-scoring, multi-turn, chat template, system prompt 0.55
  76. 76 Claude 3.5 Haiku generative-scoring, multi-turn, chat template, system prompt 0.55
  77. 77 GigaChat 2 generative-scoring, multi-turn, chat template, system prompt 0.54
  78. 78 Gemini 1.5 Flash 8B generative-scoring, multi-turn, chat template, system prompt 0.54
  79. 79 RuadaptQwen2.5-7B-Lite-v1 generative-scoring, multi-turn, chat template, system prompt 0.54
  80. 80 Qwen2.5 14B Instruct generative-scoring, chat template, system prompt 0.53
  81. 81 Mistral Small 3.1 24B Instruct generative-scoring, multi-turn, chat template, system prompt 0.53
  82. 82 Llama-3.1-Nemotron-70B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.53
  83. 83 Llama 4 Scout generative-scoring, multi-turn, chat template, system prompt 0.53
  84. 84 gemma-2-9b-it log-likelihood 0.52
  85. 85 Zero-Mistral-Small-24B-Instruct-2501 generative-scoring, multi-turn, chat template, system prompt 0.52
  86. 86 Gemma 2 27B log-likelihood 0.52
  87. 87 Qwen2-57B-A14B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.52
  88. 88 Mixtral-8x7B-Instruct-v0.1 log-likelihood 0.52
  89. 89 GigaChat-20B-A3B generative-scoring, multi-turn, chat template 0.51
  90. 90 GigaChat Pro generative-scoring, multi-turn, chat template, system prompt 0.51
  91. 91 T-lite-0.1 log-likelihood 0.51
  92. 92 phi-3-medium 14B log-likelihood 0.51
  93. 93 Phi-3-medium-128k-instruct log-likelihood 0.51
  94. 94 Qwen2.5-Math-72B-Instruct log-likelihood, system prompt 0.51
  95. 95 Qwen2.5-Math-72B log-likelihood, system prompt 0.51
  96. 96 GigaChat-3.1-Lightning generative-scoring, multi-turn, chat template, system prompt 0.50
  97. 97 Phi-3.5-MoE log-likelihood 0.50
  98. 98 GigaChat generative-scoring, multi-turn, chat template, system prompt 0.50
  99. 99 Qwen2-57B-A14B log-likelihood 0.49
  100. 100 Qwen2.5-7B-Instruct log-likelihood, system prompt 0.49
  101. 101 Mixtral 8x7B log-likelihood 0.49
  102. 102 Qwen3-30B-A3B-Instruct Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.49
  103. 103 Meta-Llama-3.1-8B-Instruct log-likelihood 0.49
  104. 104 Qwen1.5-32B-Chat log-likelihood, multi-turn, chat template 0.49
  105. 105 Qwen1.5-32B log-likelihood 0.48
  106. 106 Gemma 4 E4B generative-scoring, multi-turn, chat template, system prompt 0.48
  107. 107 Yi-1.5-34B-32K log-likelihood 0.48
  108. 108 Meta-Llama-3-8B-Instruct log-likelihood 0.48
  109. 109 Qwen2-7B-Instruct log-likelihood 0.47
  110. 110 Olmo-3.1-32B-Instruct-DPO generative-scoring, multi-turn, chat template, system prompt 0.47
  111. 111 Yi-1.5-34B log-likelihood 0.47
  112. 112 qwen3-4b-instruct-2507 generative-scoring, multi-turn, chat template, system prompt 0.47
  113. 113 phi-3-small 7.4B log-likelihood 0.47
  114. 114 Qwen2.5-Coder-7B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.46
  115. 115 Olmo-3.1-32B-Instruct-SFT generative-scoring, multi-turn, chat template, system prompt 0.46
  116. 116 Qwen2.5-7B log-likelihood, system prompt 0.46
  117. 117 Command R (03-2024) generative-scoring, multi-turn, chat template, system prompt 0.46
  118. 118 Qwen2-7B log-likelihood 0.46
  119. 119 Gemma 2 9B log-likelihood 0.45
  120. 120 ruadapt llama3-8B-instruct lep ft log-likelihood, multi-turn, chat template, system prompt 0.45
  121. 121 Llama 3.1-8B log-likelihood 0.45
  122. 122 DeepSeek-R1-Distill-Qwen-14B log-likelihood 0.45
  123. 123 Llama 3-8B generative-scoring 0.45
  124. 124 Yi-1.5-34B-Chat-16K log-likelihood 0.44
  125. 125 Qwen2.5-3B-Instruct log-likelihood, system prompt 0.44
  126. 126 Yi-34B log-likelihood 0.44
  127. 127 GPT-3.5 Turbo Jan 2024, generative-scoring, multi-turn, chat template 0.44
  128. 128 Yi-1.5-34B-Chat log-likelihood 0.44
  129. 129 Mistral-7B-Instruct-v0.3 log-likelihood 0.43
  130. 130 Olmo 3.1 32B Instruct generative-scoring, multi-turn, chat template, system prompt 0.43
  131. 131 Yi-1.5-9B generative-scoring 0.43
  132. 132 Yi-1.5-9B-32K log-likelihood 0.43
  133. 133 Yi-34B-200K log-likelihood 0.43
  134. 134 Yi-34B-Chat log-likelihood 0.42
  135. 135 Watari-7b-v1 generative-scoring, chat template, system prompt 0.42
  136. 136 Mistral NeMo generative-scoring, multi-turn, chat template, system prompt 0.42
  137. 137 ruadapt_qwen2.5_3B_ext_u48_instruct_v4 generative-scoring, multi-turn, chat template, system prompt 0.41
  138. 138 Qwen2.5-3B log-likelihood, system prompt 0.41
  139. 139 Mistral-7B-Instruct-v0.2 log-likelihood 0.41
  140. 140 Mistral 7B v0.1 log-likelihood 0.40
  141. 141 Gemma 4 E2B generative-scoring, multi-turn, chat template, system prompt 0.40
  142. 142 Yi-1.5-9B-Chat-16K log-likelihood 0.40
  143. 143 phi-3-mini 3.8B log-likelihood 0.40
  144. 144 Yi-Coder-9B-Chat log-likelihood, system prompt 0.40
  145. 145 Mistral 7B v0.3 log-likelihood 0.40
  146. 146 Phi-3-mini-128k-instruct log-likelihood 0.40
  147. 147 Qwen3-0.6B generative-scoring, multi-turn, chat template, system prompt 0.39
  148. 148 Yi-1.5-9B-Chat log-likelihood 0.39
  149. 149 Gemma 2 2B IT log-likelihood 0.39
  150. 150 Gamayun-1.5B log-likelihood 0.38
  151. 151 Llama-3.2-3B-Instruct log-likelihood, system prompt 0.38
  152. 152 Yi-Coder-9B log-likelihood, system prompt 0.37
  153. 153 Qwen1.5-7B log-likelihood 0.37
  154. 154 LFM2-24B-A2B generative-scoring, multi-turn, chat template, system prompt 0.37
  155. 155 Mistral-7B-Instruct-v0.1 log-likelihood 0.37
  156. 156 Qwen1.5-7B-Chat log-likelihood 0.37
  157. 157 Meno-Tiny-0.1 generative-scoring, multi-turn, chat template, system prompt 0.36
  158. 158 Llama 3.2 3B log-likelihood, system prompt 0.36
  159. 159 Qwen2.5 1.5B Instruct generative-scoring, multi-turn, chat template, system prompt 0.36
  160. 160 Yi-1.5-6B log-likelihood 0.36
  161. 161 Qwen2.5-1.5B-Instruct log-likelihood, system prompt 0.35
  162. 162 LFM-3B generative-scoring, multi-turn, chat template, system prompt 0.35
  163. 163 Qwen3.5-2B generative-scoring, multi-turn, chat template, system prompt 0.35
  164. 164 Qwen2.5-Math-7B-Instruct log-likelihood, system prompt 0.35
  165. 165 Llama-2-13b-hf log-likelihood 0.34
  166. 166 Qwen2-Math-7B-Instruct log-likelihood, chat template, system prompt 0.34
  167. 167 Olmo-3-7B-Instruct generative-scoring, multi-turn, chat template 0.34
  168. 168 Qwen2-1.5B-Instruct log-likelihood 0.34
  169. 169 Qwen3-1.7B generative-scoring, multi-turn, chat template, system prompt 0.34
  170. 170 Qwen1.5-MoE-A2.7B-Chat log-likelihood, multi-turn, chat template, system prompt 0.34
  171. 171 Yi-1.5-6B-Chat log-likelihood 0.33
  172. 172 Qwen2-1.5B log-likelihood 0.33
  173. 173 Qwen1.5-4B log-likelihood 0.32
  174. 174 Yi 6B generative-scoring 0.32
  175. 175 Qwen2.5-Coder (1.5B) log-likelihood, system prompt 0.32
  176. 176 gemma-2-2b log-likelihood 0.31
  177. 177 Qwen1.5-4B-Chat log-likelihood, multi-turn, chat template, system prompt 0.31
  178. 178 Qwen2.5-Math-1.5B log-likelihood, system prompt 0.31
  179. 179 Qwen2.5-Coder-1.5B-Instruct log-likelihood, system prompt 0.31
  180. 180 Yi-6B-200K log-likelihood 0.31
  181. 181 Llama-2-7b-chat-hf log-likelihood 0.30
  182. 182 Qwen3.5-0.8B generative-scoring, multi-turn, chat template, system prompt 0.30
  183. 183 Llama-2-7b-hf log-likelihood 0.30
  184. 184 Llama-3.2-1B-Instruct log-likelihood, system prompt 0.30
  185. 185 Qwen2.5-0.5B-Instruct log-likelihood, system prompt 0.29
  186. 186 Qwen2.5-Math-1.5B-Instruct log-likelihood, system prompt 0.28
  187. 187 LFM2-2.6B generative-scoring, multi-turn, chat template, system prompt 0.28
  188. 188 LFM2-8B-A1B generative-scoring, multi-turn, chat template, system prompt 0.28
  189. 189 Qwen1.5-1.8B log-likelihood 0.27
  190. 190 Qwen1.5-1.8B-Chat log-likelihood 0.26
  191. 191 Qwen2-0.5B-Instruct log-likelihood 0.26
  192. 192 Llama-2-13b-chat-hf log-likelihood, multi-turn, chat template, system prompt 0.25
  193. 193 Llama 3.2 1B log-likelihood, system prompt 0.25
  194. 194 Qwen2-0.5B log-likelihood 0.25
  195. 195 LFM2-700M generative-scoring, multi-turn, chat template, system prompt 0.24
  196. 196 Yi-6B-Chat log-likelihood, chat template 0.23
  197. 197 Megra 1.1 709M (SFT-4096) log-likelihood 0.22
  198. 198 Qwen1.5-0.5B log-likelihood 0.22
  199. 199 Qwen1.5-0.5B-Chat log-likelihood, multi-turn, chat template, system prompt 0.22
  200. 200 LFM2-350M generative-scoring, multi-turn, chat template, system prompt 0.21
  201. 201 ruGPT-3.5 log-likelihood 0.21
  202. 202 ruGPT3-medium log-likelihood 0.21
  203. 203 ruGPT3-large log-likelihood 0.20
  204. 204 ruGPT-3-small log-likelihood 0.20
  205. 205 LFM2.5-1.2B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.20
  206. 206 ruGPT-3 XL 1.3B log-likelihood 0.20
  207. 207 ruT5-base log-likelihood 0.19
  208. 208 Qwen2-Math-1.5B-Instruct log-likelihood, chat template, system prompt 0.19
  209. 209 ruGPT-3-medium generative-scoring 0.17

Source: Official MERA leaderboard

102 models. Scroll sideways for the rest.

  1. 0.86 Claude Opus 4.6
  2. 0.82 GPT-5.4
  3. 0.80 GLM-5.1
  4. 0.80 Qwen3-235B-A22B-Thinking
  5. 0.79 Qwen 3.6 35B-A3B
  6. 0.79 Qwen3.6 27B
  7. 0.78 Qwen3.5-27B
  8. 0.77 Qwen3.5-122B-A10B
  9. 0.76 Claude Opus 4.5
  10. 0.75 Qwen3-30B-A3B-Thinking
  11. 0.75 Kimi K2.5
  12. 0.73 Qwen3-32B
  13. 0.73 Qwen3.5-35B-A3B
  14. 0.71 Qwen3-Next-80B-A3B
  15. 0.71 Gemma 4 31B IT
  16. 0.71 Qwen3-4B-Thinking-2507
  17. 0.71 GPT-5.2
  18. 0.69 MiniMax-M2.7
  19. 0.69 Qwen3-14B
  20. 0.69 DeepSeek-V4-Pro
  21. 0.68 Claude 3.7 Sonnet
  22. 0.68 Gemini 2.0 Flash
  23. 0.68 Qwen3.5-9B
  24. 0.68 DeepSeek-V3
  25. 0.68 DeepSeek-V4-Flash
  26. 0.68 Gemini 1.5 Pro
  27. 0.67 Qwen3-4B
  28. 0.66 Qwen2.5-Max
  29. 0.65 Qwen3-8B
  30. 0.64 GPT-4o
  31. 0.64 Gemini 2.0 Flash-Lite
  32. 0.63 Gemini 1.5 Flash
  33. 0.62 Qwen2.5-72B
  34. 0.62 DeepSeek-R1-Distill-Qwen-32B
  35. 0.62 gpt-oss-20b
  36. 0.61 Llama 4 Maverick
  37. 0.60 Gemma 4 26B A4B
  38. 0.57 Llama 3.1-70B
  39. 0.57 Qwen2-72B
  40. 0.57 GPT-4o mini
  41. 0.57 Gemma 3 27B
  42. 0.57 Qwen3.5-4B
  43. 0.56 Llama 3-70B
  44. 0.56 Llama 3.3 70B Instruct
  45. 0.55 Qwen-Turbo
  46. 0.55 Claude 3.5 Haiku
  47. 0.54 Gemini 1.5 Flash 8B
  48. 0.53 Llama-3.1-Nemotron-70B-Instruct
  49. 0.53 Llama 4 Scout
  50. 0.52 Gemma 2 27B
  51. 0.51 phi-3-medium 14B
  52. 0.51 Phi-3-medium-128k-instruct
  53. 0.51 Qwen2.5-Math-72B-Instruct
  54. 0.50 Phi-3.5-MoE
  55. 0.49 Mixtral 8x7B
  56. 0.49 Qwen3-30B-A3B-Instruct
  57. 0.48 Qwen1.5-32B
  58. 0.48 Gemma 4 E4B
  59. 0.47 Qwen2-7B-Instruct
  60. 0.47 Yi-1.5-34B
  61. 0.47 qwen3-4b-instruct-2507
  62. 0.47 phi-3-small 7.4B
  63. 0.46 Qwen2.5-7B
  64. 0.46 Command R (03-2024)
  65. 0.46 Qwen2-7B
  66. 0.45 Gemma 2 9B
  67. 0.45 Llama 3.1-8B
  68. 0.45 DeepSeek-R1-Distill-Qwen-14B
  69. 0.45 Llama 3-8B
  70. 0.44 Yi-34B
  71. 0.44 GPT-3.5 Turbo
  72. 0.43 Olmo 3.1 32B Instruct
  73. 0.42 Mistral NeMo
  74. 0.41 Qwen2.5-3B
  75. 0.41 Mistral-7B-Instruct-v0.2
  76. 0.40 Mistral 7B v0.1
  77. 0.40 Gemma 4 E2B
  78. 0.40 phi-3-mini 3.8B
  79. 0.40 Mistral 7B v0.3
  80. 0.40 Phi-3-mini-128k-instruct
  81. 0.39 Qwen3-0.6B
  82. 0.39 Yi-1.5-9B-Chat
  83. 0.39 Gemma 2 2B IT
  84. 0.37 Qwen1.5-7B
  85. 0.37 Mistral-7B-Instruct-v0.1
  86. 0.37 Qwen1.5-7B-Chat
  87. 0.36 Llama 3.2 3B
  88. 0.35 Qwen3.5-2B
  89. 0.34 Qwen2-1.5B-Instruct
  90. 0.34 Qwen3-1.7B
  91. 0.33 Yi-1.5-6B-Chat
  92. 0.33 Qwen2-1.5B
  93. 0.32 Yi 6B
  94. 0.32 Qwen2.5-Coder (1.5B)
  95. 0.31 Qwen1.5-4B-Chat
  96. 0.30 Qwen3.5-0.8B
  97. 0.28 LFM2-8B-A1B
  98. 0.26 Qwen2-0.5B-Instruct
  99. 0.25 Llama 3.2 1B
  100. 0.25 Qwen2-0.5B
  101. 0.23 Yi-6B-Chat
  102. 0.20 LFM2.5-1.2B-Instruct
  1. 1 Claude Opus 4.6 generative-scoring, multi-turn, chat template, system prompt 0.86
  2. 2 GPT-5.4 generative-scoring, multi-turn, chat template, system prompt 0.82
  3. 3 GLM-5.1 generative-scoring, multi-turn, chat template, system prompt 0.80
  4. 4 Qwen3-235B-A22B-Thinking Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.80
  5. 5 Qwen 3.6 35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.79
  6. 6 Qwen3.6 27B generative-scoring, multi-turn, chat template, system prompt 0.79
  7. 7 Qwen3.5-27B generative-scoring, multi-turn, chat template, system prompt 0.78
  8. 8 Qwen3.5-122B-A10B generative-scoring, multi-turn, chat template, system prompt 0.77
  9. 9 Claude Opus 4.5 generative-scoring, multi-turn, chat template, system prompt 0.76
  10. 10 Qwen3-30B-A3B-Thinking Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.75
  11. 11 Kimi K2.5 generative-scoring, multi-turn, chat template, system prompt 0.75
  12. 12 Qwen3-32B generative-scoring, multi-turn, chat template, system prompt 0.73
  13. 13 Qwen3.5-35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.73
  14. 14 Qwen3-Next-80B-A3B generative-scoring, multi-turn, chat template, system prompt 0.71
  15. 15 Gemma 4 31B IT generative-scoring, multi-turn, chat template, system prompt 0.71
  16. 16 Qwen3-4B-Thinking-2507 generative-scoring, multi-turn, chat template, system prompt 0.71
  17. 17 GPT-5.2 generative-scoring, multi-turn, chat template, system prompt 0.71
  18. 18 MiniMax-M2.7 generative-scoring, multi-turn 0.69
  19. 19 Qwen3-14B generative-scoring, multi-turn, chat template, system prompt 0.69
  20. 20 DeepSeek-V4-Pro generative-scoring, multi-turn, chat template, system prompt 0.69
  21. 21 Claude 3.7 Sonnet generative-scoring, multi-turn, chat template, system prompt 0.68
  22. 22 Gemini 2.0 Flash Feb 2025, generative-scoring, multi-turn, chat template, system prompt 0.68
  23. 23 Qwen3.5-9B generative-scoring, multi-turn, chat template, system prompt 0.68
  24. 24 DeepSeek-V3 Mar 2025, generative-scoring, multi-turn, chat template, system prompt 0.68
  25. 25 DeepSeek-V4-Flash generative-scoring, multi-turn, chat template, system prompt 0.68
  26. 26 Gemini 1.5 Pro Sep 2024, generative-scoring, multi-turn, chat template, system prompt 0.68
  27. 27 Qwen3-4B generative-scoring, multi-turn, chat template, system prompt 0.67
  28. 28 Qwen2.5-Max generative-scoring, multi-turn, chat template, system prompt 0.66
  29. 29 Qwen3-8B generative-scoring, multi-turn, chat template, system prompt 0.65
  30. 30 GPT-4o Mar 2025, generative-scoring, multi-turn, chat template, system prompt 0.64
  31. 31 Gemini 2.0 Flash-Lite generative-scoring, multi-turn, chat template, system prompt 0.64
  32. 32 Gemini 1.5 Flash Sep 2024, generative-scoring, multi-turn, chat template, system prompt 0.63
  33. 33 Qwen2.5-72B log-likelihood, system prompt 0.62
  34. 34 DeepSeek-R1-Distill-Qwen-32B generative-scoring, chat template, system prompt 0.62
  35. 35 gpt-oss-20b generative-scoring, multi-turn, chat template, system prompt 0.62
  36. 36 Llama 4 Maverick generative-scoring, multi-turn, chat template, system prompt 0.61
  37. 37 Gemma 4 26B A4B generative-scoring, multi-turn, chat template, system prompt 0.60
  38. 38 Llama 3.1-70B log-likelihood 0.57
  39. 39 Qwen2-72B generative-scoring, multi-turn, chat template, system prompt 0.57
  40. 40 GPT-4o mini generative-scoring, multi-turn, chat template, system prompt 0.57
  41. 41 Gemma 3 27B generative-scoring, multi-turn, chat template, system prompt 0.57
  42. 42 Qwen3.5-4B generative-scoring, multi-turn, chat template, system prompt 0.57
  43. 43 Llama 3-70B generative-scoring 0.56
  44. 44 Llama 3.3 70B Instruct generative-scoring, multi-turn, chat template, system prompt 0.56
  45. 45 Qwen-Turbo generative-scoring, multi-turn, chat template, system prompt 0.55
  46. 46 Claude 3.5 Haiku generative-scoring, multi-turn, chat template, system prompt 0.55
  47. 47 Gemini 1.5 Flash 8B generative-scoring, multi-turn, chat template, system prompt 0.54
  48. 48 Llama-3.1-Nemotron-70B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.53
  49. 49 Llama 4 Scout generative-scoring, multi-turn, chat template, system prompt 0.53
  50. 50 Gemma 2 27B log-likelihood 0.52
  51. 51 phi-3-medium 14B log-likelihood 0.51
  52. 52 Phi-3-medium-128k-instruct log-likelihood 0.51
  53. 53 Qwen2.5-Math-72B-Instruct log-likelihood, system prompt 0.51
  54. 54 Phi-3.5-MoE log-likelihood 0.50
  55. 55 Mixtral 8x7B log-likelihood 0.49
  56. 56 Qwen3-30B-A3B-Instruct Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.49
  57. 57 Qwen1.5-32B log-likelihood 0.48
  58. 58 Gemma 4 E4B generative-scoring, multi-turn, chat template, system prompt 0.48
  59. 59 Qwen2-7B-Instruct log-likelihood 0.47
  60. 60 Yi-1.5-34B log-likelihood 0.47
  61. 61 qwen3-4b-instruct-2507 generative-scoring, multi-turn, chat template, system prompt 0.47
  62. 62 phi-3-small 7.4B log-likelihood 0.47
  63. 63 Qwen2.5-7B log-likelihood, system prompt 0.46
  64. 64 Command R (03-2024) generative-scoring, multi-turn, chat template, system prompt 0.46
  65. 65 Qwen2-7B log-likelihood 0.46
  66. 66 Gemma 2 9B log-likelihood 0.45
  67. 67 Llama 3.1-8B log-likelihood 0.45
  68. 68 DeepSeek-R1-Distill-Qwen-14B log-likelihood 0.45
  69. 69 Llama 3-8B generative-scoring 0.45
  70. 70 Yi-34B log-likelihood 0.44
  71. 71 GPT-3.5 Turbo Jan 2024, generative-scoring, multi-turn, chat template 0.44
  72. 72 Olmo 3.1 32B Instruct generative-scoring, multi-turn, chat template, system prompt 0.43
  73. 73 Mistral NeMo generative-scoring, multi-turn, chat template, system prompt 0.42
  74. 74 Qwen2.5-3B log-likelihood, system prompt 0.41
  75. 75 Mistral-7B-Instruct-v0.2 log-likelihood 0.41
  76. 76 Mistral 7B v0.1 log-likelihood 0.40
  77. 77 Gemma 4 E2B generative-scoring, multi-turn, chat template, system prompt 0.40
  78. 78 phi-3-mini 3.8B log-likelihood 0.40
  79. 79 Mistral 7B v0.3 log-likelihood 0.40
  80. 80 Phi-3-mini-128k-instruct log-likelihood 0.40
  81. 81 Qwen3-0.6B generative-scoring, multi-turn, chat template, system prompt 0.39
  82. 82 Yi-1.5-9B-Chat log-likelihood 0.39
  83. 83 Gemma 2 2B IT log-likelihood 0.39
  84. 84 Qwen1.5-7B log-likelihood 0.37
  85. 85 Mistral-7B-Instruct-v0.1 log-likelihood 0.37
  86. 86 Qwen1.5-7B-Chat log-likelihood 0.37
  87. 87 Llama 3.2 3B log-likelihood, system prompt 0.36
  88. 88 Qwen3.5-2B generative-scoring, multi-turn, chat template, system prompt 0.35
  89. 89 Qwen2-1.5B-Instruct log-likelihood 0.34
  90. 90 Qwen3-1.7B generative-scoring, multi-turn, chat template, system prompt 0.34
  91. 91 Yi-1.5-6B-Chat log-likelihood 0.33
  92. 92 Qwen2-1.5B log-likelihood 0.33
  93. 93 Yi 6B generative-scoring 0.32
  94. 94 Qwen2.5-Coder (1.5B) log-likelihood, system prompt 0.32
  95. 95 Qwen1.5-4B-Chat log-likelihood, multi-turn, chat template, system prompt 0.31
  96. 96 Qwen3.5-0.8B generative-scoring, multi-turn, chat template, system prompt 0.30
  97. 97 LFM2-8B-A1B generative-scoring, multi-turn, chat template, system prompt 0.28
  98. 98 Qwen2-0.5B-Instruct log-likelihood 0.26
  99. 99 Llama 3.2 1B log-likelihood, system prompt 0.25
  100. 100 Qwen2-0.5B log-likelihood 0.25
  101. 101 Yi-6B-Chat log-likelihood, chat template 0.23
  102. 102 LFM2.5-1.2B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.20

Source: Official MERA leaderboard

#EntryScoreSettingSubmitted byDateLinkSource
1 Claude Opus 4.6 0.86 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
2 BerryLM-XL closed weights, 358B 0.83 generative-scoring, multi-turn, chat template, system prompt Wildberries & Russ mera.a-ai.ru Official leaderboard
3 Cotype Pro 3 closed weights, 27B 0.82 generative-scoring, multi-turn, chat template, system prompt MWS AI mera.a-ai.ru Official leaderboard
4 BerryLM-L-v2-early-ckpt closed weights, 0B 0.82 generative-scoring, multi-turn, system prompt Wildberrie&Russ mera.a-ai.ru Official leaderboard
5 GPT-5.4 0.82 generative-scoring, multi-turn, chat template, system prompt MERA Mar 2026 mera.a-ai.ru Official leaderboard
6 BerryLM-v2-reasoning-budget-low closed weights, 30B 0.81 generative-scoring, multi-turn, chat template, system prompt Wildberries & Russ mera.a-ai.ru Official leaderboard
7 GLM-5.1 0.80 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
8 Qwen3-235B-A22B-Thinking Jul 2025 0.80 generative-scoring, multi-turn, chat template, system prompt MERA Jul 2025 mera.a-ai.ru Official leaderboard
9 Cotype Light 3 closed weights, 9B 0.79 generative-scoring, multi-turn, chat template, system prompt MWS AI mera.a-ai.ru Official leaderboard
10 Qwen 3.6 35B-A3B 0.79 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
11 Qwen3.6 27B 0.79 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
12 Qwen3.5-27B 0.78 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
13 BerryLM-L closed weights, 120B 0.78 generative-scoring, multi-turn, chat template, system prompt Wildberries & Russ mera.a-ai.ru Official leaderboard
14 Qwen3.5-122B-A10B 0.77 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
15 Claude Opus 4.5 0.76 generative-scoring, multi-turn, chat template, system prompt Claude Opus 4.5 Nov 2025 mera.a-ai.ru Official leaderboard
16 Qwen3-30B-A3B-Thinking Jul 2025 0.75 generative-scoring, multi-turn, chat template, system prompt MERA Jul 2025 mera.a-ai.ru Official leaderboard
17 Kimi K2.5 0.75 generative-scoring, multi-turn, chat template, system prompt MERA Jan 2026 mera.a-ai.ru Official leaderboard
18 BerryLM-MT closed weights, 20B 0.74 generative-scoring, multi-turn, chat template, system prompt Wildberries & Russ mera.a-ai.ru Official leaderboard
19 BerryLM closed weights, 20B 0.74 generative-scoring, chat template, system prompt Wildberries & Russ mera.a-ai.ru Official leaderboard
20 Qwen3-32B 0.73 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
21 Qwen3.5-35B-A3B 0.73 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
22 GigaChat-3.1-Ultra open weights, 702B 0.71 generative-scoring, multi-turn, chat template, system prompt GigaChat mera.a-ai.ru Official leaderboard
23 Qwen3-Next-80B-A3B 0.71 generative-scoring, multi-turn, chat template, system prompt MERA Sep 2025 mera.a-ai.ru Official leaderboard
24 Gemma 4 31B IT 0.71 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
25 Qwen3-4B-Thinking-2507 0.71 generative-scoring, multi-turn, chat template, system prompt MERA Aug 2025 mera.a-ai.ru Official leaderboard
26 GPT-5.2 0.71 generative-scoring, multi-turn, chat template, system prompt GPT-5.2 Dec 2025 mera.a-ai.ru Official leaderboard
27 MiniMax-M2.7 0.69 generative-scoring, multi-turn MERA Mar 2026 mera.a-ai.ru Official leaderboard
28 Qwen3-14B 0.69 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
29 DeepSeek-V4-Pro 0.69 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
30 Claude 3.7 Sonnet 0.68 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Feb 2025 mera.a-ai.ru Official leaderboard
31 Gemini 2.0 Flash Feb 2025 0.68 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Feb 2025 mera.a-ai.ru Official leaderboard
32 Qwen3.5-9B 0.68 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
33 DeepSeek-V3 Mar 2025 0.68 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Mar 2025 mera.a-ai.ru Official leaderboard
34 DeepSeek-V4-Flash 0.68 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
35 Gemini 1.5 Pro Sep 2024 0.68 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Sep 2024 mera.a-ai.ru Official leaderboard
36 Cotype Pro 2.5 closed weights, 32.5B 0.67 generative-scoring, multi-turn, chat template, system prompt MWS AI mera.a-ai.ru Official leaderboard
37 Qwen3-4B 0.67 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
38 GigaChat 2 Max closed weights 0.67 generative-scoring, multi-turn, chat template, system prompt GIGACHAT mera.a-ai.ru Official leaderboard
39 Qwen2.5-Max 0.66 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Jan 2025 mera.a-ai.ru Official leaderboard
40 T-pro-it-2.0 open weights, 32.8B 0.66 generative-scoring, multi-turn, chat template, system prompt T-Tech mera.a-ai.ru Official leaderboard
41 GigaChat 2 Pro closed weights 0.65 generative-scoring, multi-turn, chat template, system prompt GIGACHAT mera.a-ai.ru Official leaderboard
42 Qwen3-8B 0.65 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
43 GPT-4o Mar 2025 0.64 generative-scoring, multi-turn, chat template, system prompt MERA Mar 2025 mera.a-ai.ru Official leaderboard
44 Gemini 2.0 Flash-Lite 0.64 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Feb 2025 mera.a-ai.ru Official leaderboard
45 Meta-Llama-3.1-405B-Instruct open weights, 405B 0.64 log-likelihood MERA mera.a-ai.ru Official leaderboard
46 Cotype (preview) closed weights 0.63 generative-scoring, multi-turn, chat template, system prompt MTS AI mera.a-ai.ru Official leaderboard
47 T-pro-it-1.0 open weights, 32B 0.63 generative-scoring, multi-turn, chat template, system prompt T-Tech mera.a-ai.ru Official leaderboard
48 Gemini 1.5 Flash Sep 2024 0.63 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Aug 2024 mera.a-ai.ru Official leaderboard
49 Zero-Mistral-24B open weights, 23.6B 0.62 generative-scoring, multi-turn, chat template, system prompt ZeroAgency mera.a-ai.ru Official leaderboard
50 Qwen2.5-72B 0.62 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
51 A-vibe open weights, 8B 0.62 generative-scoring, multi-turn, chat template, system prompt Avito mera.a-ai.ru Official leaderboard
52 DeepSeek-R1-Distill-Qwen-32B 0.62 generative-scoring, chat template, system prompt НГУ Jan 2025 mera.a-ai.ru Official leaderboard
53 gpt-oss-20b 0.62 generative-scoring, multi-turn, chat template, system prompt openai Aug 2025 mera.a-ai.ru Official leaderboard
54 Nova Pro 1.0 closed weights 0.61 generative-scoring, multi-turn, chat template, system prompt llmarena.ru mera.a-ai.ru Official leaderboard
55 RuadaptQwen-32B-instruct open weights, 32B 0.61 generative-scoring, multi-turn, chat template, system prompt RCC MSU mera.a-ai.ru Official leaderboard
56 Llama 4 Maverick 0.61 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Apr 2025 mera.a-ai.ru Official leaderboard
57 Gemma 4 26B A4B 0.60 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
58 Qwen2.5-32B-Instruct open weights, 32B 0.60 generative-scoring, multi-turn, chat template, system prompt RCC MSU mera.a-ai.ru Official leaderboard
59 Supernova-N1-32B-Preview closed weights, 32B 0.60 generative-scoring, multi-turn, chat template, system prompt Nova mera.a-ai.ru Official leaderboard
60 Meta-Llama-3.1-70B-Instruct open weights, 70.6B 0.60 log-likelihood MERA mera.a-ai.ru Official leaderboard
61 Mixtral-8x22B-Instruct-v0.1 open weights, 140.6B 0.59 log-likelihood MERA mera.a-ai.ru Official leaderboard
62 Mistral-Large-Instruct-2407 open weights, 123B 0.59 log-likelihood, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
63 GigaChat Max closed weights 0.59 generative-scoring, multi-turn, chat template, system prompt GIGACHAT mera.a-ai.ru Official leaderboard
64 Meta-Llama-3-70B-Instruct open weights, 70.6B 0.59 log-likelihood MERA mera.a-ai.ru Official leaderboard
65 gemma-2-27b-it open weights, 27.2B 0.58 log-likelihood MERA mera.a-ai.ru Official leaderboard
66 Llama 3.1-70B 0.57 log-likelihood MERA Jul 2024 mera.a-ai.ru Official leaderboard
67 Qwen2-72B 0.57 generative-scoring, multi-turn, chat template, system prompt MERA Jun 2024 mera.a-ai.ru Official leaderboard
68 GPT-4o mini 0.57 generative-scoring, multi-turn, chat template, system prompt MERA Jul 2024 mera.a-ai.ru Official leaderboard
69 Gemma 3 27B 0.57 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Mar 2025 mera.a-ai.ru Official leaderboard
70 Qwen3.5-4B 0.57 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
71 Llama 3-70B 0.56 generative-scoring MERA Apr 2024 mera.a-ai.ru Official leaderboard
72 Meno-Lite-0.1 open weights, 7B 0.56 generative-scoring, multi-turn, chat template, system prompt НГУ mera.a-ai.ru Official leaderboard
73 Llama 3.3 70B Instruct 0.56 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Dec 2024 mera.a-ai.ru Official leaderboard
74 T-lite-it-1.0 open weights, 7B 0.55 generative-scoring, multi-turn, chat template, system prompt T-Tech mera.a-ai.ru Official leaderboard
75 Qwen-Turbo 0.55 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Nov 2024 mera.a-ai.ru Official leaderboard
76 Claude 3.5 Haiku 0.55 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Oct 2024 mera.a-ai.ru Official leaderboard
77 GigaChat 2 closed weights 0.54 generative-scoring, multi-turn, chat template, system prompt GIGACHAT mera.a-ai.ru Official leaderboard
78 Gemini 1.5 Flash 8B 0.54 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Oct 2024 mera.a-ai.ru Official leaderboard
79 RuadaptQwen2.5-7B-Lite-v1 open weights, 7B 0.54 generative-scoring, multi-turn, chat template, system prompt RCC MSU mera.a-ai.ru Official leaderboard
80 Qwen2.5 14B Instruct open weights, 14.8B 0.53 generative-scoring, chat template, system prompt НГУ mera.a-ai.ru Official leaderboard
81 Mistral Small 3.1 24B Instruct open weights, 0B 0.53 generative-scoring, multi-turn, chat template, system prompt llmarena.ru mera.a-ai.ru Official leaderboard
82 Llama-3.1-Nemotron-70B-Instruct 0.53 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Oct 2024 mera.a-ai.ru Official leaderboard
83 Llama 4 Scout 0.53 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Apr 2025 mera.a-ai.ru Official leaderboard
84 gemma-2-9b-it open weights, 9.2B 0.52 log-likelihood MERA mera.a-ai.ru Official leaderboard
85 Zero-Mistral-Small-24B-Instruct-2501 open weights, 0B 0.52 generative-scoring, multi-turn, chat template, system prompt ZeroAgency mera.a-ai.ru Official leaderboard
86 Gemma 2 27B 0.52 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
87 Qwen2-57B-A14B-Instruct open weights, 57.4B 0.52 log-likelihood, multi-turn, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
88 Mixtral-8x7B-Instruct-v0.1 open weights, 46.7B 0.52 log-likelihood MERA mera.a-ai.ru Official leaderboard
89 GigaChat-20B-A3B open weights, 20B 0.51 generative-scoring, multi-turn, chat template GIGACHAT mera.a-ai.ru Official leaderboard
90 GigaChat Pro closed weights 0.51 generative-scoring, multi-turn, chat template, system prompt GIGACHAT mera.a-ai.ru Official leaderboard
91 T-lite-0.1 open weights, 8B 0.51 log-likelihood Т-Банк mera.a-ai.ru Official leaderboard
92 phi-3-medium 14B 0.51 log-likelihood MERA Apr 2024 mera.a-ai.ru Official leaderboard
93 Phi-3-medium-128k-instruct 0.51 log-likelihood MERA May 2024 mera.a-ai.ru Official leaderboard
94 Qwen2.5-Math-72B-Instruct 0.51 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
95 Qwen2.5-Math-72B open weights, 72.7B 0.51 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
96 GigaChat-3.1-Lightning open weights, 11B 0.50 generative-scoring, multi-turn, chat template, system prompt GigaChat mera.a-ai.ru Official leaderboard
97 Phi-3.5-MoE 0.50 log-likelihood MERA Aug 2024 mera.a-ai.ru Official leaderboard
98 GigaChat closed weights 0.50 generative-scoring, multi-turn, chat template, system prompt GIGACHAT mera.a-ai.ru Official leaderboard
99 Qwen2-57B-A14B open weights, 57.4B 0.49 log-likelihood MERA mera.a-ai.ru Official leaderboard
100 Qwen2.5-7B-Instruct open weights, 7B 0.49 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
101 Mixtral 8x7B 0.49 log-likelihood MERA Dec 2023 mera.a-ai.ru Official leaderboard
102 Qwen3-30B-A3B-Instruct Jul 2025 0.49 generative-scoring, multi-turn, chat template, system prompt MERA Jul 2025 mera.a-ai.ru Official leaderboard
103 Meta-Llama-3.1-8B-Instruct open weights, 8B 0.49 log-likelihood MERA mera.a-ai.ru Official leaderboard
104 Qwen1.5-32B-Chat open weights, 32.5B 0.49 log-likelihood, multi-turn, chat template MERA mera.a-ai.ru Official leaderboard
105 Qwen1.5-32B 0.48 log-likelihood MERA Feb 2024 mera.a-ai.ru Official leaderboard
106 Gemma 4 E4B 0.48 generative-scoring, multi-turn, chat template, system prompt MERA Mar 2026 mera.a-ai.ru Official leaderboard
107 Yi-1.5-34B-32K open weights, 34.4B 0.48 log-likelihood MERA mera.a-ai.ru Official leaderboard
108 Meta-Llama-3-8B-Instruct open weights, 8B 0.48 log-likelihood MERA mera.a-ai.ru Official leaderboard
109 Qwen2-7B-Instruct 0.47 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
110 Olmo-3.1-32B-Instruct-DPO open weights, 32B 0.47 generative-scoring, multi-turn, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
111 Yi-1.5-34B 0.47 log-likelihood MERA May 2024 mera.a-ai.ru Official leaderboard
112 qwen3-4b-instruct-2507 0.47 generative-scoring, multi-turn, chat template, system prompt MERA Aug 2025 mera.a-ai.ru Official leaderboard
113 phi-3-small 7.4B 0.47 log-likelihood MERA Apr 2024 mera.a-ai.ru Official leaderboard
114 Qwen2.5-Coder-7B-Instruct open weights, 7B 0.46 log-likelihood, multi-turn, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
115 Olmo-3.1-32B-Instruct-SFT open weights, 32B 0.46 generative-scoring, multi-turn, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
116 Qwen2.5-7B 0.46 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
117 Command R (03-2024) 0.46 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Mar 2024 mera.a-ai.ru Official leaderboard
118 Qwen2-7B 0.46 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
119 Gemma 2 9B 0.45 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
120 ruadapt llama3-8B-instruct lep ft open weights, 8.4B 0.45 log-likelihood, multi-turn, chat template, system prompt RCC MSU mera.a-ai.ru Official leaderboard
121 Llama 3.1-8B 0.45 log-likelihood MERA Jul 2024 mera.a-ai.ru Official leaderboard
122 DeepSeek-R1-Distill-Qwen-14B 0.45 log-likelihood НГУ Jan 2025 mera.a-ai.ru Official leaderboard
123 Llama 3-8B 0.45 generative-scoring MERA Apr 2024 mera.a-ai.ru Official leaderboard
124 Yi-1.5-34B-Chat-16K open weights, 34.4B 0.44 log-likelihood MERA mera.a-ai.ru Official leaderboard
125 Qwen2.5-3B-Instruct open weights, 3B 0.44 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
126 Yi-34B 0.44 log-likelihood MERA Nov 2023 mera.a-ai.ru Official leaderboard
127 GPT-3.5 Turbo Jan 2024 0.44 generative-scoring, multi-turn, chat template MERA Jan 2024 mera.a-ai.ru Official leaderboard
128 Yi-1.5-34B-Chat open weights, 34.4B 0.44 log-likelihood MERA mera.a-ai.ru Official leaderboard
129 Mistral-7B-Instruct-v0.3 open weights, 7.2B 0.43 log-likelihood MERA mera.a-ai.ru Official leaderboard
130 Olmo 3.1 32B Instruct 0.43 generative-scoring, multi-turn, chat template, system prompt MERA Dec 2025 mera.a-ai.ru Official leaderboard
131 Yi-1.5-9B open weights, 8.8B 0.43 generative-scoring MERA mera.a-ai.ru Official leaderboard
132 Yi-1.5-9B-32K open weights, 8.8B 0.43 log-likelihood MERA mera.a-ai.ru Official leaderboard
133 Yi-34B-200K open weights, 34.4B 0.43 log-likelihood MERA mera.a-ai.ru Official leaderboard
134 Yi-34B-Chat open weights, 34.4B 0.42 log-likelihood MERA mera.a-ai.ru Official leaderboard
135 Watari-7b-v1 open weights, 7.6B 0.42 generative-scoring, chat template, system prompt Attention Signs mera.a-ai.ru Official leaderboard
136 Mistral NeMo 0.42 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Jul 2024 mera.a-ai.ru Official leaderboard
137 ruadapt_qwen2.5_3B_ext_u48_instruct_v4 open weights, 3B 0.41 generative-scoring, multi-turn, chat template, system prompt RCC MSU mera.a-ai.ru Official leaderboard
138 Qwen2.5-3B 0.41 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
139 Mistral-7B-Instruct-v0.2 0.41 log-likelihood MERA Dec 2023 mera.a-ai.ru Official leaderboard
140 Mistral 7B v0.1 0.40 log-likelihood MERA Sep 2023 mera.a-ai.ru Official leaderboard
141 Gemma 4 E2B 0.40 generative-scoring, multi-turn, chat template, system prompt MERA Mar 2026 mera.a-ai.ru Official leaderboard
142 Yi-1.5-9B-Chat-16K open weights, 8.8B 0.40 log-likelihood MERA mera.a-ai.ru Official leaderboard
143 phi-3-mini 3.8B 0.40 log-likelihood MERA Apr 2024 mera.a-ai.ru Official leaderboard
144 Yi-Coder-9B-Chat open weights, 9B 0.40 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
145 Mistral 7B v0.3 0.40 log-likelihood MERA Sep 2023 mera.a-ai.ru Official leaderboard
146 Phi-3-mini-128k-instruct 0.40 log-likelihood MERA Apr 2024 mera.a-ai.ru Official leaderboard
147 Qwen3-0.6B 0.39 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
148 Yi-1.5-9B-Chat 0.39 log-likelihood MERA May 2024 mera.a-ai.ru Official leaderboard
149 Gemma 2 2B IT 0.39 log-likelihood MERA Jul 2024 mera.a-ai.ru Official leaderboard
150 Gamayun-1.5B closed weights, 1.5B 0.38 log-likelihood Gamayun Team mera.a-ai.ru Official leaderboard
151 Llama-3.2-3B-Instruct open weights, 3B 0.38 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
152 Yi-Coder-9B open weights, 9B 0.37 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
153 Qwen1.5-7B 0.37 log-likelihood MERA Feb 2024 mera.a-ai.ru Official leaderboard
154 LFM2-24B-A2B open weights, 24B 0.37 generative-scoring, multi-turn, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
155 Mistral-7B-Instruct-v0.1 0.37 log-likelihood MERA Sep 2023 mera.a-ai.ru Official leaderboard
156 Qwen1.5-7B-Chat 0.37 log-likelihood MERA Feb 2024 mera.a-ai.ru Official leaderboard
157 Meno-Tiny-0.1 open weights, 1.5B 0.36 generative-scoring, multi-turn, chat template, system prompt НГУ mera.a-ai.ru Official leaderboard
158 Llama 3.2 3B 0.36 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
159 Qwen2.5 1.5B Instruct open weights, 1.5B 0.36 generative-scoring, multi-turn, chat template, system prompt НГУ mera.a-ai.ru Official leaderboard
160 Yi-1.5-6B open weights, 6.1B 0.36 log-likelihood MERA mera.a-ai.ru Official leaderboard
161 Qwen2.5-1.5B-Instruct open weights, 1.5B 0.35 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
162 LFM-3B closed weights, 3B 0.35 generative-scoring, multi-turn, chat template, system prompt llmarena.ru mera.a-ai.ru Official leaderboard
163 Qwen3.5-2B 0.35 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
164 Qwen2.5-Math-7B-Instruct open weights, 7B 0.35 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
165 Llama-2-13b-hf open weights, 13B 0.34 log-likelihood MERA mera.a-ai.ru Official leaderboard
166 Qwen2-Math-7B-Instruct open weights 0.34 log-likelihood, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
167 Olmo-3-7B-Instruct open weights, 7B 0.34 generative-scoring, multi-turn, chat template MERA mera.a-ai.ru Official leaderboard
168 Qwen2-1.5B-Instruct 0.34 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
169 Qwen3-1.7B 0.34 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
170 Qwen1.5-MoE-A2.7B-Chat open weights, 14.3B 0.34 log-likelihood, multi-turn, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
171 Yi-1.5-6B-Chat 0.33 log-likelihood MERA May 2024 mera.a-ai.ru Official leaderboard
172 Qwen2-1.5B 0.33 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
173 Qwen1.5-4B open weights, 4B 0.32 log-likelihood MERA mera.a-ai.ru Official leaderboard
174 Yi 6B 0.32 generative-scoring MERA Nov 2023 mera.a-ai.ru Official leaderboard
175 Qwen2.5-Coder (1.5B) 0.32 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
176 gemma-2-2b open weights, 2.6B 0.31 log-likelihood MERA mera.a-ai.ru Official leaderboard
177 Qwen1.5-4B-Chat 0.31 log-likelihood, multi-turn, chat template, system prompt MERA Feb 2024 mera.a-ai.ru Official leaderboard
178 Qwen2.5-Math-1.5B open weights, 1.5B 0.31 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
179 Qwen2.5-Coder-1.5B-Instruct open weights, 1.5B 0.31 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
180 Yi-6B-200K open weights, 6.1B 0.31 log-likelihood MERA mera.a-ai.ru Official leaderboard
181 Llama-2-7b-chat-hf open weights, 6.7B 0.30 log-likelihood MERA mera.a-ai.ru Official leaderboard
182 Qwen3.5-0.8B 0.30 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
183 Llama-2-7b-hf open weights, 6.7B 0.30 log-likelihood MERA mera.a-ai.ru Official leaderboard
184 Llama-3.2-1B-Instruct open weights, 1B 0.30 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
185 Qwen2.5-0.5B-Instruct open weights, 0.5B 0.29 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
186 Qwen2.5-Math-1.5B-Instruct open weights, 1.5B 0.28 log-likelihood, system prompt MERA mera.a-ai.ru Official leaderboard
187 LFM2-2.6B open weights, 2.6B 0.28 generative-scoring, multi-turn, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
188 LFM2-8B-A1B 0.28 generative-scoring, multi-turn, chat template, system prompt MERA Oct 2025 mera.a-ai.ru Official leaderboard
189 Qwen1.5-1.8B open weights, 1.8B 0.27 log-likelihood MERA mera.a-ai.ru Official leaderboard
190 Qwen1.5-1.8B-Chat open weights, 1.8B 0.26 log-likelihood MERA mera.a-ai.ru Official leaderboard
191 Qwen2-0.5B-Instruct 0.26 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
192 Llama-2-13b-chat-hf open weights, 13B 0.25 log-likelihood, multi-turn, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
193 Llama 3.2 1B 0.25 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
194 Qwen2-0.5B 0.25 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
195 LFM2-700M open weights, 0.7B 0.24 generative-scoring, multi-turn, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
196 Yi-6B-Chat 0.23 log-likelihood, chat template MERA Nov 2023 mera.a-ai.ru Official leaderboard
197 Megra 1.1 709M (SFT-4096) closed weights, 0.7B 0.22 log-likelihood Promezis Lab mera.a-ai.ru Official leaderboard
198 Qwen1.5-0.5B open weights, 0.5B 0.22 log-likelihood MERA mera.a-ai.ru Official leaderboard
199 Qwen1.5-0.5B-Chat open weights, 0.5B 0.22 log-likelihood, multi-turn, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
200 LFM2-350M open weights, 0.4B 0.21 generative-scoring, multi-turn, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
201 ruGPT-3.5 open weights, 12.9B 0.21 log-likelihood RussianNLP mera.a-ai.ru Official leaderboard
202 ruGPT3-medium open weights, 0.4B 0.21 log-likelihood RussianNLP mera.a-ai.ru Official leaderboard
203 ruGPT3-large open weights, 0.8B 0.20 log-likelihood RussianNLP mera.a-ai.ru Official leaderboard
204 ruGPT-3-small open weights, 0.1B 0.20 log-likelihood RussianNLP mera.a-ai.ru Official leaderboard
205 LFM2.5-1.2B-Instruct 0.20 generative-scoring, multi-turn, chat template, system prompt MERA Jan 2026 mera.a-ai.ru Official leaderboard
206 ruGPT-3 XL 1.3B open weights, 1.3B 0.20 log-likelihood ai-forever mera.a-ai.ru Official leaderboard
207 ruT5-base open weights, 0.2B 0.19 log-likelihood RussianNLP mera.a-ai.ru Official leaderboard
208 Qwen2-Math-1.5B-Instruct open weights, 1.5B 0.19 log-likelihood, chat template, system prompt MERA mera.a-ai.ru Official leaderboard
209 ruGPT-3-medium open weights, 0.4B 0.17 generative-scoring RussianNLP mera.a-ai.ru Official leaderboard
#ModelScoreSettingSubmitted byDateLinkSource
1 Claude Opus 4.6 0.86 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
2 GPT-5.4 0.82 generative-scoring, multi-turn, chat template, system prompt MERA Mar 2026 mera.a-ai.ru Official leaderboard
3 GLM-5.1 0.80 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
4 Qwen3-235B-A22B-Thinking Jul 2025 0.80 generative-scoring, multi-turn, chat template, system prompt MERA Jul 2025 mera.a-ai.ru Official leaderboard
5 Qwen 3.6 35B-A3B 0.79 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
6 Qwen3.6 27B 0.79 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
7 Qwen3.5-27B 0.78 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
8 Qwen3.5-122B-A10B 0.77 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
9 Claude Opus 4.5 0.76 generative-scoring, multi-turn, chat template, system prompt Claude Opus 4.5 Nov 2025 mera.a-ai.ru Official leaderboard
10 Qwen3-30B-A3B-Thinking Jul 2025 0.75 generative-scoring, multi-turn, chat template, system prompt MERA Jul 2025 mera.a-ai.ru Official leaderboard
11 Kimi K2.5 0.75 generative-scoring, multi-turn, chat template, system prompt MERA Jan 2026 mera.a-ai.ru Official leaderboard
12 Qwen3-32B 0.73 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
13 Qwen3.5-35B-A3B 0.73 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
14 Qwen3-Next-80B-A3B 0.71 generative-scoring, multi-turn, chat template, system prompt MERA Sep 2025 mera.a-ai.ru Official leaderboard
15 Gemma 4 31B IT 0.71 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
16 Qwen3-4B-Thinking-2507 0.71 generative-scoring, multi-turn, chat template, system prompt MERA Aug 2025 mera.a-ai.ru Official leaderboard
17 GPT-5.2 0.71 generative-scoring, multi-turn, chat template, system prompt GPT-5.2 Dec 2025 mera.a-ai.ru Official leaderboard
18 MiniMax-M2.7 0.69 generative-scoring, multi-turn MERA Mar 2026 mera.a-ai.ru Official leaderboard
19 Qwen3-14B 0.69 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
20 DeepSeek-V4-Pro 0.69 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
21 Claude 3.7 Sonnet 0.68 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Feb 2025 mera.a-ai.ru Official leaderboard
22 Gemini 2.0 Flash Feb 2025 0.68 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Feb 2025 mera.a-ai.ru Official leaderboard
23 Qwen3.5-9B 0.68 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
24 DeepSeek-V3 Mar 2025 0.68 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Mar 2025 mera.a-ai.ru Official leaderboard
25 DeepSeek-V4-Flash 0.68 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
26 Gemini 1.5 Pro Sep 2024 0.68 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Sep 2024 mera.a-ai.ru Official leaderboard
27 Qwen3-4B 0.67 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
28 Qwen2.5-Max 0.66 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Jan 2025 mera.a-ai.ru Official leaderboard
29 Qwen3-8B 0.65 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
30 GPT-4o Mar 2025 0.64 generative-scoring, multi-turn, chat template, system prompt MERA Mar 2025 mera.a-ai.ru Official leaderboard
31 Gemini 2.0 Flash-Lite 0.64 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Feb 2025 mera.a-ai.ru Official leaderboard
32 Gemini 1.5 Flash Sep 2024 0.63 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Aug 2024 mera.a-ai.ru Official leaderboard
33 Qwen2.5-72B 0.62 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
34 DeepSeek-R1-Distill-Qwen-32B 0.62 generative-scoring, chat template, system prompt НГУ Jan 2025 mera.a-ai.ru Official leaderboard
35 gpt-oss-20b 0.62 generative-scoring, multi-turn, chat template, system prompt openai Aug 2025 mera.a-ai.ru Official leaderboard
36 Llama 4 Maverick 0.61 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Apr 2025 mera.a-ai.ru Official leaderboard
37 Gemma 4 26B A4B 0.60 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2026 mera.a-ai.ru Official leaderboard
38 Llama 3.1-70B 0.57 log-likelihood MERA Jul 2024 mera.a-ai.ru Official leaderboard
39 Qwen2-72B 0.57 generative-scoring, multi-turn, chat template, system prompt MERA Jun 2024 mera.a-ai.ru Official leaderboard
40 GPT-4o mini 0.57 generative-scoring, multi-turn, chat template, system prompt MERA Jul 2024 mera.a-ai.ru Official leaderboard
41 Gemma 3 27B 0.57 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Mar 2025 mera.a-ai.ru Official leaderboard
42 Qwen3.5-4B 0.57 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
43 Llama 3-70B 0.56 generative-scoring MERA Apr 2024 mera.a-ai.ru Official leaderboard
44 Llama 3.3 70B Instruct 0.56 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Dec 2024 mera.a-ai.ru Official leaderboard
45 Qwen-Turbo 0.55 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Nov 2024 mera.a-ai.ru Official leaderboard
46 Claude 3.5 Haiku 0.55 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Oct 2024 mera.a-ai.ru Official leaderboard
47 Gemini 1.5 Flash 8B 0.54 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Oct 2024 mera.a-ai.ru Official leaderboard
48 Llama-3.1-Nemotron-70B-Instruct 0.53 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Oct 2024 mera.a-ai.ru Official leaderboard
49 Llama 4 Scout 0.53 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Apr 2025 mera.a-ai.ru Official leaderboard
50 Gemma 2 27B 0.52 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
51 phi-3-medium 14B 0.51 log-likelihood MERA Apr 2024 mera.a-ai.ru Official leaderboard
52 Phi-3-medium-128k-instruct 0.51 log-likelihood MERA May 2024 mera.a-ai.ru Official leaderboard
53 Qwen2.5-Math-72B-Instruct 0.51 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
54 Phi-3.5-MoE 0.50 log-likelihood MERA Aug 2024 mera.a-ai.ru Official leaderboard
55 Mixtral 8x7B 0.49 log-likelihood MERA Dec 2023 mera.a-ai.ru Official leaderboard
56 Qwen3-30B-A3B-Instruct Jul 2025 0.49 generative-scoring, multi-turn, chat template, system prompt MERA Jul 2025 mera.a-ai.ru Official leaderboard
57 Qwen1.5-32B 0.48 log-likelihood MERA Feb 2024 mera.a-ai.ru Official leaderboard
58 Gemma 4 E4B 0.48 generative-scoring, multi-turn, chat template, system prompt MERA Mar 2026 mera.a-ai.ru Official leaderboard
59 Qwen2-7B-Instruct 0.47 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
60 Yi-1.5-34B 0.47 log-likelihood MERA May 2024 mera.a-ai.ru Official leaderboard
61 qwen3-4b-instruct-2507 0.47 generative-scoring, multi-turn, chat template, system prompt MERA Aug 2025 mera.a-ai.ru Official leaderboard
62 phi-3-small 7.4B 0.47 log-likelihood MERA Apr 2024 mera.a-ai.ru Official leaderboard
63 Qwen2.5-7B 0.46 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
64 Command R (03-2024) 0.46 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Mar 2024 mera.a-ai.ru Official leaderboard
65 Qwen2-7B 0.46 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
66 Gemma 2 9B 0.45 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
67 Llama 3.1-8B 0.45 log-likelihood MERA Jul 2024 mera.a-ai.ru Official leaderboard
68 DeepSeek-R1-Distill-Qwen-14B 0.45 log-likelihood НГУ Jan 2025 mera.a-ai.ru Official leaderboard
69 Llama 3-8B 0.45 generative-scoring MERA Apr 2024 mera.a-ai.ru Official leaderboard
70 Yi-34B 0.44 log-likelihood MERA Nov 2023 mera.a-ai.ru Official leaderboard
71 GPT-3.5 Turbo Jan 2024 0.44 generative-scoring, multi-turn, chat template MERA Jan 2024 mera.a-ai.ru Official leaderboard
72 Olmo 3.1 32B Instruct 0.43 generative-scoring, multi-turn, chat template, system prompt MERA Dec 2025 mera.a-ai.ru Official leaderboard
73 Mistral NeMo 0.42 generative-scoring, multi-turn, chat template, system prompt llmarena.ru Jul 2024 mera.a-ai.ru Official leaderboard
74 Qwen2.5-3B 0.41 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
75 Mistral-7B-Instruct-v0.2 0.41 log-likelihood MERA Dec 2023 mera.a-ai.ru Official leaderboard
76 Mistral 7B v0.1 0.40 log-likelihood MERA Sep 2023 mera.a-ai.ru Official leaderboard
77 Gemma 4 E2B 0.40 generative-scoring, multi-turn, chat template, system prompt MERA Mar 2026 mera.a-ai.ru Official leaderboard
78 phi-3-mini 3.8B 0.40 log-likelihood MERA Apr 2024 mera.a-ai.ru Official leaderboard
79 Mistral 7B v0.3 0.40 log-likelihood MERA Sep 2023 mera.a-ai.ru Official leaderboard
80 Phi-3-mini-128k-instruct 0.40 log-likelihood MERA Apr 2024 mera.a-ai.ru Official leaderboard
81 Qwen3-0.6B 0.39 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
82 Yi-1.5-9B-Chat 0.39 log-likelihood MERA May 2024 mera.a-ai.ru Official leaderboard
83 Gemma 2 2B IT 0.39 log-likelihood MERA Jul 2024 mera.a-ai.ru Official leaderboard
84 Qwen1.5-7B 0.37 log-likelihood MERA Feb 2024 mera.a-ai.ru Official leaderboard
85 Mistral-7B-Instruct-v0.1 0.37 log-likelihood MERA Sep 2023 mera.a-ai.ru Official leaderboard
86 Qwen1.5-7B-Chat 0.37 log-likelihood MERA Feb 2024 mera.a-ai.ru Official leaderboard
87 Llama 3.2 3B 0.36 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
88 Qwen3.5-2B 0.35 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
89 Qwen2-1.5B-Instruct 0.34 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
90 Qwen3-1.7B 0.34 generative-scoring, multi-turn, chat template, system prompt MERA Apr 2025 mera.a-ai.ru Official leaderboard
91 Yi-1.5-6B-Chat 0.33 log-likelihood MERA May 2024 mera.a-ai.ru Official leaderboard
92 Qwen2-1.5B 0.33 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
93 Yi 6B 0.32 generative-scoring MERA Nov 2023 mera.a-ai.ru Official leaderboard
94 Qwen2.5-Coder (1.5B) 0.32 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
95 Qwen1.5-4B-Chat 0.31 log-likelihood, multi-turn, chat template, system prompt MERA Feb 2024 mera.a-ai.ru Official leaderboard
96 Qwen3.5-0.8B 0.30 generative-scoring, multi-turn, chat template, system prompt MERA Feb 2026 mera.a-ai.ru Official leaderboard
97 LFM2-8B-A1B 0.28 generative-scoring, multi-turn, chat template, system prompt MERA Oct 2025 mera.a-ai.ru Official leaderboard
98 Qwen2-0.5B-Instruct 0.26 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
99 Llama 3.2 1B 0.25 log-likelihood, system prompt MERA Sep 2024 mera.a-ai.ru Official leaderboard
100 Qwen2-0.5B 0.25 log-likelihood MERA Jun 2024 mera.a-ai.ru Official leaderboard
101 Yi-6B-Chat 0.23 log-likelihood, chat template MERA Nov 2023 mera.a-ai.ru Official leaderboard
102 LFM2.5-1.2B-Instruct 0.20 generative-scoring, multi-turn, chat template, system prompt MERA Jan 2026 mera.a-ai.ru Official leaderboard

Rankings by part

We read 205 parts from the benchmark’s own leaderboard in 23 groups: bps, lcs, rcb, use, rwsd, parus, ruhhh, rutie, multiq, rummlu, chegeka, rudetox, rumodar, mamuramu, simplear, ruethics, rumultiar, rucodeeval, mathlogicqa, ruhumaneval, ruworldtree, ruhatespeech and ruopenbookqa. Each ranks models on one part of the test. Claude Opus 4.6 leads 62 of them, and 34 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.

BPS acc

101 models we track have a score here.

  1. 1 Qwen2.5-72B ↑32 1.00
  2. 2 Claude Opus 4.5 ↑7 1.00
  3. 3 Qwen3.5-122B-A10B ↑5 1.00
  4. 4 Qwen3.5-27B ↑3 1.00
  5. 5 DeepSeek-V3 Mar 2025 ↑19 1.00
  6. 6 GPT-5.4 ↓4 1.00
  7. 7 Qwen3.6 27B ↓1 1.00
  8. 8 Qwen2.5-Max ↑20 1.00
  9. 9 Gemini 2.0 Flash Feb 2025 ↑13 1.00
  10. 10 Gemini 2.0 Flash-Lite ↑21 1.00

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Qwen2.5-72B
generative-… log-likelih…
0.62
Llama 3.1-70B
generative-…
0.57
Qwen2-72B
generative-…
0.57
Llama 3-70B
log-likelih…
0.56
phi-3-medium 14B
generative-… log-likelih…
0.51
Phi-3-medium-128k-instruct
generative-… log-likelih… log-likelih… generative-…
0.51
Qwen2.5-Math-72B-Instruct
generative-… log-likelih… log-likelih… log-likelih…
0.51
Phi-3.5-MoE
generative-…
0.50
Mixtral 8x7B
generative-…
0.49
Qwen1.5-32B
generative-…
0.48
Qwen2-7B-Instruct
generative-…
0.47
Yi-1.5-34B
generative-…
0.47

Source: Official MERA leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 4.6 generative-scoring, multi-turn, chat template, system prompt 0.86
GPT-5.4 generative-scoring, multi-turn, chat template, system prompt 0.82
GLM-5.1 generative-scoring, multi-turn, chat template, system prompt 0.80
Qwen3-235B-A22B-Thinking Jul 2025 generative-scoring, multi-turn, chat template, system prompt 0.80
Qwen 3.6 35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.79
Qwen3.6 27B generative-scoring, multi-turn, chat template, system prompt 0.79
Qwen3.5-27B generative-scoring, multi-turn, chat template, system prompt 0.78
Qwen3.5-122B-A10B generative-scoring, multi-turn, chat template, system prompt 0.77
Claude Opus 4.5 generative-scoring, multi-turn, chat template, system prompt 0.76
Qwen3-30B-A3B-Thinking Jul 2025 generative-scoring, multi-turn, chat template, system prompt 0.75
Kimi K2.5 generative-scoring, multi-turn, chat template, system prompt 0.75
Qwen3-32B generative-scoring, multi-turn, chat template, system prompt 0.73
Qwen3.5-35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.73
Qwen3-Next-80B-A3B generative-scoring, multi-turn, chat template, system prompt 0.71
Gemma 4 31B IT generative-scoring, multi-turn, chat template, system prompt 0.71
Qwen3-4B-Thinking-2507 generative-scoring, multi-turn, chat template, system prompt 0.71
GPT-5.2 generative-scoring, multi-turn, chat template, system prompt 0.71
MiniMax-M2.7 generative-scoring, multi-turn 0.69
Qwen3-14B generative-scoring, multi-turn, chat template, system prompt 0.69
DeepSeek-V4-Pro generative-scoring, multi-turn, chat template, system prompt 0.69
Claude 3.7 Sonnet generative-scoring, multi-turn, chat template, system prompt 0.68
Gemini 2.0 Flash Feb 2025 generative-scoring, multi-turn, chat template, system prompt 0.68
Qwen3.5-9B generative-scoring, multi-turn, chat template, system prompt 0.68
DeepSeek-V3 Mar 2025 generative-scoring, multi-turn, chat template, system prompt 0.68
DeepSeek-V4-Flash generative-scoring, multi-turn, chat template, system prompt 0.68
Gemini 1.5 Pro Sep 2024 generative-scoring, multi-turn, chat template, system prompt 0.68
DeepSeek-V3 Mar 2025 generative-scoring, multi-turn, chat template, system prompt 0.67
Qwen3-4B generative-scoring, multi-turn, chat template, system prompt 0.67
Qwen2.5-Max generative-scoring, multi-turn, chat template, system prompt 0.66
Qwen3-8B generative-scoring, multi-turn, chat template, system prompt 0.65
GPT-4o Mar 2025 generative-scoring, multi-turn, chat template, system prompt 0.64
Gemini 2.0 Flash-Lite generative-scoring, multi-turn, chat template, system prompt 0.64
Gemini 1.5 Flash Sep 2024 generative-scoring, multi-turn, chat template, system prompt 0.63
Qwen2.5-72B log-likelihood, system prompt 0.62
DeepSeek-R1-Distill-Qwen-32B generative-scoring, chat template, system prompt 0.62
gpt-oss-20b generative-scoring, multi-turn, chat template, system prompt 0.62
Qwen2.5-72B generative-scoring, system prompt 0.61
Llama 4 Maverick generative-scoring, multi-turn, chat template, system prompt 0.61
Gemma 4 26B A4B generative-scoring, multi-turn, chat template, system prompt 0.60
Qwen2.5-72B generative-scoring, multi-turn, chat template, system prompt 0.60
Qwen2.5-72B log-likelihood, multi-turn, chat template, system prompt 0.60
Qwen2.5-72B generative-scoring, chat template, system prompt 0.59
Qwen2.5-72B log-likelihood, chat template, system prompt 0.59
Qwen2.5-72B log-likelihood, system prompt 0.59
Qwen2.5-72B generative-scoring, system prompt 0.58
Llama 3.1-70B log-likelihood 0.57
Qwen2-72B generative-scoring, multi-turn, chat template, system prompt 0.57
GPT-4o mini generative-scoring, multi-turn, chat template, system prompt 0.57
Llama 3.1-70B generative-scoring 0.57
Qwen2-72B log-likelihood, multi-turn, chat template, system prompt 0.57
Gemma 3 27B generative-scoring, multi-turn, chat template, system prompt 0.57
Qwen3.5-4B generative-scoring, multi-turn, chat template, system prompt 0.57
Qwen2-72B log-likelihood 0.57
Qwen2-72B generative-scoring 0.56
Qwen2-72B log-likelihood 0.56
Llama 3-70B generative-scoring 0.56
Llama 3-70B log-likelihood 0.56
Llama 3.3 70B Instruct generative-scoring, multi-turn, chat template, system prompt 0.56
Qwen2-72B generative-scoring 0.55
Qwen2-72B log-likelihood, chat template 0.55
Qwen2-72B generative-scoring, chat template 0.55
Qwen-Turbo generative-scoring, multi-turn, chat template, system prompt 0.55
Claude 3.5 Haiku generative-scoring, multi-turn, chat template, system prompt 0.55
Gemini 1.5 Flash 8B generative-scoring, multi-turn, chat template, system prompt 0.54
Llama-3.1-Nemotron-70B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.53
Llama 4 Scout generative-scoring, multi-turn, chat template, system prompt 0.53
Gemma 2 27B log-likelihood 0.52
phi-3-medium 14B log-likelihood 0.51
phi-3-medium 14B generative-scoring 0.51
Phi-3-medium-128k-instruct log-likelihood 0.51
Qwen2.5-Math-72B-Instruct log-likelihood, system prompt 0.51
Phi-3-medium-128k-instruct generative-scoring 0.51
Phi-3.5-MoE log-likelihood 0.50
Qwen2.5-Math-72B-Instruct generative-scoring, system prompt 0.49
Mixtral 8x7B log-likelihood 0.49
Qwen3-30B-A3B-Instruct Jul 2025 generative-scoring, multi-turn, chat template, system prompt 0.49
Phi-3.5-MoE log-likelihood, multi-turn, chat template, system prompt 0.49
Phi-3.5-MoE generative-scoring, multi-turn, chat template, system prompt 0.49
Qwen1.5-32B log-likelihood 0.48
Phi-3.5-MoE log-likelihood, chat template, system prompt 0.48
Mixtral 8x7B generative-scoring 0.48
Gemma 4 E4B generative-scoring, multi-turn, chat template, system prompt 0.48
Qwen1.5-32B generative-scoring 0.48
phi-3-medium 14B log-likelihood, multi-turn, chat template, system prompt 0.48
Phi-3.5-MoE generative-scoring, chat template, system prompt 0.48
Qwen2-7B-Instruct log-likelihood 0.47
phi-3-medium 14B log-likelihood, multi-turn, chat template 0.47
Yi-1.5-34B log-likelihood 0.47
qwen3-4b-instruct-2507 generative-scoring, multi-turn, chat template, system prompt 0.47
Phi-3-medium-128k-instruct log-likelihood, multi-turn, chat template, system prompt 0.47
phi-3-medium 14B generative-scoring, multi-turn, chat template, system prompt 0.47
phi-3-small 7.4B log-likelihood 0.47
Yi-1.5-34B generative-scoring 0.47
Qwen2-7B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.46
Qwen2.5-Math-72B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.46
Phi-3-medium-128k-instruct log-likelihood, multi-turn, chat template 0.46
Qwen2-7B-Instruct generative-scoring 0.46
Phi-3.5-MoE generative-scoring 0.46
Qwen2.5-Math-72B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.46
phi-3-medium 14B generative-scoring, multi-turn, chat template 0.46
phi-3-small 7.4B generative-scoring 0.46
Qwen2.5-7B log-likelihood, system prompt 0.46
Command R (03-2024) generative-scoring, multi-turn, chat template, system prompt 0.46
phi-3-small 7.4B log-likelihood, multi-turn, chat template, system prompt 0.46
Qwen2-7B log-likelihood 0.46
Qwen2-7B generative-scoring 0.46
Qwen2-7B-Instruct log-likelihood, multi-turn, chat template 0.46
Gemma 2 9B log-likelihood 0.45
phi-3-medium 14B log-likelihood, chat template 0.45
Gemma 2 9B generative-scoring 0.45
Llama 3.1-8B log-likelihood 0.45
Qwen2-7B-Instruct generative-scoring, multi-turn, chat template 0.45
phi-3-medium 14B generative-scoring, chat template 0.45
DeepSeek-R1-Distill-Qwen-14B log-likelihood 0.45
Llama 3-8B generative-scoring 0.45
Qwen2-7B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.45
Llama 3.1-8B generative-scoring 0.45
Qwen2.5-7B generative-scoring, system prompt 0.45
Llama 3-8B generative-scoring 0.44
Qwen2-7B-Instruct log-likelihood, chat template, system prompt 0.44
Phi-3-medium-128k-instruct generative-scoring, multi-turn, chat template, system prompt 0.44
Yi-34B log-likelihood 0.44
Qwen2-7B-Instruct log-likelihood, chat template 0.44
Qwen2-7B-Instruct log-likelihood, chat template 0.44
phi-3-medium 14B log-likelihood, chat template, system prompt 0.44
Qwen2-7B-Instruct generative-scoring, chat template, system prompt 0.44
GPT-3.5 Turbo Jan 2024 generative-scoring, multi-turn, chat template 0.44
phi-3-medium 14B generative-scoring, chat template 0.44
phi-3-small 7.4B generative-scoring, multi-turn, chat template, system prompt 0.44
Qwen2-7B-Instruct generative-scoring, chat template 0.44
Qwen2-7B-Instruct generative-scoring, chat template 0.44
Yi-34B generative-scoring 0.44
phi-3-medium 14B generative-scoring, chat template 0.44
phi-3-small 7.4B log-likelihood, chat template, system prompt 0.44
Phi-3-medium-128k-instruct generative-scoring, multi-turn, chat template 0.44
Olmo 3.1 32B Instruct generative-scoring, multi-turn, chat template, system prompt 0.43
phi-3-medium 14B generative-scoring, chat template, system prompt 0.43
phi-3-small 7.4B generative-scoring, chat template, system prompt 0.42
phi-3-small 7.4B log-likelihood, chat template 0.42
Mistral NeMo generative-scoring, multi-turn, chat template, system prompt 0.42
Phi-3-medium-128k-instruct log-likelihood, chat template 0.42
Phi-3-medium-128k-instruct log-likelihood, chat template 0.42
Phi-3-medium-128k-instruct log-likelihood, chat template, system prompt 0.42
phi-3-small 7.4B generative-scoring, chat template 0.41
Qwen2.5-3B log-likelihood, system prompt 0.41
Mistral-7B-Instruct-v0.2 log-likelihood 0.41
Mistral 7B v0.1 log-likelihood 0.40
Qwen2.5-3B generative-scoring, system prompt 0.40
Gemma 4 E2B generative-scoring, multi-turn, chat template, system prompt 0.40
phi-3-mini 3.8B log-likelihood 0.40
Mistral 7B v0.3 log-likelihood 0.40
Phi-3-mini-128k-instruct log-likelihood 0.40
Mistral 7B v0.1 generative-scoring 0.39
phi-3-mini 3.8B generative-scoring 0.39
Qwen3-0.6B generative-scoring, multi-turn, chat template, system prompt 0.39
Yi-1.5-9B-Chat log-likelihood 0.39
Yi-1.5-9B-Chat generative-scoring 0.39
Mistral 7B v0.3 generative-scoring 0.39
Phi-3-mini-128k-instruct generative-scoring 0.39
Gemma 2 2B IT log-likelihood 0.39
Gemma 2 2B IT generative-scoring 0.39
Mistral-7B-Instruct-v0.2 generative-scoring 0.38
Qwen1.5-7B log-likelihood 0.37
Mistral-7B-Instruct-v0.1 log-likelihood 0.37
Qwen1.5-7B-Chat log-likelihood 0.37
Mistral-7B-Instruct-v0.1 log-likelihood 0.37
Qwen1.5-7B generative-scoring 0.37
Qwen1.5-7B-Chat log-likelihood, multi-turn, chat template, system prompt 0.36
Yi-1.5-9B-Chat log-likelihood, multi-turn, chat template, system prompt 0.36
Llama 3.2 3B log-likelihood, system prompt 0.36
Qwen1.5-7B-Chat log-likelihood, multi-turn, chat template 0.36
Phi-3-medium-128k-instruct generative-scoring, chat template 0.35
Qwen1.5-7B-Chat log-likelihood, chat template, system prompt 0.35
Phi-3-medium-128k-instruct generative-scoring, chat template, system prompt 0.35
Phi-3-medium-128k-instruct generative-scoring, chat template 0.35
Qwen3.5-2B generative-scoring, multi-turn, chat template, system prompt 0.35
Llama 3.2 3B generative-scoring, system prompt 0.35
Mistral-7B-Instruct-v0.2 log-likelihood, multi-turn, chat template, system prompt 0.35
Qwen1.5-7B-Chat generative-scoring, multi-turn, chat template, system prompt 0.35
phi-3-mini 3.8B log-likelihood, multi-turn, chat template, system prompt 0.35
Yi-1.5-9B-Chat generative-scoring, multi-turn, chat template, system prompt 0.35
Phi-3-mini-128k-instruct log-likelihood, multi-turn, chat template, system prompt 0.35
Qwen1.5-7B-Chat generative-scoring, chat template, system prompt 0.35
Qwen1.5-7B-Chat generative-scoring 0.34
Phi-3-mini-128k-instruct generative-scoring, multi-turn, chat template, system prompt 0.34
Yi-1.5-9B-Chat log-likelihood, chat template, system prompt 0.34
phi-3-mini 3.8B generative-scoring, multi-turn, chat template, system prompt 0.34
Yi-1.5-9B-Chat log-likelihood, chat template 0.34
Qwen2-1.5B-Instruct log-likelihood 0.34
Qwen3-1.7B generative-scoring, multi-turn, chat template, system prompt 0.34
Qwen1.5-7B-Chat log-likelihood, chat template 0.34
Qwen1.5-7B-Chat generative-scoring, multi-turn, chat template 0.34
Yi-1.5-9B-Chat generative-scoring, chat template 0.34
Yi-1.5-6B-Chat log-likelihood 0.33
Phi-3-mini-128k-instruct log-likelihood, chat template 0.33
Yi-1.5-9B-Chat generative-scoring, chat template, system prompt 0.33
Yi-1.5-6B-Chat generative-scoring 0.33
Qwen2-1.5B-Instruct generative-scoring 0.33
Qwen1.5-7B-Chat generative-scoring, chat template 0.33
Mistral-7B-Instruct-v0.2 log-likelihood, chat template 0.33
Phi-3-mini-128k-instruct generative-scoring, chat template 0.33
Qwen2-1.5B log-likelihood 0.33
Qwen2-1.5B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.32
Qwen2.5-Math-72B-Instruct log-likelihood, chat template, system prompt 0.32
Qwen2-1.5B generative-scoring 0.32
Qwen2-1.5B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.32
Mistral-7B-Instruct-v0.2 generative-scoring, multi-turn, chat template, system prompt 0.32
Yi 6B generative-scoring 0.32
Qwen1.5-7B log-likelihood, chat template 0.32
Qwen2.5-Coder (1.5B) log-likelihood, system prompt 0.32
phi-3-mini 3.8B log-likelihood, chat template 0.32
Mistral-7B-Instruct-v0.1 log-likelihood, multi-turn, chat template, system prompt 0.31
Qwen1.5-4B-Chat log-likelihood, multi-turn, chat template, system prompt 0.31
Mistral-7B-Instruct-v0.1 log-likelihood, chat template 0.31
Qwen1.5-4B-Chat log-likelihood, multi-turn, chat template 0.31
Mistral-7B-Instruct-v0.1 generative-scoring, chat template 0.31
Qwen1.5-4B-Chat generative-scoring, multi-turn, chat template 0.31
Mistral-7B-Instruct-v0.1 generative-scoring, multi-turn, chat template, system prompt 0.31
phi-3-mini 3.8B generative-scoring, chat template 0.31
Qwen2-1.5B-Instruct generative-scoring, chat template 0.30
Qwen2-1.5B-Instruct generative-scoring, chat template 0.30
Qwen2-1.5B-Instruct log-likelihood, chat template 0.30
Yi 6B generative-scoring 0.30
Yi-1.5-6B-Chat log-likelihood, multi-turn, chat template, system prompt 0.30
Qwen2-1.5B-Instruct log-likelihood, chat template 0.30
Qwen2-7B log-likelihood, chat template 0.30
Qwen1.5-4B-Chat log-likelihood 0.30
Qwen3.5-0.8B generative-scoring, multi-turn, chat template, system prompt 0.30
Qwen1.5-4B-Chat generative-scoring, multi-turn, chat template, system prompt 0.30
Qwen2.5-Coder (1.5B) generative-scoring, system prompt 0.30
Gemma 2 2B IT generative-scoring, multi-turn, chat template 0.29
Gemma 2 2B IT log-likelihood, chat template 0.29
Qwen1.5-4B-Chat log-likelihood, chat template, system prompt 0.29
Qwen1.5-4B-Chat generative-scoring, chat template, system prompt 0.29
Qwen1.5-4B-Chat generative-scoring, chat template 0.28
Mistral-7B-Instruct-v0.2 generative-scoring, chat template 0.28
Yi-1.5-6B-Chat generative-scoring, multi-turn, chat template, system prompt 0.28
Qwen1.5-4B-Chat log-likelihood, chat template 0.28
Qwen1.5-7B generative-scoring, chat template 0.28
LFM2-8B-A1B generative-scoring, multi-turn, chat template, system prompt 0.28
Gemma 2 2B IT generative-scoring, chat template 0.27
Qwen1.5-4B-Chat generative-scoring 0.27
Yi-1.5-6B-Chat generative-scoring, chat template 0.27
Yi-1.5-6B-Chat log-likelihood, chat template 0.27
Qwen2-0.5B-Instruct log-likelihood 0.26
Qwen2-0.5B-Instruct log-likelihood, multi-turn, chat template 0.25
Qwen2-0.5B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.25
Llama 3.2 1B log-likelihood, system prompt 0.25
Qwen2-0.5B log-likelihood 0.25
Qwen2-0.5B-Instruct generative-scoring, multi-turn, chat template 0.25
Qwen2-0.5B-Instruct generative-scoring 0.24
Qwen2-0.5B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.24
Qwen2-0.5B generative-scoring 0.24
Llama 3.2 1B generative-scoring, system prompt 0.23
Qwen2-0.5B-Instruct log-likelihood, chat template 0.23
Qwen2-0.5B-Instruct log-likelihood, chat template 0.23
Yi-6B-Chat log-likelihood, chat template 0.23
Yi-6B-Chat log-likelihood, chat template 0.23
Qwen2-0.5B-Instruct generative-scoring, chat template 0.22
Qwen2-0.5B-Instruct generative-scoring, chat template 0.22
Qwen2-0.5B-Instruct log-likelihood, chat template, system prompt 0.22
Qwen2-0.5B-Instruct generative-scoring, chat template, system prompt 0.22
Yi-6B-Chat log-likelihood, system prompt 0.21
LFM2.5-1.2B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.20
Yi-6B-Chat generative-scoring, chat template 0.19
Yi-6B-Chat generative-scoring, chat template 0.19
Qwen2-1.5B log-likelihood, chat template 0.19
Qwen2-1.5B log-likelihood, chat template 0.19
Yi-6B-Chat generative-scoring, system prompt 0.19
Qwen2-7B generative-scoring, chat template 0.19
Qwen2-0.5B log-likelihood, chat template 0.18
Qwen2-0.5B log-likelihood, chat template 0.18
Qwen2.5-Math-72B-Instruct generative-scoring, chat template, system prompt 0.17
Qwen2-1.5B generative-scoring, chat template 0.10
Qwen2-1.5B generative-scoring, chat template 0.10
Qwen2-0.5B generative-scoring, chat template 0.06
Qwen2-0.5B generative-scoring, chat template 0.06

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Qwen2.5-72B
generative-… log-likelih…
0.62
Llama 3.1-70B
generative-…
0.57
Qwen2-72B
generative-…
0.57
Llama 3-70B
log-likelih…
0.56
phi-3-medium 14B
generative-… log-likelih…
0.51
Phi-3-medium-128k-instruct
generative-… log-likelih… log-likelih… generative-…
0.51
Qwen2.5-Math-72B-Instruct
generative-… log-likelih… log-likelih… log-likelih…
0.51
Phi-3.5-MoE
generative-…
0.50
Mixtral 8x7B
generative-…
0.49
Qwen1.5-32B
generative-…
0.48
Qwen2-7B-Instruct
generative-…
0.47
Yi-1.5-34B
generative-…
0.47

Source: Official MERA leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 4.6 generative-scoring, multi-turn, chat template, system prompt 0.86
GPT-5.4 generative-scoring, multi-turn, chat template, system prompt 0.82
GLM-5.1 generative-scoring, multi-turn, chat template, system prompt 0.80
Qwen3-235B-A22B-Thinking Jul 2025 generative-scoring, multi-turn, chat template, system prompt 0.80
Qwen 3.6 35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.79
Qwen3.6 27B generative-scoring, multi-turn, chat template, system prompt 0.79
Qwen3.5-27B generative-scoring, multi-turn, chat template, system prompt 0.78
Qwen3.5-122B-A10B generative-scoring, multi-turn, chat template, system prompt 0.77
Claude Opus 4.5 generative-scoring, multi-turn, chat template, system prompt 0.76
Qwen3-30B-A3B-Thinking Jul 2025 generative-scoring, multi-turn, chat template, system prompt 0.75
Kimi K2.5 generative-scoring, multi-turn, chat template, system prompt 0.75
Qwen3-32B generative-scoring, multi-turn, chat template, system prompt 0.73
Qwen3.5-35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.73
Qwen3-Next-80B-A3B generative-scoring, multi-turn, chat template, system prompt 0.71
Gemma 4 31B IT generative-scoring, multi-turn, chat template, system prompt 0.71
Qwen3-4B-Thinking-2507 generative-scoring, multi-turn, chat template, system prompt 0.71
GPT-5.2 generative-scoring, multi-turn, chat template, system prompt 0.71
MiniMax-M2.7 generative-scoring, multi-turn 0.69
Qwen3-14B generative-scoring, multi-turn, chat template, system prompt 0.69
DeepSeek-V4-Pro generative-scoring, multi-turn, chat template, system prompt 0.69
Claude 3.7 Sonnet generative-scoring, multi-turn, chat template, system prompt 0.68
Gemini 2.0 Flash Feb 2025 generative-scoring, multi-turn, chat template, system prompt 0.68
Qwen3.5-9B generative-scoring, multi-turn, chat template, system prompt 0.68
DeepSeek-V3 Mar 2025 generative-scoring, multi-turn, chat template, system prompt 0.68
DeepSeek-V4-Flash generative-scoring, multi-turn, chat template, system prompt 0.68
Gemini 1.5 Pro Sep 2024 generative-scoring, multi-turn, chat template, system prompt 0.68
DeepSeek-V3 Mar 2025 generative-scoring, multi-turn, chat template, system prompt 0.67
Qwen3-4B generative-scoring, multi-turn, chat template, system prompt 0.67
Qwen2.5-Max generative-scoring, multi-turn, chat template, system prompt 0.66
Qwen3-8B generative-scoring, multi-turn, chat template, system prompt 0.65
GPT-4o Mar 2025 generative-scoring, multi-turn, chat template, system prompt 0.64
Gemini 2.0 Flash-Lite generative-scoring, multi-turn, chat template, system prompt 0.64
Gemini 1.5 Flash Sep 2024 generative-scoring, multi-turn, chat template, system prompt 0.63
Qwen2.5-72B log-likelihood, system prompt 0.62
DeepSeek-R1-Distill-Qwen-32B generative-scoring, chat template, system prompt 0.62
gpt-oss-20b generative-scoring, multi-turn, chat template, system prompt 0.62
Qwen2.5-72B generative-scoring, system prompt 0.61
Llama 4 Maverick generative-scoring, multi-turn, chat template, system prompt 0.61
Gemma 4 26B A4B generative-scoring, multi-turn, chat template, system prompt 0.60
Qwen2.5-72B generative-scoring, multi-turn, chat template, system prompt 0.60
Qwen2.5-72B log-likelihood, multi-turn, chat template, system prompt 0.60
Qwen2.5-72B generative-scoring, chat template, system prompt 0.59
Qwen2.5-72B log-likelihood, chat template, system prompt 0.59
Qwen2.5-72B log-likelihood, system prompt 0.59
Qwen2.5-72B generative-scoring, system prompt 0.58
Llama 3.1-70B log-likelihood 0.57
Qwen2-72B generative-scoring, multi-turn, chat template, system prompt 0.57
GPT-4o mini generative-scoring, multi-turn, chat template, system prompt 0.57
Llama 3.1-70B generative-scoring 0.57
Qwen2-72B log-likelihood, multi-turn, chat template, system prompt 0.57
Gemma 3 27B generative-scoring, multi-turn, chat template, system prompt 0.57
Qwen3.5-4B generative-scoring, multi-turn, chat template, system prompt 0.57
Qwen2-72B log-likelihood 0.57
Qwen2-72B generative-scoring 0.56
Qwen2-72B log-likelihood 0.56
Llama 3-70B generative-scoring 0.56
Llama 3-70B log-likelihood 0.56
Llama 3.3 70B Instruct generative-scoring, multi-turn, chat template, system prompt 0.56
Qwen2-72B generative-scoring 0.55
Qwen2-72B log-likelihood, chat template 0.55
Qwen2-72B generative-scoring, chat template 0.55
Qwen-Turbo generative-scoring, multi-turn, chat template, system prompt 0.55
Claude 3.5 Haiku generative-scoring, multi-turn, chat template, system prompt 0.55
Gemini 1.5 Flash 8B generative-scoring, multi-turn, chat template, system prompt 0.54
Llama-3.1-Nemotron-70B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.53
Llama 4 Scout generative-scoring, multi-turn, chat template, system prompt 0.53
Gemma 2 27B log-likelihood 0.52
phi-3-medium 14B log-likelihood 0.51
phi-3-medium 14B generative-scoring 0.51
Phi-3-medium-128k-instruct log-likelihood 0.51
Qwen2.5-Math-72B-Instruct log-likelihood, system prompt 0.51
Phi-3-medium-128k-instruct generative-scoring 0.51
Phi-3.5-MoE log-likelihood 0.50
Qwen2.5-Math-72B-Instruct generative-scoring, system prompt 0.49
Mixtral 8x7B log-likelihood 0.49
Qwen3-30B-A3B-Instruct Jul 2025 generative-scoring, multi-turn, chat template, system prompt 0.49
Phi-3.5-MoE log-likelihood, multi-turn, chat template, system prompt 0.49
Phi-3.5-MoE generative-scoring, multi-turn, chat template, system prompt 0.49
Qwen1.5-32B log-likelihood 0.48
Phi-3.5-MoE log-likelihood, chat template, system prompt 0.48
Mixtral 8x7B generative-scoring 0.48
Gemma 4 E4B generative-scoring, multi-turn, chat template, system prompt 0.48
Qwen1.5-32B generative-scoring 0.48
phi-3-medium 14B log-likelihood, multi-turn, chat template, system prompt 0.48
Phi-3.5-MoE generative-scoring, chat template, system prompt 0.48
Qwen2-7B-Instruct log-likelihood 0.47
phi-3-medium 14B log-likelihood, multi-turn, chat template 0.47
Yi-1.5-34B log-likelihood 0.47
qwen3-4b-instruct-2507 generative-scoring, multi-turn, chat template, system prompt 0.47
Phi-3-medium-128k-instruct log-likelihood, multi-turn, chat template, system prompt 0.47
phi-3-medium 14B generative-scoring, multi-turn, chat template, system prompt 0.47
phi-3-small 7.4B log-likelihood 0.47
Yi-1.5-34B generative-scoring 0.47
Qwen2-7B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.46
Qwen2.5-Math-72B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.46
Phi-3-medium-128k-instruct log-likelihood, multi-turn, chat template 0.46
Qwen2-7B-Instruct generative-scoring 0.46
Phi-3.5-MoE generative-scoring 0.46
Qwen2.5-Math-72B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.46
phi-3-medium 14B generative-scoring, multi-turn, chat template 0.46
phi-3-small 7.4B generative-scoring 0.46
Qwen2.5-7B log-likelihood, system prompt 0.46
Command R (03-2024) generative-scoring, multi-turn, chat template, system prompt 0.46
phi-3-small 7.4B log-likelihood, multi-turn, chat template, system prompt 0.46
Qwen2-7B log-likelihood 0.46
Qwen2-7B generative-scoring 0.46
Qwen2-7B-Instruct log-likelihood, multi-turn, chat template 0.46
Gemma 2 9B log-likelihood 0.45
phi-3-medium 14B log-likelihood, chat template 0.45
Gemma 2 9B generative-scoring 0.45
Llama 3.1-8B log-likelihood 0.45
Qwen2-7B-Instruct generative-scoring, multi-turn, chat template 0.45
phi-3-medium 14B generative-scoring, chat template 0.45
DeepSeek-R1-Distill-Qwen-14B log-likelihood 0.45
Llama 3-8B generative-scoring 0.45
Qwen2-7B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.45
Llama 3.1-8B generative-scoring 0.45
Qwen2.5-7B generative-scoring, system prompt 0.45
Llama 3-8B generative-scoring 0.44
Qwen2-7B-Instruct log-likelihood, chat template, system prompt 0.44
Phi-3-medium-128k-instruct generative-scoring, multi-turn, chat template, system prompt 0.44
Yi-34B log-likelihood 0.44
Qwen2-7B-Instruct log-likelihood, chat template 0.44
Qwen2-7B-Instruct log-likelihood, chat template 0.44
phi-3-medium 14B log-likelihood, chat template, system prompt 0.44
Qwen2-7B-Instruct generative-scoring, chat template, system prompt 0.44
GPT-3.5 Turbo Jan 2024 generative-scoring, multi-turn, chat template 0.44
phi-3-medium 14B generative-scoring, chat template 0.44
phi-3-small 7.4B generative-scoring, multi-turn, chat template, system prompt 0.44
Qwen2-7B-Instruct generative-scoring, chat template 0.44
Qwen2-7B-Instruct generative-scoring, chat template 0.44
Yi-34B generative-scoring 0.44
phi-3-medium 14B generative-scoring, chat template 0.44
phi-3-small 7.4B log-likelihood, chat template, system prompt 0.44
Phi-3-medium-128k-instruct generative-scoring, multi-turn, chat template 0.44
Olmo 3.1 32B Instruct generative-scoring, multi-turn, chat template, system prompt 0.43
phi-3-medium 14B generative-scoring, chat template, system prompt 0.43
phi-3-small 7.4B generative-scoring, chat template, system prompt 0.42
phi-3-small 7.4B log-likelihood, chat template 0.42
Mistral NeMo generative-scoring, multi-turn, chat template, system prompt 0.42
Phi-3-medium-128k-instruct log-likelihood, chat template 0.42
Phi-3-medium-128k-instruct log-likelihood, chat template 0.42
Phi-3-medium-128k-instruct log-likelihood, chat template, system prompt 0.42
phi-3-small 7.4B generative-scoring, chat template 0.41
Qwen2.5-3B log-likelihood, system prompt 0.41
Mistral-7B-Instruct-v0.2 log-likelihood 0.41
Mistral 7B v0.1 log-likelihood 0.40
Qwen2.5-3B generative-scoring, system prompt 0.40
Gemma 4 E2B generative-scoring, multi-turn, chat template, system prompt 0.40
phi-3-mini 3.8B log-likelihood 0.40
Mistral 7B v0.3 log-likelihood 0.40
Phi-3-mini-128k-instruct log-likelihood 0.40
Mistral 7B v0.1 generative-scoring 0.39
phi-3-mini 3.8B generative-scoring 0.39
Qwen3-0.6B generative-scoring, multi-turn, chat template, system prompt 0.39
Yi-1.5-9B-Chat log-likelihood 0.39
Yi-1.5-9B-Chat generative-scoring 0.39
Mistral 7B v0.3 generative-scoring 0.39
Phi-3-mini-128k-instruct generative-scoring 0.39
Gemma 2 2B IT log-likelihood 0.39
Gemma 2 2B IT generative-scoring 0.39
Mistral-7B-Instruct-v0.2 generative-scoring 0.38
Qwen1.5-7B log-likelihood 0.37
Mistral-7B-Instruct-v0.1 log-likelihood 0.37
Qwen1.5-7B-Chat log-likelihood 0.37
Mistral-7B-Instruct-v0.1 log-likelihood 0.37
Qwen1.5-7B generative-scoring 0.37
Qwen1.5-7B-Chat log-likelihood, multi-turn, chat template, system prompt 0.36
Yi-1.5-9B-Chat log-likelihood, multi-turn, chat template, system prompt 0.36
Llama 3.2 3B log-likelihood, system prompt 0.36
Qwen1.5-7B-Chat log-likelihood, multi-turn, chat template 0.36
Phi-3-medium-128k-instruct generative-scoring, chat template 0.35
Qwen1.5-7B-Chat log-likelihood, chat template, system prompt 0.35
Phi-3-medium-128k-instruct generative-scoring, chat template, system prompt 0.35
Phi-3-medium-128k-instruct generative-scoring, chat template 0.35
Qwen3.5-2B generative-scoring, multi-turn, chat template, system prompt 0.35
Llama 3.2 3B generative-scoring, system prompt 0.35
Mistral-7B-Instruct-v0.2 log-likelihood, multi-turn, chat template, system prompt 0.35
Qwen1.5-7B-Chat generative-scoring, multi-turn, chat template, system prompt 0.35
phi-3-mini 3.8B log-likelihood, multi-turn, chat template, system prompt 0.35
Yi-1.5-9B-Chat generative-scoring, multi-turn, chat template, system prompt 0.35
Phi-3-mini-128k-instruct log-likelihood, multi-turn, chat template, system prompt 0.35
Qwen1.5-7B-Chat generative-scoring, chat template, system prompt 0.35
Qwen1.5-7B-Chat generative-scoring 0.34
Phi-3-mini-128k-instruct generative-scoring, multi-turn, chat template, system prompt 0.34
Yi-1.5-9B-Chat log-likelihood, chat template, system prompt 0.34
phi-3-mini 3.8B generative-scoring, multi-turn, chat template, system prompt 0.34
Yi-1.5-9B-Chat log-likelihood, chat template 0.34
Qwen2-1.5B-Instruct log-likelihood 0.34
Qwen3-1.7B generative-scoring, multi-turn, chat template, system prompt 0.34
Qwen1.5-7B-Chat log-likelihood, chat template 0.34
Qwen1.5-7B-Chat generative-scoring, multi-turn, chat template 0.34
Yi-1.5-9B-Chat generative-scoring, chat template 0.34
Yi-1.5-6B-Chat log-likelihood 0.33
Phi-3-mini-128k-instruct log-likelihood, chat template 0.33
Yi-1.5-9B-Chat generative-scoring, chat template, system prompt 0.33
Yi-1.5-6B-Chat generative-scoring 0.33
Qwen2-1.5B-Instruct generative-scoring 0.33
Qwen1.5-7B-Chat generative-scoring, chat template 0.33
Mistral-7B-Instruct-v0.2 log-likelihood, chat template 0.33
Phi-3-mini-128k-instruct generative-scoring, chat template 0.33
Qwen2-1.5B log-likelihood 0.33
Qwen2-1.5B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.32
Qwen2.5-Math-72B-Instruct log-likelihood, chat template, system prompt 0.32
Qwen2-1.5B generative-scoring 0.32
Qwen2-1.5B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.32
Mistral-7B-Instruct-v0.2 generative-scoring, multi-turn, chat template, system prompt 0.32
Yi 6B generative-scoring 0.32
Qwen1.5-7B log-likelihood, chat template 0.32
Qwen2.5-Coder (1.5B) log-likelihood, system prompt 0.32
phi-3-mini 3.8B log-likelihood, chat template 0.32
Mistral-7B-Instruct-v0.1 log-likelihood, multi-turn, chat template, system prompt 0.31
Qwen1.5-4B-Chat log-likelihood, multi-turn, chat template, system prompt 0.31
Mistral-7B-Instruct-v0.1 log-likelihood, chat template 0.31
Qwen1.5-4B-Chat log-likelihood, multi-turn, chat template 0.31
Mistral-7B-Instruct-v0.1 generative-scoring, chat template 0.31
Qwen1.5-4B-Chat generative-scoring, multi-turn, chat template 0.31
Mistral-7B-Instruct-v0.1 generative-scoring, multi-turn, chat template, system prompt 0.31
phi-3-mini 3.8B generative-scoring, chat template 0.31
Qwen2-1.5B-Instruct generative-scoring, chat template 0.30
Qwen2-1.5B-Instruct generative-scoring, chat template 0.30
Qwen2-1.5B-Instruct log-likelihood, chat template 0.30
Yi 6B generative-scoring 0.30
Yi-1.5-6B-Chat log-likelihood, multi-turn, chat template, system prompt 0.30
Qwen2-1.5B-Instruct log-likelihood, chat template 0.30
Qwen2-7B log-likelihood, chat template 0.30
Qwen1.5-4B-Chat log-likelihood 0.30
Qwen3.5-0.8B generative-scoring, multi-turn, chat template, system prompt 0.30
Qwen1.5-4B-Chat generative-scoring, multi-turn, chat template, system prompt 0.30
Qwen2.5-Coder (1.5B) generative-scoring, system prompt 0.30
Gemma 2 2B IT generative-scoring, multi-turn, chat template 0.29
Gemma 2 2B IT log-likelihood, chat template 0.29
Qwen1.5-4B-Chat log-likelihood, chat template, system prompt 0.29
Qwen1.5-4B-Chat generative-scoring, chat template, system prompt 0.29
Qwen1.5-4B-Chat generative-scoring, chat template 0.28
Mistral-7B-Instruct-v0.2 generative-scoring, chat template 0.28
Yi-1.5-6B-Chat generative-scoring, multi-turn, chat template, system prompt 0.28
Qwen1.5-4B-Chat log-likelihood, chat template 0.28
Qwen1.5-7B generative-scoring, chat template 0.28
LFM2-8B-A1B generative-scoring, multi-turn, chat template, system prompt 0.28
Gemma 2 2B IT generative-scoring, chat template 0.27
Qwen1.5-4B-Chat generative-scoring 0.27
Yi-1.5-6B-Chat generative-scoring, chat template 0.27
Yi-1.5-6B-Chat log-likelihood, chat template 0.27
Qwen2-0.5B-Instruct log-likelihood 0.26
Qwen2-0.5B-Instruct log-likelihood, multi-turn, chat template 0.25
Qwen2-0.5B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.25
Llama 3.2 1B log-likelihood, system prompt 0.25
Qwen2-0.5B log-likelihood 0.25
Qwen2-0.5B-Instruct generative-scoring, multi-turn, chat template 0.25
Qwen2-0.5B-Instruct generative-scoring 0.24
Qwen2-0.5B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.24
Qwen2-0.5B generative-scoring 0.24
Llama 3.2 1B generative-scoring, system prompt 0.23
Qwen2-0.5B-Instruct log-likelihood, chat template 0.23
Qwen2-0.5B-Instruct log-likelihood, chat template 0.23
Yi-6B-Chat log-likelihood, chat template 0.23
Yi-6B-Chat log-likelihood, chat template 0.23
Qwen2-0.5B-Instruct generative-scoring, chat template 0.22
Qwen2-0.5B-Instruct generative-scoring, chat template 0.22
Qwen2-0.5B-Instruct log-likelihood, chat template, system prompt 0.22
Qwen2-0.5B-Instruct generative-scoring, chat template, system prompt 0.22
Yi-6B-Chat log-likelihood, system prompt 0.21
LFM2.5-1.2B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.20
Yi-6B-Chat generative-scoring, chat template 0.19
Yi-6B-Chat generative-scoring, chat template 0.19
Qwen2-1.5B log-likelihood, chat template 0.19
Qwen2-1.5B log-likelihood, chat template 0.19
Yi-6B-Chat generative-scoring, system prompt 0.19
Qwen2-7B generative-scoring, chat template 0.19
Qwen2-0.5B log-likelihood, chat template 0.18
Qwen2-0.5B log-likelihood, chat template 0.18
Qwen2.5-Math-72B-Instruct generative-scoring, chat template, system prompt 0.17
Qwen2-1.5B generative-scoring, chat template 0.10
Qwen2-1.5B generative-scoring, chat template 0.10
Qwen2-0.5B generative-scoring, chat template 0.06
Qwen2-0.5B generative-scoring, chat template 0.06

13 models have held number one on the official leaderboard since its release in November 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Mistral AI 01.AI Meta Qwen Google DeepMind Anthropic
Own leaderboard Not tested since Apr 2026

Source: Official MERA leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
0.00 0.20 0.40 0.60 0.80 1.00 2024 2025 2026 0.00 0.20 0.40 0.60 0.80 1.00 2024 2025 2026

Source: Official MERA leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Multilingual

All 545 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%

Data audit

8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (209 entries).
  • Pass Models scored 102 models we track, plus 107 other entries on its leaderboard.
  • Warning Newest score The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash.
  • Note Frontier models None of the top 10 has a score. The strongest tested is GPT-5.4 (#28).
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on mera.a-ai.ru: 277 rows for models we track and 107 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check