MERA
Models complete Russian-language tasks with several answer formats. The hub port grades short answers from Part 1 of the Russian Unified State Exam.
The 2024 MERA paper compares 19 models. Mistral led overall at 40.0, and ruGPT-3.5 led the original USE task at 2.5 on the 100 point scale.
- Made by
- MERA Evaluation team
- Released
- November 2023
- Kind
- Knowledge in many languages
- Level
- University
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- All public
- Measures
- Capability
- Subject
- Mathematics, Languages and literature
- Language
- Russian
- Interaction
- Single turn
- Items from
- Exams and contests
- Refresh
- New versions replace it
- Run by
- A host
- Strongest model tested
- GPT-5.4 #28 on the capability index
- Last model added
- April 2026
- Scores refreshed
- November 20th, 2026
Claude Opus 4.6 leads with 0.86 among models released by April 2026 Claude Opus 4.6 leads with 0.86 among models released by April 2026
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
The newest model with a score here came out in April 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 0.86 Claude Opus 4.6
- 0.83 BerryLM-XL
- 0.82 Cotype Pro 3
- 0.82 BerryLM-L-v2-early-ckpt
- 0.82 GPT-5.4
- 0.81 BerryLM-v2-reasoning-budget-low
- 0.80 GLM-5.1
- 0.80
Qwen3-235B-A22B-Thinking
- 0.79 Cotype Light 3
- 0.79
Qwen 3.6 35B-A3B
- 0.79
Qwen3.6 27B
- 0.78
Qwen3.5-27B
- 0.78 BerryLM-L
- 0.77
Qwen3.5-122B-A10B
- 0.76 Claude Opus 4.5
- 0.75
Qwen3-30B-A3B-Thinking
- 0.75 Kimi K2.5
- 0.74 BerryLM-MT
- 0.74 BerryLM
- 0.73
Qwen3-32B
- 0.73
Qwen3.5-35B-A3B
- 0.71 GigaChat-3.1-Ultra
- 0.71
Qwen3-Next-80B-A3B
- 0.71
Gemma 4 31B IT
- 0.71
Qwen3-4B-Thinking-2507
- 0.71 GPT-5.2
- 0.69
MiniMax-M2.7
- 0.69
Qwen3-14B
- 0.69
DeepSeek-V4-Pro
- 0.68 Claude 3.7 Sonnet
- 0.68
Gemini 2.0 Flash
- 0.68
Qwen3.5-9B
- 0.68
DeepSeek-V3
- 0.68
DeepSeek-V4-Flash
- 0.68
Gemini 1.5 Pro
- 0.67 Cotype Pro 2.5
- 0.67
Qwen3-4B
- 0.67 GigaChat 2 Max
- 0.66
Qwen2.5-Max
- 0.66 T-pro-it-2.0
- 0.65 GigaChat 2 Pro
- 0.65
Qwen3-8B
- 0.64 GPT-4o
- 0.64
Gemini 2.0 Flash-Lite
- 0.64 Meta-Llama-3.1-405B-Instruct
- 0.63 Cotype (preview)
- 0.63 T-pro-it-1.0
- 0.63
Gemini 1.5 Flash
- 0.62 Zero-Mistral-24B
- 0.62
Qwen2.5-72B
- 0.62 A-vibe
- 0.62
DeepSeek-R1-Distill-Qwen-32B
- 0.62 gpt-oss-20b
- 0.61 Nova Pro 1.0
- 0.61 RuadaptQwen-32B-instruct
- 0.61
Llama 4 Maverick
- 0.60
Gemma 4 26B A4B
- 0.60 Qwen2.5-32B-Instruct
- 0.60 Supernova-N1-32B-Preview
- 0.60 Meta-Llama-3.1-70B-Instruct
- 0.59 Mixtral-8x22B-Instruct-v0.1
- 0.59 Mistral-Large-Instruct-2407
- 0.59 GigaChat Max
- 0.59 Meta-Llama-3-70B-Instruct
- 0.58 gemma-2-27b-it
- 0.57
Llama 3.1-70B
- 0.57
Qwen2-72B
- 0.57 GPT-4o mini
- 0.57
Gemma 3 27B
- 0.57
Qwen3.5-4B
- 0.56
Llama 3-70B
- 0.56 Meno-Lite-0.1
- 0.56
Llama 3.3 70B Instruct
- 0.55 T-lite-it-1.0
- 0.55
Qwen-Turbo
- 0.55 Claude 3.5 Haiku
- 0.54 GigaChat 2
- 0.54
Gemini 1.5 Flash 8B
- 0.54 RuadaptQwen2.5-7B-Lite-v1
- 0.53 Qwen2.5 14B Instruct
- 0.53 Mistral Small 3.1 24B Instruct
- 0.53
Llama-3.1-Nemotron-70B-Instruct
- 0.53
Llama 4 Scout
- 0.52 gemma-2-9b-it
- 0.52 Zero-Mistral-Small-24B-Instruct-2501
- 0.52
Gemma 2 27B
- 0.52 Qwen2-57B-A14B-Instruct
- 0.52 Mixtral-8x7B-Instruct-v0.1
- 0.51 GigaChat-20B-A3B
- 0.51 GigaChat Pro
- 0.51 T-lite-0.1
- 0.51
phi-3-medium 14B
- 0.51
Phi-3-medium-128k-instruct
- 0.51
Qwen2.5-Math-72B-Instruct
- 0.51 Qwen2.5-Math-72B
- 0.50 GigaChat-3.1-Lightning
- 0.50
Phi-3.5-MoE
- 0.50 GigaChat
- 0.49 Qwen2-57B-A14B
- 0.49 Qwen2.5-7B-Instruct
- 0.49
Mixtral 8x7B
- 0.49
Qwen3-30B-A3B-Instruct
- 0.49 Meta-Llama-3.1-8B-Instruct
- 0.49 Qwen1.5-32B-Chat
- 0.48
Qwen1.5-32B
- 0.48
Gemma 4 E4B
- 0.48 Yi-1.5-34B-32K
- 0.48 Meta-Llama-3-8B-Instruct
- 0.47
Qwen2-7B-Instruct
- 0.47 Olmo-3.1-32B-Instruct-DPO
- 0.47
Yi-1.5-34B
- 0.47 qwen3-4b-instruct-2507
- 0.47
phi-3-small 7.4B
- 0.46 Qwen2.5-Coder-7B-Instruct
- 0.46 Olmo-3.1-32B-Instruct-SFT
- 0.46
Qwen2.5-7B
- 0.46
Command R (03-2024)
- 0.46
Qwen2-7B
- 0.45
Gemma 2 9B
- 0.45 ruadapt llama3-8B-instruct lep ft
- 0.45
Llama 3.1-8B
- 0.45
DeepSeek-R1-Distill-Qwen-14B
- 0.45
Llama 3-8B
- 0.44 Yi-1.5-34B-Chat-16K
- 0.44 Qwen2.5-3B-Instruct
- 0.44
Yi-34B
- 0.44 GPT-3.5 Turbo
- 0.44 Yi-1.5-34B-Chat
- 0.43 Mistral-7B-Instruct-v0.3
- 0.43
Olmo 3.1 32B Instruct
- 0.43 Yi-1.5-9B
- 0.43 Yi-1.5-9B-32K
- 0.43 Yi-34B-200K
- 0.42 Yi-34B-Chat
- 0.42 Watari-7b-v1
- 0.42
Mistral NeMo
- 0.41 ruadapt_qwen2.5_3B_ext_u48_instruct_v4
- 0.41
Qwen2.5-3B
- 0.41
Mistral-7B-Instruct-v0.2
- 0.40
Mistral 7B v0.1
- 0.40
Gemma 4 E2B
- 0.40 Yi-1.5-9B-Chat-16K
- 0.40
phi-3-mini 3.8B
- 0.40 Yi-Coder-9B-Chat
- 0.40
Mistral 7B v0.3
- 0.40
Phi-3-mini-128k-instruct
- 0.39
Qwen3-0.6B
- 0.39
Yi-1.5-9B-Chat
- 0.39
Gemma 2 2B IT
- 0.38 Gamayun-1.5B
- 0.38 Llama-3.2-3B-Instruct
- 0.37 Yi-Coder-9B
- 0.37
Qwen1.5-7B
- 0.37 LFM2-24B-A2B
- 0.37
Mistral-7B-Instruct-v0.1
- 0.37
Qwen1.5-7B-Chat
- 0.36 Meno-Tiny-0.1
- 0.36
Llama 3.2 3B
- 0.36 Qwen2.5 1.5B Instruct
- 0.36 Yi-1.5-6B
- 0.35 Qwen2.5-1.5B-Instruct
- 0.35 LFM-3B
- 0.35
Qwen3.5-2B
- 0.35 Qwen2.5-Math-7B-Instruct
- 0.34 Llama-2-13b-hf
- 0.34 Qwen2-Math-7B-Instruct
- 0.34 Olmo-3-7B-Instruct
- 0.34
Qwen2-1.5B-Instruct
- 0.34
Qwen3-1.7B
- 0.34 Qwen1.5-MoE-A2.7B-Chat
- 0.33
Yi-1.5-6B-Chat
- 0.33
Qwen2-1.5B
- 0.32 Qwen1.5-4B
- 0.32
Yi 6B
- 0.32
Qwen2.5-Coder (1.5B)
- 0.31 gemma-2-2b
- 0.31
Qwen1.5-4B-Chat
- 0.31 Qwen2.5-Math-1.5B
- 0.31 Qwen2.5-Coder-1.5B-Instruct
- 0.31 Yi-6B-200K
- 0.30 Llama-2-7b-chat-hf
- 0.30
Qwen3.5-0.8B
- 0.30 Llama-2-7b-hf
- 0.30 Llama-3.2-1B-Instruct
- 0.29 Qwen2.5-0.5B-Instruct
- 0.28 Qwen2.5-Math-1.5B-Instruct
- 0.28 LFM2-2.6B
- 0.28 LFM2-8B-A1B
- 0.27 Qwen1.5-1.8B
- 0.26 Qwen1.5-1.8B-Chat
- 0.26
Qwen2-0.5B-Instruct
- 0.25 Llama-2-13b-chat-hf
- 0.25
Llama 3.2 1B
- 0.25
Qwen2-0.5B
- 0.24 LFM2-700M
- 0.23
Yi-6B-Chat
- 0.22 Megra 1.1 709M (SFT-4096)
- 0.22 Qwen1.5-0.5B
- 0.22 Qwen1.5-0.5B-Chat
- 0.21 LFM2-350M
- 0.21 ruGPT-3.5
- 0.21 ruGPT3-medium
- 0.20 ruGPT3-large
- 0.20 ruGPT-3-small
- 0.20 LFM2.5-1.2B-Instruct
- 0.20 ruGPT-3 XL 1.3B
- 0.19 ruT5-base
- 0.19 Qwen2-Math-1.5B-Instruct
- 0.17 ruGPT-3-medium
- 1 Claude Opus 4.6 generative-scoring, multi-turn, chat template, system prompt 0.86
- 2 BerryLM-XL generative-scoring, multi-turn, chat template, system prompt 0.83
- 3 Cotype Pro 3 generative-scoring, multi-turn, chat template, system prompt 0.82
- 4 BerryLM-L-v2-early-ckpt generative-scoring, multi-turn, system prompt 0.82
- 5 GPT-5.4 generative-scoring, multi-turn, chat template, system prompt 0.82
- 6 BerryLM-v2-reasoning-budget-low generative-scoring, multi-turn, chat template, system prompt 0.81
- 7 GLM-5.1 generative-scoring, multi-turn, chat template, system prompt 0.80
- 8
Qwen3-235B-A22B-Thinking Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.80
- 9 Cotype Light 3 generative-scoring, multi-turn, chat template, system prompt 0.79
- 10
Qwen 3.6 35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.79
- 11
Qwen3.6 27B generative-scoring, multi-turn, chat template, system prompt 0.79
- 12
Qwen3.5-27B generative-scoring, multi-turn, chat template, system prompt 0.78
- 13 BerryLM-L generative-scoring, multi-turn, chat template, system prompt 0.78
- 14
Qwen3.5-122B-A10B generative-scoring, multi-turn, chat template, system prompt 0.77
- 15 Claude Opus 4.5 generative-scoring, multi-turn, chat template, system prompt 0.76
- 16
Qwen3-30B-A3B-Thinking Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.75
- 17 Kimi K2.5 generative-scoring, multi-turn, chat template, system prompt 0.75
- 18 BerryLM-MT generative-scoring, multi-turn, chat template, system prompt 0.74
- 19 BerryLM generative-scoring, chat template, system prompt 0.74
- 20
Qwen3-32B generative-scoring, multi-turn, chat template, system prompt 0.73
- 21
Qwen3.5-35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.73
- 22 GigaChat-3.1-Ultra generative-scoring, multi-turn, chat template, system prompt 0.71
- 23
Qwen3-Next-80B-A3B generative-scoring, multi-turn, chat template, system prompt 0.71
- 24
Gemma 4 31B IT generative-scoring, multi-turn, chat template, system prompt 0.71
- 25
Qwen3-4B-Thinking-2507 generative-scoring, multi-turn, chat template, system prompt 0.71
- 26 GPT-5.2 generative-scoring, multi-turn, chat template, system prompt 0.71
- 27
MiniMax-M2.7 generative-scoring, multi-turn 0.69
- 28
Qwen3-14B generative-scoring, multi-turn, chat template, system prompt 0.69
- 29
DeepSeek-V4-Pro generative-scoring, multi-turn, chat template, system prompt 0.69
- 30 Claude 3.7 Sonnet generative-scoring, multi-turn, chat template, system prompt 0.68
- 31
Gemini 2.0 Flash Feb 2025, generative-scoring, multi-turn, chat template, system prompt 0.68
- 32
Qwen3.5-9B generative-scoring, multi-turn, chat template, system prompt 0.68
- 33
DeepSeek-V3 Mar 2025, generative-scoring, multi-turn, chat template, system prompt 0.68
- 34
DeepSeek-V4-Flash generative-scoring, multi-turn, chat template, system prompt 0.68
- 35
Gemini 1.5 Pro Sep 2024, generative-scoring, multi-turn, chat template, system prompt 0.68
- 36 Cotype Pro 2.5 generative-scoring, multi-turn, chat template, system prompt 0.67
- 37
Qwen3-4B generative-scoring, multi-turn, chat template, system prompt 0.67
- 38 GigaChat 2 Max generative-scoring, multi-turn, chat template, system prompt 0.67
- 39
Qwen2.5-Max generative-scoring, multi-turn, chat template, system prompt 0.66
- 40 T-pro-it-2.0 generative-scoring, multi-turn, chat template, system prompt 0.66
- 41 GigaChat 2 Pro generative-scoring, multi-turn, chat template, system prompt 0.65
- 42
Qwen3-8B generative-scoring, multi-turn, chat template, system prompt 0.65
- 43 GPT-4o Mar 2025, generative-scoring, multi-turn, chat template, system prompt 0.64
- 44
Gemini 2.0 Flash-Lite generative-scoring, multi-turn, chat template, system prompt 0.64
- 45 Meta-Llama-3.1-405B-Instruct log-likelihood 0.64
- 46 Cotype (preview) generative-scoring, multi-turn, chat template, system prompt 0.63
- 47 T-pro-it-1.0 generative-scoring, multi-turn, chat template, system prompt 0.63
- 48
Gemini 1.5 Flash Sep 2024, generative-scoring, multi-turn, chat template, system prompt 0.63
- 49 Zero-Mistral-24B generative-scoring, multi-turn, chat template, system prompt 0.62
- 50
Qwen2.5-72B log-likelihood, system prompt 0.62
- 51 A-vibe generative-scoring, multi-turn, chat template, system prompt 0.62
- 52
DeepSeek-R1-Distill-Qwen-32B generative-scoring, chat template, system prompt 0.62
- 53 gpt-oss-20b generative-scoring, multi-turn, chat template, system prompt 0.62
- 54 Nova Pro 1.0 generative-scoring, multi-turn, chat template, system prompt 0.61
- 55 RuadaptQwen-32B-instruct generative-scoring, multi-turn, chat template, system prompt 0.61
- 56
Llama 4 Maverick generative-scoring, multi-turn, chat template, system prompt 0.61
- 57
Gemma 4 26B A4B generative-scoring, multi-turn, chat template, system prompt 0.60
- 58 Qwen2.5-32B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.60
- 59 Supernova-N1-32B-Preview generative-scoring, multi-turn, chat template, system prompt 0.60
- 60 Meta-Llama-3.1-70B-Instruct log-likelihood 0.60
- 61 Mixtral-8x22B-Instruct-v0.1 log-likelihood 0.59
- 62 Mistral-Large-Instruct-2407 log-likelihood, chat template, system prompt 0.59
- 63 GigaChat Max generative-scoring, multi-turn, chat template, system prompt 0.59
- 64 Meta-Llama-3-70B-Instruct log-likelihood 0.59
- 65 gemma-2-27b-it log-likelihood 0.58
- 66
Llama 3.1-70B log-likelihood 0.57
- 67
Qwen2-72B generative-scoring, multi-turn, chat template, system prompt 0.57
- 68 GPT-4o mini generative-scoring, multi-turn, chat template, system prompt 0.57
- 69
Gemma 3 27B generative-scoring, multi-turn, chat template, system prompt 0.57
- 70
Qwen3.5-4B generative-scoring, multi-turn, chat template, system prompt 0.57
- 71
Llama 3-70B generative-scoring 0.56
- 72 Meno-Lite-0.1 generative-scoring, multi-turn, chat template, system prompt 0.56
- 73
Llama 3.3 70B Instruct generative-scoring, multi-turn, chat template, system prompt 0.56
- 74 T-lite-it-1.0 generative-scoring, multi-turn, chat template, system prompt 0.55
- 75
Qwen-Turbo generative-scoring, multi-turn, chat template, system prompt 0.55
- 76 Claude 3.5 Haiku generative-scoring, multi-turn, chat template, system prompt 0.55
- 77 GigaChat 2 generative-scoring, multi-turn, chat template, system prompt 0.54
- 78
Gemini 1.5 Flash 8B generative-scoring, multi-turn, chat template, system prompt 0.54
- 79 RuadaptQwen2.5-7B-Lite-v1 generative-scoring, multi-turn, chat template, system prompt 0.54
- 80 Qwen2.5 14B Instruct generative-scoring, chat template, system prompt 0.53
- 81 Mistral Small 3.1 24B Instruct generative-scoring, multi-turn, chat template, system prompt 0.53
- 82
Llama-3.1-Nemotron-70B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.53
- 83
Llama 4 Scout generative-scoring, multi-turn, chat template, system prompt 0.53
- 84 gemma-2-9b-it log-likelihood 0.52
- 85 Zero-Mistral-Small-24B-Instruct-2501 generative-scoring, multi-turn, chat template, system prompt 0.52
- 86
Gemma 2 27B log-likelihood 0.52
- 87 Qwen2-57B-A14B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.52
- 88 Mixtral-8x7B-Instruct-v0.1 log-likelihood 0.52
- 89 GigaChat-20B-A3B generative-scoring, multi-turn, chat template 0.51
- 90 GigaChat Pro generative-scoring, multi-turn, chat template, system prompt 0.51
- 91 T-lite-0.1 log-likelihood 0.51
- 92
phi-3-medium 14B log-likelihood 0.51
- 93
Phi-3-medium-128k-instruct log-likelihood 0.51
- 94
Qwen2.5-Math-72B-Instruct log-likelihood, system prompt 0.51
- 95 Qwen2.5-Math-72B log-likelihood, system prompt 0.51
- 96 GigaChat-3.1-Lightning generative-scoring, multi-turn, chat template, system prompt 0.50
- 97
Phi-3.5-MoE log-likelihood 0.50
- 98 GigaChat generative-scoring, multi-turn, chat template, system prompt 0.50
- 99 Qwen2-57B-A14B log-likelihood 0.49
- 100 Qwen2.5-7B-Instruct log-likelihood, system prompt 0.49
- 101
Mixtral 8x7B log-likelihood 0.49
- 102
Qwen3-30B-A3B-Instruct Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.49
- 103 Meta-Llama-3.1-8B-Instruct log-likelihood 0.49
- 104 Qwen1.5-32B-Chat log-likelihood, multi-turn, chat template 0.49
- 105
Qwen1.5-32B log-likelihood 0.48
- 106
Gemma 4 E4B generative-scoring, multi-turn, chat template, system prompt 0.48
- 107 Yi-1.5-34B-32K log-likelihood 0.48
- 108 Meta-Llama-3-8B-Instruct log-likelihood 0.48
- 109
Qwen2-7B-Instruct log-likelihood 0.47
- 110 Olmo-3.1-32B-Instruct-DPO generative-scoring, multi-turn, chat template, system prompt 0.47
- 111
Yi-1.5-34B log-likelihood 0.47
- 112 qwen3-4b-instruct-2507 generative-scoring, multi-turn, chat template, system prompt 0.47
- 113
phi-3-small 7.4B log-likelihood 0.47
- 114 Qwen2.5-Coder-7B-Instruct log-likelihood, multi-turn, chat template, system prompt 0.46
- 115 Olmo-3.1-32B-Instruct-SFT generative-scoring, multi-turn, chat template, system prompt 0.46
- 116
Qwen2.5-7B log-likelihood, system prompt 0.46
- 117
Command R (03-2024) generative-scoring, multi-turn, chat template, system prompt 0.46
- 118
Qwen2-7B log-likelihood 0.46
- 119
Gemma 2 9B log-likelihood 0.45
- 120 ruadapt llama3-8B-instruct lep ft log-likelihood, multi-turn, chat template, system prompt 0.45
- 121
Llama 3.1-8B log-likelihood 0.45
- 122
DeepSeek-R1-Distill-Qwen-14B log-likelihood 0.45
- 123
Llama 3-8B generative-scoring 0.45
- 124 Yi-1.5-34B-Chat-16K log-likelihood 0.44
- 125 Qwen2.5-3B-Instruct log-likelihood, system prompt 0.44
- 126
Yi-34B log-likelihood 0.44
- 127 GPT-3.5 Turbo Jan 2024, generative-scoring, multi-turn, chat template 0.44
- 128 Yi-1.5-34B-Chat log-likelihood 0.44
- 129 Mistral-7B-Instruct-v0.3 log-likelihood 0.43
- 130
Olmo 3.1 32B Instruct generative-scoring, multi-turn, chat template, system prompt 0.43
- 131 Yi-1.5-9B generative-scoring 0.43
- 132 Yi-1.5-9B-32K log-likelihood 0.43
- 133 Yi-34B-200K log-likelihood 0.43
- 134 Yi-34B-Chat log-likelihood 0.42
- 135 Watari-7b-v1 generative-scoring, chat template, system prompt 0.42
- 136
Mistral NeMo generative-scoring, multi-turn, chat template, system prompt 0.42
- 137 ruadapt_qwen2.5_3B_ext_u48_instruct_v4 generative-scoring, multi-turn, chat template, system prompt 0.41
- 138
Qwen2.5-3B log-likelihood, system prompt 0.41
- 139
Mistral-7B-Instruct-v0.2 log-likelihood 0.41
- 140
Mistral 7B v0.1 log-likelihood 0.40
- 141
Gemma 4 E2B generative-scoring, multi-turn, chat template, system prompt 0.40
- 142 Yi-1.5-9B-Chat-16K log-likelihood 0.40
- 143
phi-3-mini 3.8B log-likelihood 0.40
- 144 Yi-Coder-9B-Chat log-likelihood, system prompt 0.40
- 145
Mistral 7B v0.3 log-likelihood 0.40
- 146
Phi-3-mini-128k-instruct log-likelihood 0.40
- 147
Qwen3-0.6B generative-scoring, multi-turn, chat template, system prompt 0.39
- 148
Yi-1.5-9B-Chat log-likelihood 0.39
- 149
Gemma 2 2B IT log-likelihood 0.39
- 150 Gamayun-1.5B log-likelihood 0.38
- 151 Llama-3.2-3B-Instruct log-likelihood, system prompt 0.38
- 152 Yi-Coder-9B log-likelihood, system prompt 0.37
- 153
Qwen1.5-7B log-likelihood 0.37
- 154 LFM2-24B-A2B generative-scoring, multi-turn, chat template, system prompt 0.37
- 155
Mistral-7B-Instruct-v0.1 log-likelihood 0.37
- 156
Qwen1.5-7B-Chat log-likelihood 0.37
- 157 Meno-Tiny-0.1 generative-scoring, multi-turn, chat template, system prompt 0.36
- 158
Llama 3.2 3B log-likelihood, system prompt 0.36
- 159 Qwen2.5 1.5B Instruct generative-scoring, multi-turn, chat template, system prompt 0.36
- 160 Yi-1.5-6B log-likelihood 0.36
- 161 Qwen2.5-1.5B-Instruct log-likelihood, system prompt 0.35
- 162 LFM-3B generative-scoring, multi-turn, chat template, system prompt 0.35
- 163
Qwen3.5-2B generative-scoring, multi-turn, chat template, system prompt 0.35
- 164 Qwen2.5-Math-7B-Instruct log-likelihood, system prompt 0.35
- 165 Llama-2-13b-hf log-likelihood 0.34
- 166 Qwen2-Math-7B-Instruct log-likelihood, chat template, system prompt 0.34
- 167 Olmo-3-7B-Instruct generative-scoring, multi-turn, chat template 0.34
- 168
Qwen2-1.5B-Instruct log-likelihood 0.34
- 169
Qwen3-1.7B generative-scoring, multi-turn, chat template, system prompt 0.34
- 170 Qwen1.5-MoE-A2.7B-Chat log-likelihood, multi-turn, chat template, system prompt 0.34
- 171
Yi-1.5-6B-Chat log-likelihood 0.33
- 172
Qwen2-1.5B log-likelihood 0.33
- 173 Qwen1.5-4B log-likelihood 0.32
- 174
Yi 6B generative-scoring 0.32
- 175
Qwen2.5-Coder (1.5B) log-likelihood, system prompt 0.32
- 176 gemma-2-2b log-likelihood 0.31
- 177
Qwen1.5-4B-Chat log-likelihood, multi-turn, chat template, system prompt 0.31
- 178 Qwen2.5-Math-1.5B log-likelihood, system prompt 0.31
- 179 Qwen2.5-Coder-1.5B-Instruct log-likelihood, system prompt 0.31
- 180 Yi-6B-200K log-likelihood 0.31
- 181 Llama-2-7b-chat-hf log-likelihood 0.30
- 182
Qwen3.5-0.8B generative-scoring, multi-turn, chat template, system prompt 0.30
- 183 Llama-2-7b-hf log-likelihood 0.30
- 184 Llama-3.2-1B-Instruct log-likelihood, system prompt 0.30
- 185 Qwen2.5-0.5B-Instruct log-likelihood, system prompt 0.29
- 186 Qwen2.5-Math-1.5B-Instruct log-likelihood, system prompt 0.28
- 187 LFM2-2.6B generative-scoring, multi-turn, chat template, system prompt 0.28
- 188 LFM2-8B-A1B generative-scoring, multi-turn, chat template, system prompt 0.28
- 189 Qwen1.5-1.8B log-likelihood 0.27
- 190 Qwen1.5-1.8B-Chat log-likelihood 0.26
- 191
Qwen2-0.5B-Instruct log-likelihood 0.26
- 192 Llama-2-13b-chat-hf log-likelihood, multi-turn, chat template, system prompt 0.25
- 193
Llama 3.2 1B log-likelihood, system prompt 0.25
- 194
Qwen2-0.5B log-likelihood 0.25
- 195 LFM2-700M generative-scoring, multi-turn, chat template, system prompt 0.24
- 196
Yi-6B-Chat log-likelihood, chat template 0.23
- 197 Megra 1.1 709M (SFT-4096) log-likelihood 0.22
- 198 Qwen1.5-0.5B log-likelihood 0.22
- 199 Qwen1.5-0.5B-Chat log-likelihood, multi-turn, chat template, system prompt 0.22
- 200 LFM2-350M generative-scoring, multi-turn, chat template, system prompt 0.21
- 201 ruGPT-3.5 log-likelihood 0.21
- 202 ruGPT3-medium log-likelihood 0.21
- 203 ruGPT3-large log-likelihood 0.20
- 204 ruGPT-3-small log-likelihood 0.20
- 205 LFM2.5-1.2B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.20
- 206 ruGPT-3 XL 1.3B log-likelihood 0.20
- 207 ruT5-base log-likelihood 0.19
- 208 Qwen2-Math-1.5B-Instruct log-likelihood, chat template, system prompt 0.19
- 209 ruGPT-3-medium generative-scoring 0.17
Source: Official MERA leaderboard
- 0.86 Claude Opus 4.6
- 0.82 GPT-5.4
- 0.80 GLM-5.1
- 0.80
Qwen3-235B-A22B-Thinking
- 0.79
Qwen 3.6 35B-A3B
- 0.79
Qwen3.6 27B
- 0.78
Qwen3.5-27B
- 0.77
Qwen3.5-122B-A10B
- 0.76 Claude Opus 4.5
- 0.75
Qwen3-30B-A3B-Thinking
- 0.75 Kimi K2.5
- 0.73
Qwen3-32B
- 0.73
Qwen3.5-35B-A3B
- 0.71
Qwen3-Next-80B-A3B
- 0.71
Gemma 4 31B IT
- 0.71
Qwen3-4B-Thinking-2507
- 0.71 GPT-5.2
- 0.69
MiniMax-M2.7
- 0.69
Qwen3-14B
- 0.69
DeepSeek-V4-Pro
- 0.68 Claude 3.7 Sonnet
- 0.68
Gemini 2.0 Flash
- 0.68
Qwen3.5-9B
- 0.68
DeepSeek-V3
- 0.68
DeepSeek-V4-Flash
- 0.68
Gemini 1.5 Pro
- 0.67
Qwen3-4B
- 0.66
Qwen2.5-Max
- 0.65
Qwen3-8B
- 0.64 GPT-4o
- 0.64
Gemini 2.0 Flash-Lite
- 0.63
Gemini 1.5 Flash
- 0.62
Qwen2.5-72B
- 0.62
DeepSeek-R1-Distill-Qwen-32B
- 0.62 gpt-oss-20b
- 0.61
Llama 4 Maverick
- 0.60
Gemma 4 26B A4B
- 0.57
Llama 3.1-70B
- 0.57
Qwen2-72B
- 0.57 GPT-4o mini
- 0.57
Gemma 3 27B
- 0.57
Qwen3.5-4B
- 0.56
Llama 3-70B
- 0.56
Llama 3.3 70B Instruct
- 0.55
Qwen-Turbo
- 0.55 Claude 3.5 Haiku
- 0.54
Gemini 1.5 Flash 8B
- 0.53
Llama-3.1-Nemotron-70B-Instruct
- 0.53
Llama 4 Scout
- 0.52
Gemma 2 27B
- 0.51
phi-3-medium 14B
- 0.51
Phi-3-medium-128k-instruct
- 0.51
Qwen2.5-Math-72B-Instruct
- 0.50
Phi-3.5-MoE
- 0.49
Mixtral 8x7B
- 0.49
Qwen3-30B-A3B-Instruct
- 0.48
Qwen1.5-32B
- 0.48
Gemma 4 E4B
- 0.47
Qwen2-7B-Instruct
- 0.47
Yi-1.5-34B
- 0.47 qwen3-4b-instruct-2507
- 0.47
phi-3-small 7.4B
- 0.46
Qwen2.5-7B
- 0.46
Command R (03-2024)
- 0.46
Qwen2-7B
- 0.45
Gemma 2 9B
- 0.45
Llama 3.1-8B
- 0.45
DeepSeek-R1-Distill-Qwen-14B
- 0.45
Llama 3-8B
- 0.44
Yi-34B
- 0.44 GPT-3.5 Turbo
- 0.43
Olmo 3.1 32B Instruct
- 0.42
Mistral NeMo
- 0.41
Qwen2.5-3B
- 0.41
Mistral-7B-Instruct-v0.2
- 0.40
Mistral 7B v0.1
- 0.40
Gemma 4 E2B
- 0.40
phi-3-mini 3.8B
- 0.40
Mistral 7B v0.3
- 0.40
Phi-3-mini-128k-instruct
- 0.39
Qwen3-0.6B
- 0.39
Yi-1.5-9B-Chat
- 0.39
Gemma 2 2B IT
- 0.37
Qwen1.5-7B
- 0.37
Mistral-7B-Instruct-v0.1
- 0.37
Qwen1.5-7B-Chat
- 0.36
Llama 3.2 3B
- 0.35
Qwen3.5-2B
- 0.34
Qwen2-1.5B-Instruct
- 0.34
Qwen3-1.7B
- 0.33
Yi-1.5-6B-Chat
- 0.33
Qwen2-1.5B
- 0.32
Yi 6B
- 0.32
Qwen2.5-Coder (1.5B)
- 0.31
Qwen1.5-4B-Chat
- 0.30
Qwen3.5-0.8B
- 0.28 LFM2-8B-A1B
- 0.26
Qwen2-0.5B-Instruct
- 0.25
Llama 3.2 1B
- 0.25
Qwen2-0.5B
- 0.23
Yi-6B-Chat
- 0.20 LFM2.5-1.2B-Instruct
- 1 Claude Opus 4.6 generative-scoring, multi-turn, chat template, system prompt 0.86
- 2 GPT-5.4 generative-scoring, multi-turn, chat template, system prompt 0.82
- 3 GLM-5.1 generative-scoring, multi-turn, chat template, system prompt 0.80
- 4
Qwen3-235B-A22B-Thinking Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.80
- 5
Qwen 3.6 35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.79
- 6
Qwen3.6 27B generative-scoring, multi-turn, chat template, system prompt 0.79
- 7
Qwen3.5-27B generative-scoring, multi-turn, chat template, system prompt 0.78
- 8
Qwen3.5-122B-A10B generative-scoring, multi-turn, chat template, system prompt 0.77
- 9 Claude Opus 4.5 generative-scoring, multi-turn, chat template, system prompt 0.76
- 10
Qwen3-30B-A3B-Thinking Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.75
- 11 Kimi K2.5 generative-scoring, multi-turn, chat template, system prompt 0.75
- 12
Qwen3-32B generative-scoring, multi-turn, chat template, system prompt 0.73
- 13
Qwen3.5-35B-A3B generative-scoring, multi-turn, chat template, system prompt 0.73
- 14
Qwen3-Next-80B-A3B generative-scoring, multi-turn, chat template, system prompt 0.71
- 15
Gemma 4 31B IT generative-scoring, multi-turn, chat template, system prompt 0.71
- 16
Qwen3-4B-Thinking-2507 generative-scoring, multi-turn, chat template, system prompt 0.71
- 17 GPT-5.2 generative-scoring, multi-turn, chat template, system prompt 0.71
- 18
MiniMax-M2.7 generative-scoring, multi-turn 0.69
- 19
Qwen3-14B generative-scoring, multi-turn, chat template, system prompt 0.69
- 20
DeepSeek-V4-Pro generative-scoring, multi-turn, chat template, system prompt 0.69
- 21 Claude 3.7 Sonnet generative-scoring, multi-turn, chat template, system prompt 0.68
- 22
Gemini 2.0 Flash Feb 2025, generative-scoring, multi-turn, chat template, system prompt 0.68
- 23
Qwen3.5-9B generative-scoring, multi-turn, chat template, system prompt 0.68
- 24
DeepSeek-V3 Mar 2025, generative-scoring, multi-turn, chat template, system prompt 0.68
- 25
DeepSeek-V4-Flash generative-scoring, multi-turn, chat template, system prompt 0.68
- 26
Gemini 1.5 Pro Sep 2024, generative-scoring, multi-turn, chat template, system prompt 0.68
- 27
Qwen3-4B generative-scoring, multi-turn, chat template, system prompt 0.67
- 28
Qwen2.5-Max generative-scoring, multi-turn, chat template, system prompt 0.66
- 29
Qwen3-8B generative-scoring, multi-turn, chat template, system prompt 0.65
- 30 GPT-4o Mar 2025, generative-scoring, multi-turn, chat template, system prompt 0.64
- 31
Gemini 2.0 Flash-Lite generative-scoring, multi-turn, chat template, system prompt 0.64
- 32
Gemini 1.5 Flash Sep 2024, generative-scoring, multi-turn, chat template, system prompt 0.63
- 33
Qwen2.5-72B log-likelihood, system prompt 0.62
- 34
DeepSeek-R1-Distill-Qwen-32B generative-scoring, chat template, system prompt 0.62
- 35 gpt-oss-20b generative-scoring, multi-turn, chat template, system prompt 0.62
- 36
Llama 4 Maverick generative-scoring, multi-turn, chat template, system prompt 0.61
- 37
Gemma 4 26B A4B generative-scoring, multi-turn, chat template, system prompt 0.60
- 38
Llama 3.1-70B log-likelihood 0.57
- 39
Qwen2-72B generative-scoring, multi-turn, chat template, system prompt 0.57
- 40 GPT-4o mini generative-scoring, multi-turn, chat template, system prompt 0.57
- 41
Gemma 3 27B generative-scoring, multi-turn, chat template, system prompt 0.57
- 42
Qwen3.5-4B generative-scoring, multi-turn, chat template, system prompt 0.57
- 43
Llama 3-70B generative-scoring 0.56
- 44
Llama 3.3 70B Instruct generative-scoring, multi-turn, chat template, system prompt 0.56
- 45
Qwen-Turbo generative-scoring, multi-turn, chat template, system prompt 0.55
- 46 Claude 3.5 Haiku generative-scoring, multi-turn, chat template, system prompt 0.55
- 47
Gemini 1.5 Flash 8B generative-scoring, multi-turn, chat template, system prompt 0.54
- 48
Llama-3.1-Nemotron-70B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.53
- 49
Llama 4 Scout generative-scoring, multi-turn, chat template, system prompt 0.53
- 50
Gemma 2 27B log-likelihood 0.52
- 51
phi-3-medium 14B log-likelihood 0.51
- 52
Phi-3-medium-128k-instruct log-likelihood 0.51
- 53
Qwen2.5-Math-72B-Instruct log-likelihood, system prompt 0.51
- 54
Phi-3.5-MoE log-likelihood 0.50
- 55
Mixtral 8x7B log-likelihood 0.49
- 56
Qwen3-30B-A3B-Instruct Jul 2025, generative-scoring, multi-turn, chat template, system prompt 0.49
- 57
Qwen1.5-32B log-likelihood 0.48
- 58
Gemma 4 E4B generative-scoring, multi-turn, chat template, system prompt 0.48
- 59
Qwen2-7B-Instruct log-likelihood 0.47
- 60
Yi-1.5-34B log-likelihood 0.47
- 61 qwen3-4b-instruct-2507 generative-scoring, multi-turn, chat template, system prompt 0.47
- 62
phi-3-small 7.4B log-likelihood 0.47
- 63
Qwen2.5-7B log-likelihood, system prompt 0.46
- 64
Command R (03-2024) generative-scoring, multi-turn, chat template, system prompt 0.46
- 65
Qwen2-7B log-likelihood 0.46
- 66
Gemma 2 9B log-likelihood 0.45
- 67
Llama 3.1-8B log-likelihood 0.45
- 68
DeepSeek-R1-Distill-Qwen-14B log-likelihood 0.45
- 69
Llama 3-8B generative-scoring 0.45
- 70
Yi-34B log-likelihood 0.44
- 71 GPT-3.5 Turbo Jan 2024, generative-scoring, multi-turn, chat template 0.44
- 72
Olmo 3.1 32B Instruct generative-scoring, multi-turn, chat template, system prompt 0.43
- 73
Mistral NeMo generative-scoring, multi-turn, chat template, system prompt 0.42
- 74
Qwen2.5-3B log-likelihood, system prompt 0.41
- 75
Mistral-7B-Instruct-v0.2 log-likelihood 0.41
- 76
Mistral 7B v0.1 log-likelihood 0.40
- 77
Gemma 4 E2B generative-scoring, multi-turn, chat template, system prompt 0.40
- 78
phi-3-mini 3.8B log-likelihood 0.40
- 79
Mistral 7B v0.3 log-likelihood 0.40
- 80
Phi-3-mini-128k-instruct log-likelihood 0.40
- 81
Qwen3-0.6B generative-scoring, multi-turn, chat template, system prompt 0.39
- 82
Yi-1.5-9B-Chat log-likelihood 0.39
- 83
Gemma 2 2B IT log-likelihood 0.39
- 84
Qwen1.5-7B log-likelihood 0.37
- 85
Mistral-7B-Instruct-v0.1 log-likelihood 0.37
- 86
Qwen1.5-7B-Chat log-likelihood 0.37
- 87
Llama 3.2 3B log-likelihood, system prompt 0.36
- 88
Qwen3.5-2B generative-scoring, multi-turn, chat template, system prompt 0.35
- 89
Qwen2-1.5B-Instruct log-likelihood 0.34
- 90
Qwen3-1.7B generative-scoring, multi-turn, chat template, system prompt 0.34
- 91
Yi-1.5-6B-Chat log-likelihood 0.33
- 92
Qwen2-1.5B log-likelihood 0.33
- 93
Yi 6B generative-scoring 0.32
- 94
Qwen2.5-Coder (1.5B) log-likelihood, system prompt 0.32
- 95
Qwen1.5-4B-Chat log-likelihood, multi-turn, chat template, system prompt 0.31
- 96
Qwen3.5-0.8B generative-scoring, multi-turn, chat template, system prompt 0.30
- 97 LFM2-8B-A1B generative-scoring, multi-turn, chat template, system prompt 0.28
- 98
Qwen2-0.5B-Instruct log-likelihood 0.26
- 99
Llama 3.2 1B log-likelihood, system prompt 0.25
- 100
Qwen2-0.5B log-likelihood 0.25
- 101
Yi-6B-Chat log-likelihood, chat template 0.23
- 102 LFM2.5-1.2B-Instruct generative-scoring, multi-turn, chat template, system prompt 0.20
Source: Official MERA leaderboard
| # | Entry | Score | Setting | Submitted by | Date | Link | Source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.6 | 0.86 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 2 | BerryLM-XL closed weights, 358B | 0.83 | generative-scoring, multi-turn, chat template, system prompt | Wildberries & Russ | mera.a-ai.ru | Official leaderboard | |
| 3 | Cotype Pro 3 closed weights, 27B | 0.82 | generative-scoring, multi-turn, chat template, system prompt | MWS AI | mera.a-ai.ru | Official leaderboard | |
| 4 | BerryLM-L-v2-early-ckpt closed weights, 0B | 0.82 | generative-scoring, multi-turn, system prompt | Wildberrie&Russ | mera.a-ai.ru | Official leaderboard | |
| 5 | GPT-5.4 | 0.82 | generative-scoring, multi-turn, chat template, system prompt | MERA | Mar 2026 | mera.a-ai.ru | Official leaderboard |
| 6 | BerryLM-v2-reasoning-budget-low closed weights, 30B | 0.81 | generative-scoring, multi-turn, chat template, system prompt | Wildberries & Russ | mera.a-ai.ru | Official leaderboard | |
| 7 | GLM-5.1 | 0.80 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 8 | | 0.80 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jul 2025 | mera.a-ai.ru | Official leaderboard |
| 9 | Cotype Light 3 closed weights, 9B | 0.79 | generative-scoring, multi-turn, chat template, system prompt | MWS AI | mera.a-ai.ru | Official leaderboard | |
| 10 | | 0.79 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 11 | | 0.79 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 12 | | 0.78 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 13 | BerryLM-L closed weights, 120B | 0.78 | generative-scoring, multi-turn, chat template, system prompt | Wildberries & Russ | mera.a-ai.ru | Official leaderboard | |
| 14 | | 0.77 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 15 | Claude Opus 4.5 | 0.76 | generative-scoring, multi-turn, chat template, system prompt | Claude Opus 4.5 | Nov 2025 | mera.a-ai.ru | Official leaderboard |
| 16 | | 0.75 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jul 2025 | mera.a-ai.ru | Official leaderboard |
| 17 | Kimi K2.5 | 0.75 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jan 2026 | mera.a-ai.ru | Official leaderboard |
| 18 | BerryLM-MT closed weights, 20B | 0.74 | generative-scoring, multi-turn, chat template, system prompt | Wildberries & Russ | mera.a-ai.ru | Official leaderboard | |
| 19 | BerryLM closed weights, 20B | 0.74 | generative-scoring, chat template, system prompt | Wildberries & Russ | mera.a-ai.ru | Official leaderboard | |
| 20 | | 0.73 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 21 | | 0.73 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 22 | GigaChat-3.1-Ultra open weights, 702B | 0.71 | generative-scoring, multi-turn, chat template, system prompt | GigaChat | mera.a-ai.ru | Official leaderboard | |
| 23 | | 0.71 | generative-scoring, multi-turn, chat template, system prompt | MERA | Sep 2025 | mera.a-ai.ru | Official leaderboard |
| 24 | | 0.71 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 25 | | 0.71 | generative-scoring, multi-turn, chat template, system prompt | MERA | Aug 2025 | mera.a-ai.ru | Official leaderboard |
| 26 | GPT-5.2 | 0.71 | generative-scoring, multi-turn, chat template, system prompt | GPT-5.2 | Dec 2025 | mera.a-ai.ru | Official leaderboard |
| 27 | | 0.69 | generative-scoring, multi-turn | MERA | Mar 2026 | mera.a-ai.ru | Official leaderboard |
| 28 | | 0.69 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 29 | | 0.69 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 30 | Claude 3.7 Sonnet | 0.68 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Feb 2025 | mera.a-ai.ru | Official leaderboard |
| 31 | | 0.68 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Feb 2025 | mera.a-ai.ru | Official leaderboard |
| 32 | | 0.68 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 33 | | 0.68 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Mar 2025 | mera.a-ai.ru | Official leaderboard |
| 34 | | 0.68 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 35 | | 0.68 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 36 | Cotype Pro 2.5 closed weights, 32.5B | 0.67 | generative-scoring, multi-turn, chat template, system prompt | MWS AI | mera.a-ai.ru | Official leaderboard | |
| 37 | | 0.67 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 38 | GigaChat 2 Max closed weights | 0.67 | generative-scoring, multi-turn, chat template, system prompt | GIGACHAT | mera.a-ai.ru | Official leaderboard | |
| 39 | | 0.66 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Jan 2025 | mera.a-ai.ru | Official leaderboard |
| 40 | T-pro-it-2.0 open weights, 32.8B | 0.66 | generative-scoring, multi-turn, chat template, system prompt | T-Tech | mera.a-ai.ru | Official leaderboard | |
| 41 | GigaChat 2 Pro closed weights | 0.65 | generative-scoring, multi-turn, chat template, system prompt | GIGACHAT | mera.a-ai.ru | Official leaderboard | |
| 42 | | 0.65 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 43 | GPT-4o Mar 2025 | 0.64 | generative-scoring, multi-turn, chat template, system prompt | MERA | Mar 2025 | mera.a-ai.ru | Official leaderboard |
| 44 | | 0.64 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Feb 2025 | mera.a-ai.ru | Official leaderboard |
| 45 | Meta-Llama-3.1-405B-Instruct open weights, 405B | 0.64 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 46 | Cotype (preview) closed weights | 0.63 | generative-scoring, multi-turn, chat template, system prompt | MTS AI | mera.a-ai.ru | Official leaderboard | |
| 47 | T-pro-it-1.0 open weights, 32B | 0.63 | generative-scoring, multi-turn, chat template, system prompt | T-Tech | mera.a-ai.ru | Official leaderboard | |
| 48 | | 0.63 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Aug 2024 | mera.a-ai.ru | Official leaderboard |
| 49 | Zero-Mistral-24B open weights, 23.6B | 0.62 | generative-scoring, multi-turn, chat template, system prompt | ZeroAgency | mera.a-ai.ru | Official leaderboard | |
| 50 | | 0.62 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 51 | A-vibe open weights, 8B | 0.62 | generative-scoring, multi-turn, chat template, system prompt | Avito | mera.a-ai.ru | Official leaderboard | |
| 52 | | 0.62 | generative-scoring, chat template, system prompt | НГУ | Jan 2025 | mera.a-ai.ru | Official leaderboard |
| 53 | gpt-oss-20b | 0.62 | generative-scoring, multi-turn, chat template, system prompt | openai | Aug 2025 | mera.a-ai.ru | Official leaderboard |
| 54 | Nova Pro 1.0 closed weights | 0.61 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | mera.a-ai.ru | Official leaderboard | |
| 55 | RuadaptQwen-32B-instruct open weights, 32B | 0.61 | generative-scoring, multi-turn, chat template, system prompt | RCC MSU | mera.a-ai.ru | Official leaderboard | |
| 56 | | 0.61 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 57 | | 0.60 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 58 | Qwen2.5-32B-Instruct open weights, 32B | 0.60 | generative-scoring, multi-turn, chat template, system prompt | RCC MSU | mera.a-ai.ru | Official leaderboard | |
| 59 | Supernova-N1-32B-Preview closed weights, 32B | 0.60 | generative-scoring, multi-turn, chat template, system prompt | Nova | mera.a-ai.ru | Official leaderboard | |
| 60 | Meta-Llama-3.1-70B-Instruct open weights, 70.6B | 0.60 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 61 | Mixtral-8x22B-Instruct-v0.1 open weights, 140.6B | 0.59 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 62 | Mistral-Large-Instruct-2407 open weights, 123B | 0.59 | log-likelihood, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 63 | GigaChat Max closed weights | 0.59 | generative-scoring, multi-turn, chat template, system prompt | GIGACHAT | mera.a-ai.ru | Official leaderboard | |
| 64 | Meta-Llama-3-70B-Instruct open weights, 70.6B | 0.59 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 65 | gemma-2-27b-it open weights, 27.2B | 0.58 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 66 | | 0.57 | log-likelihood | MERA | Jul 2024 | mera.a-ai.ru | Official leaderboard |
| 67 | | 0.57 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 68 | GPT-4o mini | 0.57 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jul 2024 | mera.a-ai.ru | Official leaderboard |
| 69 | | 0.57 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Mar 2025 | mera.a-ai.ru | Official leaderboard |
| 70 | | 0.57 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 71 | | 0.56 | generative-scoring | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 72 | Meno-Lite-0.1 open weights, 7B | 0.56 | generative-scoring, multi-turn, chat template, system prompt | НГУ | mera.a-ai.ru | Official leaderboard | |
| 73 | | 0.56 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Dec 2024 | mera.a-ai.ru | Official leaderboard |
| 74 | T-lite-it-1.0 open weights, 7B | 0.55 | generative-scoring, multi-turn, chat template, system prompt | T-Tech | mera.a-ai.ru | Official leaderboard | |
| 75 | | 0.55 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Nov 2024 | mera.a-ai.ru | Official leaderboard |
| 76 | Claude 3.5 Haiku | 0.55 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Oct 2024 | mera.a-ai.ru | Official leaderboard |
| 77 | GigaChat 2 closed weights | 0.54 | generative-scoring, multi-turn, chat template, system prompt | GIGACHAT | mera.a-ai.ru | Official leaderboard | |
| 78 | | 0.54 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Oct 2024 | mera.a-ai.ru | Official leaderboard |
| 79 | RuadaptQwen2.5-7B-Lite-v1 open weights, 7B | 0.54 | generative-scoring, multi-turn, chat template, system prompt | RCC MSU | mera.a-ai.ru | Official leaderboard | |
| 80 | Qwen2.5 14B Instruct open weights, 14.8B | 0.53 | generative-scoring, chat template, system prompt | НГУ | mera.a-ai.ru | Official leaderboard | |
| 81 | Mistral Small 3.1 24B Instruct open weights, 0B | 0.53 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | mera.a-ai.ru | Official leaderboard | |
| 82 | | 0.53 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Oct 2024 | mera.a-ai.ru | Official leaderboard |
| 83 | | 0.53 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 84 | gemma-2-9b-it open weights, 9.2B | 0.52 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 85 | Zero-Mistral-Small-24B-Instruct-2501 open weights, 0B | 0.52 | generative-scoring, multi-turn, chat template, system prompt | ZeroAgency | mera.a-ai.ru | Official leaderboard | |
| 86 | | 0.52 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 87 | Qwen2-57B-A14B-Instruct open weights, 57.4B | 0.52 | log-likelihood, multi-turn, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 88 | Mixtral-8x7B-Instruct-v0.1 open weights, 46.7B | 0.52 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 89 | GigaChat-20B-A3B open weights, 20B | 0.51 | generative-scoring, multi-turn, chat template | GIGACHAT | mera.a-ai.ru | Official leaderboard | |
| 90 | GigaChat Pro closed weights | 0.51 | generative-scoring, multi-turn, chat template, system prompt | GIGACHAT | mera.a-ai.ru | Official leaderboard | |
| 91 | T-lite-0.1 open weights, 8B | 0.51 | log-likelihood | Т-Банк | mera.a-ai.ru | Official leaderboard | |
| 92 | | 0.51 | log-likelihood | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 93 | | 0.51 | log-likelihood | MERA | May 2024 | mera.a-ai.ru | Official leaderboard |
| 94 | | 0.51 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 95 | Qwen2.5-Math-72B open weights, 72.7B | 0.51 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 96 | GigaChat-3.1-Lightning open weights, 11B | 0.50 | generative-scoring, multi-turn, chat template, system prompt | GigaChat | mera.a-ai.ru | Official leaderboard | |
| 97 | | 0.50 | log-likelihood | MERA | Aug 2024 | mera.a-ai.ru | Official leaderboard |
| 98 | GigaChat closed weights | 0.50 | generative-scoring, multi-turn, chat template, system prompt | GIGACHAT | mera.a-ai.ru | Official leaderboard | |
| 99 | Qwen2-57B-A14B open weights, 57.4B | 0.49 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 100 | Qwen2.5-7B-Instruct open weights, 7B | 0.49 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 101 | | 0.49 | log-likelihood | MERA | Dec 2023 | mera.a-ai.ru | Official leaderboard |
| 102 | | 0.49 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jul 2025 | mera.a-ai.ru | Official leaderboard |
| 103 | Meta-Llama-3.1-8B-Instruct open weights, 8B | 0.49 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 104 | Qwen1.5-32B-Chat open weights, 32.5B | 0.49 | log-likelihood, multi-turn, chat template | MERA | mera.a-ai.ru | Official leaderboard | |
| 105 | | 0.48 | log-likelihood | MERA | Feb 2024 | mera.a-ai.ru | Official leaderboard |
| 106 | | 0.48 | generative-scoring, multi-turn, chat template, system prompt | MERA | Mar 2026 | mera.a-ai.ru | Official leaderboard |
| 107 | Yi-1.5-34B-32K open weights, 34.4B | 0.48 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 108 | Meta-Llama-3-8B-Instruct open weights, 8B | 0.48 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 109 | | 0.47 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 110 | Olmo-3.1-32B-Instruct-DPO open weights, 32B | 0.47 | generative-scoring, multi-turn, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 111 | | 0.47 | log-likelihood | MERA | May 2024 | mera.a-ai.ru | Official leaderboard |
| 112 | qwen3-4b-instruct-2507 | 0.47 | generative-scoring, multi-turn, chat template, system prompt | MERA | Aug 2025 | mera.a-ai.ru | Official leaderboard |
| 113 | | 0.47 | log-likelihood | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 114 | Qwen2.5-Coder-7B-Instruct open weights, 7B | 0.46 | log-likelihood, multi-turn, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 115 | Olmo-3.1-32B-Instruct-SFT open weights, 32B | 0.46 | generative-scoring, multi-turn, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 116 | | 0.46 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 117 | | 0.46 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Mar 2024 | mera.a-ai.ru | Official leaderboard |
| 118 | | 0.46 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 119 | | 0.45 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 120 | ruadapt llama3-8B-instruct lep ft open weights, 8.4B | 0.45 | log-likelihood, multi-turn, chat template, system prompt | RCC MSU | mera.a-ai.ru | Official leaderboard | |
| 121 | | 0.45 | log-likelihood | MERA | Jul 2024 | mera.a-ai.ru | Official leaderboard |
| 122 | | 0.45 | log-likelihood | НГУ | Jan 2025 | mera.a-ai.ru | Official leaderboard |
| 123 | | 0.45 | generative-scoring | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 124 | Yi-1.5-34B-Chat-16K open weights, 34.4B | 0.44 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 125 | Qwen2.5-3B-Instruct open weights, 3B | 0.44 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 126 | | 0.44 | log-likelihood | MERA | Nov 2023 | mera.a-ai.ru | Official leaderboard |
| 127 | GPT-3.5 Turbo Jan 2024 | 0.44 | generative-scoring, multi-turn, chat template | MERA | Jan 2024 | mera.a-ai.ru | Official leaderboard |
| 128 | Yi-1.5-34B-Chat open weights, 34.4B | 0.44 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 129 | Mistral-7B-Instruct-v0.3 open weights, 7.2B | 0.43 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 130 | | 0.43 | generative-scoring, multi-turn, chat template, system prompt | MERA | Dec 2025 | mera.a-ai.ru | Official leaderboard |
| 131 | Yi-1.5-9B open weights, 8.8B | 0.43 | generative-scoring | MERA | mera.a-ai.ru | Official leaderboard | |
| 132 | Yi-1.5-9B-32K open weights, 8.8B | 0.43 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 133 | Yi-34B-200K open weights, 34.4B | 0.43 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 134 | Yi-34B-Chat open weights, 34.4B | 0.42 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 135 | Watari-7b-v1 open weights, 7.6B | 0.42 | generative-scoring, chat template, system prompt | Attention Signs | mera.a-ai.ru | Official leaderboard | |
| 136 | | 0.42 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Jul 2024 | mera.a-ai.ru | Official leaderboard |
| 137 | ruadapt_qwen2.5_3B_ext_u48_instruct_v4 open weights, 3B | 0.41 | generative-scoring, multi-turn, chat template, system prompt | RCC MSU | mera.a-ai.ru | Official leaderboard | |
| 138 | | 0.41 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 139 | | 0.41 | log-likelihood | MERA | Dec 2023 | mera.a-ai.ru | Official leaderboard |
| 140 | | 0.40 | log-likelihood | MERA | Sep 2023 | mera.a-ai.ru | Official leaderboard |
| 141 | | 0.40 | generative-scoring, multi-turn, chat template, system prompt | MERA | Mar 2026 | mera.a-ai.ru | Official leaderboard |
| 142 | Yi-1.5-9B-Chat-16K open weights, 8.8B | 0.40 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 143 | | 0.40 | log-likelihood | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 144 | Yi-Coder-9B-Chat open weights, 9B | 0.40 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 145 | | 0.40 | log-likelihood | MERA | Sep 2023 | mera.a-ai.ru | Official leaderboard |
| 146 | | 0.40 | log-likelihood | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 147 | | 0.39 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 148 | | 0.39 | log-likelihood | MERA | May 2024 | mera.a-ai.ru | Official leaderboard |
| 149 | | 0.39 | log-likelihood | MERA | Jul 2024 | mera.a-ai.ru | Official leaderboard |
| 150 | Gamayun-1.5B closed weights, 1.5B | 0.38 | log-likelihood | Gamayun Team | mera.a-ai.ru | Official leaderboard | |
| 151 | Llama-3.2-3B-Instruct open weights, 3B | 0.38 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 152 | Yi-Coder-9B open weights, 9B | 0.37 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 153 | | 0.37 | log-likelihood | MERA | Feb 2024 | mera.a-ai.ru | Official leaderboard |
| 154 | LFM2-24B-A2B open weights, 24B | 0.37 | generative-scoring, multi-turn, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 155 | | 0.37 | log-likelihood | MERA | Sep 2023 | mera.a-ai.ru | Official leaderboard |
| 156 | | 0.37 | log-likelihood | MERA | Feb 2024 | mera.a-ai.ru | Official leaderboard |
| 157 | Meno-Tiny-0.1 open weights, 1.5B | 0.36 | generative-scoring, multi-turn, chat template, system prompt | НГУ | mera.a-ai.ru | Official leaderboard | |
| 158 | | 0.36 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 159 | Qwen2.5 1.5B Instruct open weights, 1.5B | 0.36 | generative-scoring, multi-turn, chat template, system prompt | НГУ | mera.a-ai.ru | Official leaderboard | |
| 160 | Yi-1.5-6B open weights, 6.1B | 0.36 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 161 | Qwen2.5-1.5B-Instruct open weights, 1.5B | 0.35 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 162 | LFM-3B closed weights, 3B | 0.35 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | mera.a-ai.ru | Official leaderboard | |
| 163 | | 0.35 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 164 | Qwen2.5-Math-7B-Instruct open weights, 7B | 0.35 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 165 | Llama-2-13b-hf open weights, 13B | 0.34 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 166 | Qwen2-Math-7B-Instruct open weights | 0.34 | log-likelihood, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 167 | Olmo-3-7B-Instruct open weights, 7B | 0.34 | generative-scoring, multi-turn, chat template | MERA | mera.a-ai.ru | Official leaderboard | |
| 168 | | 0.34 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 169 | | 0.34 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 170 | Qwen1.5-MoE-A2.7B-Chat open weights, 14.3B | 0.34 | log-likelihood, multi-turn, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 171 | | 0.33 | log-likelihood | MERA | May 2024 | mera.a-ai.ru | Official leaderboard |
| 172 | | 0.33 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 173 | Qwen1.5-4B open weights, 4B | 0.32 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 174 | | 0.32 | generative-scoring | MERA | Nov 2023 | mera.a-ai.ru | Official leaderboard |
| 175 | | 0.32 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 176 | gemma-2-2b open weights, 2.6B | 0.31 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 177 | | 0.31 | log-likelihood, multi-turn, chat template, system prompt | MERA | Feb 2024 | mera.a-ai.ru | Official leaderboard |
| 178 | Qwen2.5-Math-1.5B open weights, 1.5B | 0.31 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 179 | Qwen2.5-Coder-1.5B-Instruct open weights, 1.5B | 0.31 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 180 | Yi-6B-200K open weights, 6.1B | 0.31 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 181 | Llama-2-7b-chat-hf open weights, 6.7B | 0.30 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 182 | | 0.30 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 183 | Llama-2-7b-hf open weights, 6.7B | 0.30 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 184 | Llama-3.2-1B-Instruct open weights, 1B | 0.30 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 185 | Qwen2.5-0.5B-Instruct open weights, 0.5B | 0.29 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 186 | Qwen2.5-Math-1.5B-Instruct open weights, 1.5B | 0.28 | log-likelihood, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 187 | LFM2-2.6B open weights, 2.6B | 0.28 | generative-scoring, multi-turn, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 188 | LFM2-8B-A1B | 0.28 | generative-scoring, multi-turn, chat template, system prompt | MERA | Oct 2025 | mera.a-ai.ru | Official leaderboard |
| 189 | Qwen1.5-1.8B open weights, 1.8B | 0.27 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 190 | Qwen1.5-1.8B-Chat open weights, 1.8B | 0.26 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 191 | | 0.26 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 192 | Llama-2-13b-chat-hf open weights, 13B | 0.25 | log-likelihood, multi-turn, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 193 | | 0.25 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 194 | | 0.25 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 195 | LFM2-700M open weights, 0.7B | 0.24 | generative-scoring, multi-turn, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 196 | | 0.23 | log-likelihood, chat template | MERA | Nov 2023 | mera.a-ai.ru | Official leaderboard |
| 197 | Megra 1.1 709M (SFT-4096) closed weights, 0.7B | 0.22 | log-likelihood | Promezis Lab | mera.a-ai.ru | Official leaderboard | |
| 198 | Qwen1.5-0.5B open weights, 0.5B | 0.22 | log-likelihood | MERA | mera.a-ai.ru | Official leaderboard | |
| 199 | Qwen1.5-0.5B-Chat open weights, 0.5B | 0.22 | log-likelihood, multi-turn, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 200 | LFM2-350M open weights, 0.4B | 0.21 | generative-scoring, multi-turn, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 201 | ruGPT-3.5 open weights, 12.9B | 0.21 | log-likelihood | RussianNLP | mera.a-ai.ru | Official leaderboard | |
| 202 | ruGPT3-medium open weights, 0.4B | 0.21 | log-likelihood | RussianNLP | mera.a-ai.ru | Official leaderboard | |
| 203 | ruGPT3-large open weights, 0.8B | 0.20 | log-likelihood | RussianNLP | mera.a-ai.ru | Official leaderboard | |
| 204 | ruGPT-3-small open weights, 0.1B | 0.20 | log-likelihood | RussianNLP | mera.a-ai.ru | Official leaderboard | |
| 205 | LFM2.5-1.2B-Instruct | 0.20 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jan 2026 | mera.a-ai.ru | Official leaderboard |
| 206 | ruGPT-3 XL 1.3B open weights, 1.3B | 0.20 | log-likelihood | ai-forever | mera.a-ai.ru | Official leaderboard | |
| 207 | ruT5-base open weights, 0.2B | 0.19 | log-likelihood | RussianNLP | mera.a-ai.ru | Official leaderboard | |
| 208 | Qwen2-Math-1.5B-Instruct open weights, 1.5B | 0.19 | log-likelihood, chat template, system prompt | MERA | mera.a-ai.ru | Official leaderboard | |
| 209 | ruGPT-3-medium open weights, 0.4B | 0.17 | generative-scoring | RussianNLP | mera.a-ai.ru | Official leaderboard |
| # | Model | Score | Setting | Submitted by | Date | Link | Source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.6 | 0.86 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 2 | GPT-5.4 | 0.82 | generative-scoring, multi-turn, chat template, system prompt | MERA | Mar 2026 | mera.a-ai.ru | Official leaderboard |
| 3 | GLM-5.1 | 0.80 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 4 | | 0.80 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jul 2025 | mera.a-ai.ru | Official leaderboard |
| 5 | | 0.79 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 6 | | 0.79 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 7 | | 0.78 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 8 | | 0.77 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 9 | Claude Opus 4.5 | 0.76 | generative-scoring, multi-turn, chat template, system prompt | Claude Opus 4.5 | Nov 2025 | mera.a-ai.ru | Official leaderboard |
| 10 | | 0.75 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jul 2025 | mera.a-ai.ru | Official leaderboard |
| 11 | Kimi K2.5 | 0.75 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jan 2026 | mera.a-ai.ru | Official leaderboard |
| 12 | | 0.73 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 13 | | 0.73 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 14 | | 0.71 | generative-scoring, multi-turn, chat template, system prompt | MERA | Sep 2025 | mera.a-ai.ru | Official leaderboard |
| 15 | | 0.71 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 16 | | 0.71 | generative-scoring, multi-turn, chat template, system prompt | MERA | Aug 2025 | mera.a-ai.ru | Official leaderboard |
| 17 | GPT-5.2 | 0.71 | generative-scoring, multi-turn, chat template, system prompt | GPT-5.2 | Dec 2025 | mera.a-ai.ru | Official leaderboard |
| 18 | | 0.69 | generative-scoring, multi-turn | MERA | Mar 2026 | mera.a-ai.ru | Official leaderboard |
| 19 | | 0.69 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 20 | | 0.69 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 21 | Claude 3.7 Sonnet | 0.68 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Feb 2025 | mera.a-ai.ru | Official leaderboard |
| 22 | | 0.68 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Feb 2025 | mera.a-ai.ru | Official leaderboard |
| 23 | | 0.68 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 24 | | 0.68 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Mar 2025 | mera.a-ai.ru | Official leaderboard |
| 25 | | 0.68 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 26 | | 0.68 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 27 | | 0.67 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 28 | | 0.66 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Jan 2025 | mera.a-ai.ru | Official leaderboard |
| 29 | | 0.65 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 30 | GPT-4o Mar 2025 | 0.64 | generative-scoring, multi-turn, chat template, system prompt | MERA | Mar 2025 | mera.a-ai.ru | Official leaderboard |
| 31 | | 0.64 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Feb 2025 | mera.a-ai.ru | Official leaderboard |
| 32 | | 0.63 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Aug 2024 | mera.a-ai.ru | Official leaderboard |
| 33 | | 0.62 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 34 | | 0.62 | generative-scoring, chat template, system prompt | НГУ | Jan 2025 | mera.a-ai.ru | Official leaderboard |
| 35 | gpt-oss-20b | 0.62 | generative-scoring, multi-turn, chat template, system prompt | openai | Aug 2025 | mera.a-ai.ru | Official leaderboard |
| 36 | | 0.61 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 37 | | 0.60 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2026 | mera.a-ai.ru | Official leaderboard |
| 38 | | 0.57 | log-likelihood | MERA | Jul 2024 | mera.a-ai.ru | Official leaderboard |
| 39 | | 0.57 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 40 | GPT-4o mini | 0.57 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jul 2024 | mera.a-ai.ru | Official leaderboard |
| 41 | | 0.57 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Mar 2025 | mera.a-ai.ru | Official leaderboard |
| 42 | | 0.57 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 43 | | 0.56 | generative-scoring | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 44 | | 0.56 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Dec 2024 | mera.a-ai.ru | Official leaderboard |
| 45 | | 0.55 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Nov 2024 | mera.a-ai.ru | Official leaderboard |
| 46 | Claude 3.5 Haiku | 0.55 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Oct 2024 | mera.a-ai.ru | Official leaderboard |
| 47 | | 0.54 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Oct 2024 | mera.a-ai.ru | Official leaderboard |
| 48 | | 0.53 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Oct 2024 | mera.a-ai.ru | Official leaderboard |
| 49 | | 0.53 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 50 | | 0.52 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 51 | | 0.51 | log-likelihood | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 52 | | 0.51 | log-likelihood | MERA | May 2024 | mera.a-ai.ru | Official leaderboard |
| 53 | | 0.51 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 54 | | 0.50 | log-likelihood | MERA | Aug 2024 | mera.a-ai.ru | Official leaderboard |
| 55 | | 0.49 | log-likelihood | MERA | Dec 2023 | mera.a-ai.ru | Official leaderboard |
| 56 | | 0.49 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jul 2025 | mera.a-ai.ru | Official leaderboard |
| 57 | | 0.48 | log-likelihood | MERA | Feb 2024 | mera.a-ai.ru | Official leaderboard |
| 58 | | 0.48 | generative-scoring, multi-turn, chat template, system prompt | MERA | Mar 2026 | mera.a-ai.ru | Official leaderboard |
| 59 | | 0.47 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 60 | | 0.47 | log-likelihood | MERA | May 2024 | mera.a-ai.ru | Official leaderboard |
| 61 | qwen3-4b-instruct-2507 | 0.47 | generative-scoring, multi-turn, chat template, system prompt | MERA | Aug 2025 | mera.a-ai.ru | Official leaderboard |
| 62 | | 0.47 | log-likelihood | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 63 | | 0.46 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 64 | | 0.46 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Mar 2024 | mera.a-ai.ru | Official leaderboard |
| 65 | | 0.46 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 66 | | 0.45 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 67 | | 0.45 | log-likelihood | MERA | Jul 2024 | mera.a-ai.ru | Official leaderboard |
| 68 | | 0.45 | log-likelihood | НГУ | Jan 2025 | mera.a-ai.ru | Official leaderboard |
| 69 | | 0.45 | generative-scoring | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 70 | | 0.44 | log-likelihood | MERA | Nov 2023 | mera.a-ai.ru | Official leaderboard |
| 71 | GPT-3.5 Turbo Jan 2024 | 0.44 | generative-scoring, multi-turn, chat template | MERA | Jan 2024 | mera.a-ai.ru | Official leaderboard |
| 72 | | 0.43 | generative-scoring, multi-turn, chat template, system prompt | MERA | Dec 2025 | mera.a-ai.ru | Official leaderboard |
| 73 | | 0.42 | generative-scoring, multi-turn, chat template, system prompt | llmarena.ru | Jul 2024 | mera.a-ai.ru | Official leaderboard |
| 74 | | 0.41 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 75 | | 0.41 | log-likelihood | MERA | Dec 2023 | mera.a-ai.ru | Official leaderboard |
| 76 | | 0.40 | log-likelihood | MERA | Sep 2023 | mera.a-ai.ru | Official leaderboard |
| 77 | | 0.40 | generative-scoring, multi-turn, chat template, system prompt | MERA | Mar 2026 | mera.a-ai.ru | Official leaderboard |
| 78 | | 0.40 | log-likelihood | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 79 | | 0.40 | log-likelihood | MERA | Sep 2023 | mera.a-ai.ru | Official leaderboard |
| 80 | | 0.40 | log-likelihood | MERA | Apr 2024 | mera.a-ai.ru | Official leaderboard |
| 81 | | 0.39 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 82 | | 0.39 | log-likelihood | MERA | May 2024 | mera.a-ai.ru | Official leaderboard |
| 83 | | 0.39 | log-likelihood | MERA | Jul 2024 | mera.a-ai.ru | Official leaderboard |
| 84 | | 0.37 | log-likelihood | MERA | Feb 2024 | mera.a-ai.ru | Official leaderboard |
| 85 | | 0.37 | log-likelihood | MERA | Sep 2023 | mera.a-ai.ru | Official leaderboard |
| 86 | | 0.37 | log-likelihood | MERA | Feb 2024 | mera.a-ai.ru | Official leaderboard |
| 87 | | 0.36 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 88 | | 0.35 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 89 | | 0.34 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 90 | | 0.34 | generative-scoring, multi-turn, chat template, system prompt | MERA | Apr 2025 | mera.a-ai.ru | Official leaderboard |
| 91 | | 0.33 | log-likelihood | MERA | May 2024 | mera.a-ai.ru | Official leaderboard |
| 92 | | 0.33 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 93 | | 0.32 | generative-scoring | MERA | Nov 2023 | mera.a-ai.ru | Official leaderboard |
| 94 | | 0.32 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 95 | | 0.31 | log-likelihood, multi-turn, chat template, system prompt | MERA | Feb 2024 | mera.a-ai.ru | Official leaderboard |
| 96 | | 0.30 | generative-scoring, multi-turn, chat template, system prompt | MERA | Feb 2026 | mera.a-ai.ru | Official leaderboard |
| 97 | LFM2-8B-A1B | 0.28 | generative-scoring, multi-turn, chat template, system prompt | MERA | Oct 2025 | mera.a-ai.ru | Official leaderboard |
| 98 | | 0.26 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 99 | | 0.25 | log-likelihood, system prompt | MERA | Sep 2024 | mera.a-ai.ru | Official leaderboard |
| 100 | | 0.25 | log-likelihood | MERA | Jun 2024 | mera.a-ai.ru | Official leaderboard |
| 101 | | 0.23 | log-likelihood, chat template | MERA | Nov 2023 | mera.a-ai.ru | Official leaderboard |
| 102 | LFM2.5-1.2B-Instruct | 0.20 | generative-scoring, multi-turn, chat template, system prompt | MERA | Jan 2026 | mera.a-ai.ru | Official leaderboard |
Rankings by part
We read 205 parts from the benchmark’s own leaderboard in 23 groups: bps, lcs, rcb, use, rwsd, parus, ruhhh, rutie, multiq, rummlu, chegeka, rudetox, rumodar, mamuramu, simplear, ruethics, rumultiar, rucodeeval, mathlogicqa, ruhumaneval, ruworldtree, ruhatespeech and ruopenbookqa. Each ranks models on one part of the test. Claude Opus 4.6 leads 62 of them, and 34 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
BPS acc
101 models we track have a score here.
- 1
Qwen2.5-72B ↑32 1.00
- 2 Claude Opus 4.5 ↑7 1.00
- 3
Qwen3.5-122B-A10B ↑5 1.00
- 4
Qwen3.5-27B ↑3 1.00
- 5
DeepSeek-V3 Mar 2025 ↑19 1.00
- 6 GPT-5.4 ↓4 1.00
- 7
Qwen3.6 27B ↓1 1.00
- 8
Qwen2.5-Max ↑20 1.00
- 9
Gemini 2.0 Flash Feb 2025 ↑13 1.00
- 10
Gemini 2.0 Flash-Lite ↑21 1.00
LCS acc
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.97
- 2 Claude Opus 4.6 ↓1 0.96
- 3
MiniMax-M2.7 ↑15 0.86
- 4 GLM-5.1 ↓1 0.62
- 5
Qwen3.6 27B ↑1 0.57
- 6
Qwen3-32B ↑6 0.57
- 7
Qwen3-30B-A3B-Thinking Jul 2025 ↑3 0.56
- 8
Qwen 3.6 35B-A3B ↓3 0.55
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.51
- 10
Qwen3-14B ↑9 0.49
RCB acc
102 models we track have a score here.
- 1 Claude Opus 4.6 0.63
- 2 GLM-5.1 ↑1 0.63
- 3
Gemma 4 31B IT ↑12 0.61
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.61
- 5
Qwen3.5-27B ↑2 0.61
- 6
Qwen3.6 27B 0.61
- 7 Kimi K2.5 ↑4 0.60
- 8
Qwen3.5-122B-A10B 0.60
- 9
Qwen3-30B-A3B-Thinking Jul 2025 ↑1 0.60
- 10
Qwen 3.6 35B-A3B ↓5 0.60
RCB f1_macro
102 models we track have a score here.
- 1 Claude Opus 4.6 0.62
- 2 GPT-5.4 0.60
- 3
Gemma 4 31B IT ↑12 0.59
- 4
Qwen3-4B-Thinking-2507 ↑12 0.59
- 5 Claude Opus 4.5 ↑4 0.59
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.59
- 7
Qwen3-30B-A3B-Thinking Jul 2025 ↑3 0.58
- 8 Kimi K2.5 ↑3 0.58
- 9 Claude 3.7 Sonnet ↑12 0.58
- 10
Gemini 1.5 Pro Sep 2024 ↑16 0.57
USE grade_norm
102 models we track have a score here.
- 1 Claude Opus 4.6 0.89
- 2 GPT-5.4 0.86
- 3 GLM-5.1 0.78
- 4 Claude Opus 4.5 ↑5 0.74
- 5
Qwen3.5-122B-A10B ↑3 0.71
- 6
Qwen3.5-27B ↑1 0.70
- 7 Kimi K2.5 ↑4 0.69
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.69
- 9
DeepSeek-V4-Flash ↑16 0.68
- 10
DeepSeek-V4-Pro ↑10 0.68
USE grade_norm.task1
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.97
- 2 Claude Opus 4.6 ↓1 0.97
- 3 GPT-5.4 ↓1 0.93
- 4
Qwen3.5-122B-A10B ↑4 0.90
- 5
Qwen2.5-Max ↑23 0.80
- 6
DeepSeek-V3 Mar 2025 ↑18 0.80
- 7
Llama 4 Maverick ↑29 0.80
- 8
Qwen3.5-27B ↓1 0.80
- 9
Qwen3.5-35B-A3B ↑4 0.80
- 10 Claude 3.7 Sonnet ↑11 0.77
USE grade_norm.task2
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.83
- 2
Qwen2.5-Max ↑26 0.80
- 3
DeepSeek-V3 Mar 2025 ↑21 0.80
- 4
Gemini 1.5 Pro Sep 2024 ↑22 0.77
- 5 GPT-5.2 ↑12 0.77
- 6 Claude Opus 4.6 ↓5 0.77
- 7 Claude 3.7 Sonnet ↑14 0.73
- 8 GPT-5.4 ↓6 0.73
- 9
DeepSeek-V4-Flash ↑16 0.73
- 10
DeepSeek-V4-Pro ↑10 0.73
USE grade_norm.task3
102 models we track have a score here.
- 1
Qwen3.5-122B-A10B ↑7 0.90
- 2
Qwen3.5-27B ↑5 0.90
- 3
Gemma 4 26B A4B ↑34 0.90
- 4
Llama 3.1-70B ↑34 0.87
- 5
DeepSeek-V3 Mar 2025 ↑19 0.87
- 6
Qwen3-14B ↑13 0.87
- 7
Qwen3-32B ↑5 0.87
- 8
Qwen3-8B ↑21 0.87
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.87
- 10
Qwen3-30B-A3B-Instruct Jul 2025 ↑46 0.87
USE grade_norm.task4
102 models we track have a score here.
- 1 Claude Opus 4.6 0.90
- 2 Claude Opus 4.5 ↑7 0.73
- 3 GPT-5.4 ↓1 0.67
- 4
DeepSeek-V4-Flash ↑21 0.50
- 5
DeepSeek-V4-Pro ↑15 0.50
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.47
- 7 Claude 3.7 Sonnet ↑14 0.43
- 8 GLM-5.1 ↓5 0.43
- 9
Gemma 4 26B A4B ↑28 0.40
- 10
DeepSeek-V3 Mar 2025 ↑14 0.37
USE grade_norm.task5
102 models we track have a score here.
- 1 Claude Opus 4.6 0.93
- 2 Claude Opus 4.5 ↑7 0.87
- 3
Qwen3.5-122B-A10B ↑5 0.87
- 4 GPT-5.4 ↓2 0.87
- 5 GLM-5.1 ↓2 0.80
- 6 GPT-5.2 ↑11 0.73
- 7
Qwen3.5-27B 0.73
- 8 Claude 3.7 Sonnet ↑13 0.70
- 9 Kimi K2.5 ↑2 0.70
- 10
DeepSeek-V4-Flash ↑15 0.70
USE grade_norm.task6
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2
Qwen3.5-122B-A10B ↑6 0.97
- 3 GPT-5.4 ↓1 0.97
- 4 GLM-5.1 ↓1 0.97
- 5 Claude Opus 4.5 ↑4 0.93
- 6 Kimi K2.5 ↑5 0.93
- 7
DeepSeek-V4-Flash ↑18 0.93
- 8
DeepSeek-V4-Pro ↑12 0.93
- 9 GPT-5.2 ↑8 0.90
- 10
Qwen 3.6 35B-A3B ↓5 0.90
USE grade_norm.task7
102 models we track have a score here.
- 1 Kimi K2.5 ↑10 0.93
- 2 Claude Opus 4.6 ↓1 0.93
- 3 GPT-5.4 ↓1 0.93
- 4 Claude Opus 4.5 ↑5 0.83
- 5 GLM-5.1 ↓2 0.83
- 6
DeepSeek-V4-Flash ↑19 0.80
- 7
DeepSeek-V4-Pro ↑13 0.80
- 8
Qwen3.5-27B ↓1 0.73
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.67
- 10
Qwen3.5-122B-A10B ↓2 0.67
USE grade_norm.task9
102 models we track have a score here.
- 1 Claude Opus 4.6 0.90
- 2 Claude Opus 4.5 ↑7 0.83
- 3 GPT-5.4 ↓1 0.83
- 4 GLM-5.1 ↓1 0.77
- 5
Gemini 2.0 Flash Feb 2025 ↑17 0.53
- 6
DeepSeek-V4-Flash ↑19 0.50
- 7
DeepSeek-V4-Pro ↑13 0.50
- 8
Qwen2.5-Max ↑20 0.43
- 9
Gemini 2.0 Flash-Lite ↑22 0.43
- 10
Gemma 4 26B A4B ↑27 0.43
USE grade_norm.task10
102 models we track have a score here.
- 1 Claude Opus 4.6 0.80
- 2 GPT-5.4 0.67
- 3 GLM-5.1 0.47
- 4 Kimi K2.5 ↑7 0.43
- 5
MiniMax-M2.7 ↑13 0.33
- 6
DeepSeek-V4-Flash ↑19 0.33
- 7
DeepSeek-V4-Pro ↑13 0.33
- 8
Qwen3.5-122B-A10B 0.27
- 9
Qwen3.5-27B ↓2 0.23
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 0.17
USE grade_norm.task11
102 models we track have a score here.
- 1 Claude Opus 4.6 0.93
- 2 GPT-5.4 0.87
- 3 Kimi K2.5 ↑8 0.67
- 4 GLM-5.1 ↓1 0.67
- 5 GPT-5.2 ↑12 0.60
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.53
- 7
Qwen3.5-27B 0.53
- 8
Qwen3.5-122B-A10B 0.43
- 9
MiniMax-M2.7 ↑9 0.40
- 10
Qwen3-32B ↑2 0.33
USE grade_norm.task12
102 models we track have a score here.
- 1 Claude Opus 4.6 1.00
- 2 GPT-5.4 0.90
- 3 GPT-5.2 ↑14 0.70
- 4 GLM-5.1 ↓1 0.70
- 5 Kimi K2.5 ↑6 0.63
- 6
DeepSeek-V4-Flash ↑19 0.57
- 7
DeepSeek-V4-Pro ↑13 0.57
- 8
Qwen3.5-27B ↓1 0.50
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.47
- 10
Qwen3.5-122B-A10B ↓2 0.47
USE grade_norm.task13
102 models we track have a score here.
- 1 Claude Opus 4.6 1.00
- 2 GPT-5.4 1.00
- 3 GLM-5.1 0.97
- 4 Claude Opus 4.5 ↑5 0.93
- 5
DeepSeek-V4-Flash ↑20 0.90
- 6
DeepSeek-V4-Pro ↑14 0.90
- 7
Gemma 4 31B IT ↑8 0.87
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.80
- 9 Kimi K2.5 ↑2 0.80
- 10
Qwen3.5-122B-A10B ↓2 0.80
USE grade_norm.task14
102 models we track have a score here.
- 1 GPT-5.4 ↑1 1.00
- 2 GLM-5.1 ↑1 0.93
- 3 Claude Opus 4.6 ↓2 0.90
- 4 Kimi K2.5 ↑7 0.83
- 5
Qwen3.5-122B-A10B ↑3 0.83
- 6
Qwen3.5-27B ↑1 0.80
- 7
Qwen3.5-35B-A3B ↑6 0.73
- 8
DeepSeek-V4-Flash ↑17 0.73
- 9
DeepSeek-V4-Pro ↑11 0.73
- 10 Claude Opus 4.5 ↓1 0.63
USE grade_norm.task15
102 models we track have a score here.
- 1 Claude Opus 4.6 0.87
- 2 GPT-5.4 0.83
- 3 GLM-5.1 0.67
- 4
Qwen3.5-122B-A10B ↑4 0.63
- 5 Claude Opus 4.5 ↑4 0.60
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.57
- 7
Qwen3.5-27B 0.57
- 8
Qwen3.6 27B ↓2 0.53
- 9
DeepSeek-V4-Flash ↑16 0.53
- 10
DeepSeek-V4-Pro ↑10 0.53
USE grade_norm.task16
102 models we track have a score here.
- 1 Claude Opus 4.6 0.92
- 2 GPT-5.4 0.92
- 3
Qwen 3.6 35B-A3B ↑2 0.87
- 4
Qwen3.6 27B ↑2 0.85
- 5 Claude Opus 4.5 ↑4 0.83
- 6 GLM-5.1 ↓3 0.83
- 7
Qwen3.5-122B-A10B ↑1 0.80
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.75
- 9
Qwen3.5-35B-A3B ↑4 0.70
- 10 Kimi K2.5 ↑1 0.68
USE grade_norm.task17
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.97
- 2 Claude Opus 4.6 ↓1 0.93
- 3
Qwen3.5-122B-A10B ↑5 0.93
- 4 Claude Opus 4.5 ↑5 0.80
- 5
Qwen3.5-27B ↑2 0.80
- 6
Qwen3.5-35B-A3B ↑7 0.77
- 7
Qwen3-235B-A22B-Thinking Jul 2025 ↓3 0.73
- 8 GLM-5.1 ↓5 0.73
- 9
MiniMax-M2.7 ↑9 0.67
- 10 Kimi K2.5 ↑1 0.63
USE grade_norm.task18
102 models we track have a score here.
- 1 Claude Opus 4.6 0.57
- 2 GLM-5.1 ↑1 0.57
- 3
Qwen3.5-122B-A10B ↑5 0.53
- 4
Qwen3-Next-80B-A3B ↑10 0.50
- 5 GPT-5.4 ↓3 0.50
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.47
- 7
Qwen3.5-27B 0.47
- 8
Qwen3.5-35B-A3B ↑5 0.47
- 9
Gemma 4 26B A4B ↑28 0.43
- 10
Gemma 4 31B IT ↑5 0.43
USE grade_norm.task19
102 models we track have a score here.
- 1
Qwen3-235B-A22B-Thinking Jul 2025 ↑3 0.97
- 2 Claude Opus 4.6 ↓1 0.97
- 3 GPT-5.4 ↓1 0.97
- 4
Qwen3.5-122B-A10B ↑4 0.93
- 5
Qwen3.5-27B ↑2 0.93
- 6 Claude Opus 4.5 ↑3 0.90
- 7 GLM-5.1 ↓4 0.90
- 8
Qwen3.6 27B ↓2 0.90
- 9
Qwen 3.6 35B-A3B ↓4 0.87
- 10
Qwen3.5-35B-A3B ↑3 0.83
USE grade_norm.task20
102 models we track have a score here.
- 1 Claude Opus 4.6 0.73
- 2 GPT-5.4 0.73
- 3
Qwen3.6 27B ↑3 0.57
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.53
- 5
Qwen 3.6 35B-A3B 0.53
- 6 Kimi K2.5 ↑5 0.50
- 7 GLM-5.1 ↓4 0.50
- 8 Claude Opus 4.5 ↑1 0.47
- 9
Qwen3.5-35B-A3B ↑4 0.47
- 10
DeepSeek-V4-Flash ↑15 0.47
USE grade_norm.task21
102 models we track have a score here.
- 1 Claude Opus 4.6 0.93
- 2
Qwen3.5-27B ↑5 0.93
- 3
Qwen3.6 27B ↑3 0.93
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.87
- 5 Kimi K2.5 ↑6 0.87
- 6
Gemma 4 31B IT ↑9 0.87
- 7 GLM-5.1 ↓4 0.87
- 8
Qwen3.5-122B-A10B 0.83
- 9 GPT-5.4 ↓7 0.83
- 10
Qwen 3.6 35B-A3B ↓5 0.77
USE grade_norm.task22
102 models we track have a score here.
- 1 Claude Opus 4.6 0.90
- 2
Qwen3.5-27B ↑5 0.87
- 3
DeepSeek-V3 Mar 2025 ↑21 0.83
- 4 GPT-4o Mar 2025 ↑26 0.83
- 5 Claude Opus 4.5 ↑4 0.83
- 6
Qwen3.6 27B 0.83
- 7
Qwen2-72B ↑32 0.80
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.80
- 9 Kimi K2.5 ↑2 0.80
- 10
Qwen3.5-122B-A10B ↓2 0.80
USE grade_norm.task23
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.80
- 2 Claude Opus 4.6 ↓1 0.77
- 3
Qwen 3.6 35B-A3B ↑2 0.77
- 4 Claude Opus 4.5 ↑5 0.73
- 5 GLM-5.1 ↓2 0.73
- 6 Claude 3.5 Haiku ↑40 0.67
- 7
Qwen3.6 27B ↓1 0.67
- 8
Gemini 1.5 Pro Sep 2024 ↑18 0.63
- 9
Gemini 2.0 Flash Feb 2025 ↑13 0.63
- 10
DeepSeek-V3 Mar 2025 ↑14 0.63
USE grade_norm.task24
102 models we track have a score here.
- 1 Kimi K2.5 ↑10 0.77
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 0.73
- 3
Qwen 3.6 35B-A3B ↑2 0.73
- 4 Claude Opus 4.6 ↓3 0.70
- 5 GPT-5.4 ↓3 0.70
- 6
Gemma 4 31B IT ↑9 0.70
- 7
Qwen3.5-27B 0.67
- 8 GLM-5.1 ↓5 0.67
- 9 Claude Opus 4.5 0.63
- 10
Qwen3-32B ↑2 0.60
USE grade_norm.task25
102 models we track have a score here.
- 1 Claude Opus 4.6 0.90
- 2
Qwen3.5-122B-A10B ↑6 0.90
- 3 GLM-5.1 0.90
- 4
Qwen3.5-27B ↑3 0.80
- 5 GPT-5.4 ↓3 0.80
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.77
- 7
Qwen3.5-35B-A3B ↑6 0.77
- 8
Gemma 4 31B IT ↑7 0.77
- 9
Qwen3.6 27B ↓3 0.73
- 10 Kimi K2.5 ↑1 0.70
USE grade_norm.task26
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.94
- 2 Claude Opus 4.6 ↓1 0.93
- 3 Claude Opus 4.5 ↑6 0.91
- 4
Qwen 3.6 35B-A3B ↑1 0.91
- 5
Qwen3.5-27B ↑2 0.89
- 6
Qwen3.6 27B 0.89
- 7
Gemma 4 31B IT ↑8 0.88
- 8
DeepSeek-V3 Mar 2025 ↑16 0.87
- 9 GLM-5.1 ↓6 0.87
- 10 GPT-5.2 ↑7 0.86
USE grade_norm.task8_0
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.90
- 2 Claude Opus 4.5 ↑7 0.87
- 3 Claude Opus 4.6 ↓2 0.87
- 4 GLM-5.1 ↓1 0.87
- 5 Claude 3.7 Sonnet ↑16 0.83
- 6
DeepSeek-V4-Flash ↑19 0.80
- 7
DeepSeek-V4-Pro ↑13 0.80
- 8
Qwen3.5-122B-A10B 0.77
- 9
Gemma 4 31B IT ↑6 0.77
- 10 Claude 3.5 Haiku ↑36 0.70
USE grade_norm.task8_1
102 models we track have a score here.
- 1 Claude Opus 4.6 1.00
- 2 GLM-5.1 ↑1 1.00
- 3 Claude Opus 4.5 ↑6 0.97
- 4 GPT-5.4 ↓2 0.97
- 5 Claude 3.7 Sonnet ↑16 0.90
- 6
Qwen3.5-122B-A10B ↑2 0.87
- 7
Qwen 3.6 35B-A3B ↓2 0.87
- 8
Qwen3.6 27B ↓2 0.87
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.83
- 10 Kimi K2.5 ↑1 0.83
USE grade_norm.task8_2
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.90
- 2
Gemma 4 31B IT ↑13 0.90
- 3 Claude Opus 4.6 ↓2 0.87
- 4
Gemma 4 26B A4B ↑33 0.87
- 5
Gemini 1.5 Pro Sep 2024 ↑21 0.83
- 6 Claude 3.7 Sonnet ↑15 0.83
- 7
DeepSeek-V3 Mar 2025 ↑17 0.83
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.83
- 9 Kimi K2.5 ↑2 0.83
- 10
Qwen3.5-27B ↓3 0.83
USE grade_norm.task8_3
102 models we track have a score here.
- 1 Claude Opus 4.6 0.87
- 2
DeepSeek-V3 Mar 2025 ↑22 0.83
- 3 Claude Opus 4.5 ↑6 0.83
- 4 GPT-5.4 ↓2 0.83
- 5
Gemma 4 31B IT ↑10 0.83
- 6 GLM-5.1 ↓3 0.83
- 7 Claude 3.7 Sonnet ↑14 0.80
- 8
Qwen3.5-27B ↓1 0.77
- 9
Qwen2.5-Max ↑19 0.73
- 10 Kimi K2.5 ↑1 0.73
USE grade_norm.task8_4
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 1.00
- 2 Claude Opus 4.6 ↓1 1.00
- 3
Qwen3.5-122B-A10B ↑5 1.00
- 4 GPT-5.4 ↓2 1.00
- 5 GLM-5.1 ↓2 1.00
- 6 Claude 3.7 Sonnet ↑15 0.97
- 7
Qwen3-235B-A22B-Thinking Jul 2025 ↓3 0.97
- 8
Gemma 4 31B IT ↑7 0.97
- 9 GPT-5.2 ↑8 0.93
- 10
Qwen3.5-27B ↓3 0.93
RWSD acc
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.95
- 2 Claude Opus 4.6 ↓1 0.93
- 3
Qwen3.5-122B-A10B ↑5 0.91
- 4
Qwen3.5-27B ↑3 0.90
- 5 GLM-5.1 ↓2 0.90
- 6 Kimi K2.5 ↑5 0.88
- 7 Claude Opus 4.5 ↑2 0.86
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.84
- 9 GPT-5.2 ↑8 0.84
- 10
Qwen3.6 27B ↓4 0.84
PARus acc
102 models we track have a score here.
- 1
DeepSeek-R1-Distill-Qwen-32B ↑33 0.96
- 2
Qwen2.5-72B ↑31 0.96
- 3
Gemini 1.5 Pro Sep 2024 ↑23 0.96
- 4
Qwen2-72B ↑35 0.95
- 5 Claude 3.7 Sonnet ↑16 0.95
- 6 GPT-4o Mar 2025 ↑24 0.94
- 7 Claude Opus 4.5 ↑2 0.94
- 8
Gemini 2.0 Flash Feb 2025 ↑14 0.93
- 9 Claude Opus 4.6 ↓8 0.93
- 10
Qwen2.5-Max ↑18 0.93
ruHHH acc
101 models we track have a score here.
- 1
Qwen3.6 27B ↑5 0.94
- 2 Kimi K2.5 ↑9 0.92
- 3 Claude Opus 4.6 ↓2 0.92
- 4 GLM-5.1 ↓1 0.92
- 5
Qwen3.5-122B-A10B ↑3 0.91
- 6
Qwen3.5-27B ↑1 0.91
- 7
Gemma 4 31B IT ↑8 0.90
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.89
- 9 GPT-5.4 ↓7 0.89
- 10
Qwen 3.6 35B-A3B ↓5 0.89
ruHHH acc.honest
101 models we track have a score here.
- 1 Kimi K2.5 ↑10 0.93
- 2
Qwen3.6 27B ↑4 0.93
- 3 GPT-5.2 ↑14 0.92
- 4 GLM-5.1 ↓1 0.92
- 5 Claude Opus 4.6 ↓4 0.90
- 6
Qwen2.5-Max ↑22 0.89
- 7
Qwen3.5-27B 0.89
- 8
Qwen3.5-9B ↑15 0.89
- 9
Gemma 4 31B IT ↑6 0.89
- 10
Qwen2-72B ↑29 0.87
ruHHH acc.helpful
101 models we track have a score here.
- 1 GLM-5.1 ↑2 0.92
- 2
Qwen3.6 27B ↑4 0.92
- 3
Qwen2.5-72B ↑30 0.90
- 4
Qwen3-14B ↑15 0.90
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 0.90
- 6 Claude Opus 4.6 ↓5 0.90
- 7
Qwen3.5-122B-A10B ↑1 0.90
- 8
MiniMax-M2.7 ↑10 0.90
- 9
Qwen 3.6 35B-A3B ↓4 0.90
- 10
Qwen2-72B ↑29 0.88
ruHHH acc.harmless
101 models we track have a score here.
- 1
Qwen3.5-27B ↑6 0.98
- 2 Kimi K2.5 ↑9 0.97
- 3
Qwen3.5-122B-A10B ↑5 0.97
- 4
Qwen3.5-35B-A3B ↑9 0.97
- 5
Gemma 4 31B IT ↑10 0.97
- 6
Qwen3.6 27B 0.97
- 7
Qwen3-32B ↑5 0.95
- 8
Qwen3-30B-A3B-Thinking Jul 2025 ↑2 0.95
- 9
Qwen3-Next-80B-A3B ↑5 0.95
- 10 Claude Opus 4.5 ↓1 0.95
ruTiE acc
102 models we track have a score here.
- 1 Claude Opus 4.6 0.98
- 2 GPT-5.4 0.97
- 3 GLM-5.1 0.97
- 4 Claude Opus 4.5 ↑5 0.96
- 5 GPT-5.2 ↑12 0.95
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.95
- 7 Claude 3.7 Sonnet ↑14 0.93
- 8 Kimi K2.5 ↑3 0.93
- 9
Gemini 2.0 Flash Feb 2025 ↑13 0.92
- 10
Qwen2.5-Max ↑18 0.91
MultiQ em
102 models we track have a score here.
- 1
Gemma 4 31B IT ↑14 0.50
- 2 GLM-5.1 ↑1 0.50
- 3
Gemma 4 26B A4B ↑34 0.49
- 4 GPT-5.2 ↑13 0.48
- 5 Kimi K2.5 ↑6 0.47
- 6
Gemini 2.0 Flash Feb 2025 ↑16 0.47
- 7 Claude 3.7 Sonnet ↑14 0.47
- 8 GPT-5.4 ↓6 0.46
- 9
DeepSeek-V3 Mar 2025 ↑15 0.46
- 10
Qwen2-72B ↑29 0.45
MultiQ f1
102 models we track have a score here.
- 1
Gemma 4 31B IT ↑14 0.67
- 2 GPT-5.2 ↑15 0.66
- 3 Kimi K2.5 ↑8 0.65
- 4 GPT-5.4 ↓2 0.65
- 5 GLM-5.1 ↓2 0.65
- 6
Gemma 4 26B A4B ↑31 0.65
- 7
Gemini 2.0 Flash Feb 2025 ↑15 0.62
- 8 Claude 3.7 Sonnet ↑13 0.62
- 9
Qwen3.6 27B ↓3 0.62
- 10
Llama-3.1-Nemotron-70B-Instruct ↑38 0.62
ruMMLU acc
102 models we track have a score here.
- 1 Claude Opus 4.6 0.92
- 2 GPT-5.4 0.92
- 3 GLM-5.1 0.91
- 4 Claude Opus 4.5 ↑5 0.90
- 5
Qwen3.5-27B ↑2 0.89
- 6 Kimi K2.5 ↑5 0.89
- 7
Qwen3.5-122B-A10B ↑1 0.89
- 8
Qwen3.6 27B ↓2 0.89
- 9 GPT-5.2 ↑8 0.88
- 10
Qwen 3.6 35B-A3B ↓5 0.87
ruMMLU acc.anatomy
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.93
- 2 Claude Opus 4.6 ↓1 0.93
- 3 GPT-5.4 ↓1 0.93
- 4 GLM-5.1 ↓1 0.93
- 5
Qwen3.5-27B ↑2 0.91
- 6 GPT-5.2 ↑11 0.90
- 7
Qwen3.5-122B-A10B ↑1 0.90
- 8
Qwen3.6 27B ↓2 0.90
- 9
Qwen3.5-35B-A3B ↑4 0.89
- 10
MiniMax-M2.7 ↑8 0.89
ruMMLU acc.virology
102 models we track have a score here.
- 1
Qwen2-72B ↑38 0.59
- 2 GPT-5.2 ↑15 0.58
- 3
Qwen2.5-Max ↑25 0.58
- 4 Claude Opus 4.5 ↑5 0.58
- 5 Claude Opus 4.6 ↓4 0.58
- 6
Qwen3.5-27B ↑1 0.58
- 7
Qwen2.5-72B ↑26 0.57
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.57
- 9 GPT-5.4 ↓7 0.57
- 10 Claude 3.7 Sonnet ↑11 0.57
ruMMLU acc.astronomy
102 models we track have a score here.
- 1 Kimi K2.5 ↑10 0.98
- 2 GPT-5.4 0.98
- 3 Claude Opus 4.6 ↓2 0.97
- 4
Qwen3.5-27B ↑3 0.97
- 5
Qwen3.5-35B-A3B ↑8 0.97
- 6 GLM-5.1 ↓3 0.97
- 7
Qwen3.6 27B ↓1 0.97
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.97
- 9
Qwen3-Next-80B-A3B ↑5 0.97
- 10
Qwen3.5-122B-A10B ↓2 0.97
ruMMLU acc.marketing
102 models we track have a score here.
- 1 GLM-5.1 ↑2 0.94
- 2 Claude Opus 4.6 ↓1 0.94
- 3 Kimi K2.5 ↑8 0.93
- 4 GPT-5.4 ↓2 0.93
- 5 Claude Opus 4.5 ↑4 0.93
- 6
Qwen3.5-27B ↑1 0.93
- 7 Claude 3.7 Sonnet ↑14 0.92
- 8
Qwen3.6 27B ↓2 0.92
- 9 GPT-5.2 ↑8 0.92
- 10
Qwen 3.6 35B-A3B ↓5 0.92
ruMMLU acc.nutrition
102 models we track have a score here.
- 1 Claude Opus 4.6 0.94
- 2 GPT-5.4 0.94
- 3 Claude Opus 4.5 ↑6 0.94
- 4 GLM-5.1 ↓1 0.94
- 5 GPT-5.2 ↑12 0.92
- 6 Kimi K2.5 ↑5 0.92
- 7
Qwen3.5-27B 0.92
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.92
- 9
Qwen3.5-122B-A10B ↓1 0.91
- 10
Qwen3.6 27B ↓4 0.91
ruMMLU acc.sociology
102 models we track have a score here.
- 1 Claude Opus 4.6 0.95
- 2 Claude Opus 4.5 ↑7 0.94
- 3 Claude 3.7 Sonnet ↑18 0.93
- 4
Qwen3.5-27B ↑3 0.93
- 5 GLM-5.1 ↓2 0.93
- 6 Kimi K2.5 ↑5 0.93
- 7
Qwen3.6 27B ↓1 0.93
- 8 GPT-5.2 ↑9 0.92
- 9 GPT-5.4 ↓7 0.92
- 10
Gemma 4 31B IT ↑5 0.92
ruMMLU acc.management
102 models we track have a score here.
- 1 Kimi K2.5 ↑10 0.90
- 2 GPT-5.4 0.89
- 3 Claude 3.7 Sonnet ↑18 0.88
- 4 Claude Opus 4.5 ↑5 0.88
- 5 Claude Opus 4.6 ↓4 0.88
- 6
Qwen3.5-122B-A10B ↑2 0.88
- 7
Qwen3-235B-A22B-Thinking Jul 2025 ↓3 0.87
- 8
Qwen3.5-27B ↓1 0.87
- 9 GPT-5.2 ↑8 0.86
- 10
Qwen 3.6 35B-A3B ↓5 0.86
ruMMLU acc.philosophy
102 models we track have a score here.
- 1 Claude Opus 4.6 0.95
- 2 Claude Opus 4.5 ↑7 0.93
- 3 GPT-5.4 ↓1 0.91
- 4 GLM-5.1 ↓1 0.91
- 5 GPT-5.2 ↑12 0.90
- 6 Kimi K2.5 ↑5 0.90
- 7
Qwen3.5-27B 0.87
- 8
Qwen3.5-35B-A3B ↑5 0.87
- 9
Qwen3.5-122B-A10B ↓1 0.86
- 10
Qwen 3.6 35B-A3B ↓5 0.86
ruMMLU acc.prehistory
102 models we track have a score here.
- 1 Claude Opus 4.6 0.96
- 2 GPT-5.4 0.95
- 3 Claude Opus 4.5 ↑6 0.95
- 4 GPT-5.2 ↑13 0.94
- 5 GLM-5.1 ↓2 0.93
- 6 Claude 3.7 Sonnet ↑15 0.92
- 7 Kimi K2.5 ↑4 0.92
- 8
Qwen3.6 27B ↓2 0.92
- 9
Qwen 3.6 35B-A3B ↓4 0.92
- 10
Qwen3.5-35B-A3B ↑3 0.90
ruMMLU acc.human_aging
102 models we track have a score here.
- 1 Claude Opus 4.6 0.87
- 2 Claude Opus 4.5 ↑7 0.87
- 3 GPT-5.4 ↓1 0.86
- 4 GPT-5.2 ↑13 0.82
- 5 Kimi K2.5 ↑6 0.82
- 6 GLM-5.1 ↓3 0.81
- 7
Qwen3.5-122B-A10B ↑1 0.81
- 8
Gemma 4 31B IT ↑7 0.81
- 9
Qwen3.6 27B ↓3 0.80
- 10
Qwen3.5-27B ↓3 0.79
ruMMLU acc.econometrics
102 models we track have a score here.
- 1 GLM-5.1 ↑2 0.88
- 2 GPT-5.4 0.87
- 3
Qwen3.6 27B ↑3 0.87
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.85
- 5 Claude Opus 4.5 ↑4 0.84
- 6 Claude Opus 4.6 ↓5 0.84
- 7
Qwen3.5-122B-A10B ↑1 0.84
- 8
Qwen3.5-35B-A3B ↑5 0.84
- 9
Qwen 3.6 35B-A3B ↓4 0.84
- 10 Kimi K2.5 ↑1 0.83
ruMMLU acc.formal_logic
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2
Qwen3.5-27B ↑5 0.95
- 3 GPT-5.4 ↓1 0.95
- 4 GLM-5.1 ↓1 0.95
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 0.94
- 6
Qwen3.6 27B 0.94
- 7 Kimi K2.5 ↑4 0.94
- 8
Qwen3.5-122B-A10B 0.94
- 9
Qwen3.5-35B-A3B ↑4 0.94
- 10
Qwen3.5-9B ↑13 0.92
ruMMLU acc.global_facts
102 models we track have a score here.
- 1 GLM-5.1 ↑2 0.83
- 2 Claude Opus 4.5 ↑7 0.78
- 3 Claude Opus 4.6 ↓2 0.77
- 4 Kimi K2.5 ↑7 0.76
- 5 Claude 3.7 Sonnet ↑16 0.75
- 6 GPT-5.4 ↓4 0.75
- 7 GPT-5.2 ↑10 0.72
- 8
MiniMax-M2.7 ↑10 0.71
- 9
Gemini 1.5 Pro Sep 2024 ↑17 0.70
- 10
DeepSeek-V3 Mar 2025 ↑14 0.70
ruMMLU acc.jurisprudence
102 models we track have a score here.
- 1 Claude Opus 4.6 0.92
- 2 GPT-5.4 0.92
- 3
Qwen3.6 27B ↑3 0.92
- 4
Gemini 2.0 Flash Feb 2025 ↑18 0.90
- 5 GLM-5.1 ↓2 0.90
- 6
Qwen 3.6 35B-A3B ↓1 0.89
- 7
Qwen2-72B ↑32 0.88
- 8
Gemini 1.5 Pro Sep 2024 ↑18 0.88
- 9 Claude Opus 4.5 0.88
- 10 Kimi K2.5 ↑1 0.88
ruMMLU acc.miscellaneous
102 models we track have a score here.
- 1 GLM-5.1 ↑2 0.97
- 2 Claude Opus 4.6 ↓1 0.97
- 3 GPT-5.4 ↓1 0.96
- 4 Claude Opus 4.5 ↑5 0.96
- 5 Kimi K2.5 ↑6 0.96
- 6
Qwen3.5-122B-A10B ↑2 0.95
- 7 GPT-5.2 ↑10 0.95
- 8
Qwen 3.6 35B-A3B ↓3 0.94
- 9
Qwen3.5-27B ↓2 0.93
- 10
Qwen3.6 27B ↓4 0.93
ruMMLU acc.moral_disputes
102 models we track have a score here.
- 1 Claude Opus 4.6 0.89
- 2 Claude Opus 4.5 ↑7 0.88
- 3 GPT-5.4 ↓1 0.86
- 4 Kimi K2.5 ↑7 0.85
- 5 GLM-5.1 ↓2 0.85
- 6 GPT-4o Mar 2025 ↑24 0.84
- 7 GPT-5.2 ↑10 0.84
- 8
Qwen3.5-122B-A10B 0.84
- 9
Qwen3.5-27B ↓2 0.84
- 10
Qwen 3.6 35B-A3B ↓5 0.84
ruMMLU acc.business_ethics
102 models we track have a score here.
- 1
DeepSeek-V3 Mar 2025 ↑23 0.86
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 0.86
- 3 Kimi K2.5 ↑8 0.85
- 4
MiniMax-M2.7 ↑14 0.85
- 5
Llama 3-70B ↑38 0.84
- 6 Claude Opus 4.5 ↑3 0.84
- 7
Gemini 1.5 Pro Sep 2024 ↑19 0.83
- 8 Claude 3.7 Sonnet ↑13 0.83
- 9 Claude Opus 4.6 ↓8 0.83
- 10 GPT-5.4 ↓8 0.83
ruMMLU acc.college_biology
102 models we track have a score here.
- 1 Claude Opus 4.6 0.99
- 2 Claude Opus 4.5 ↑7 0.99
- 3 GPT-5.4 ↓1 0.99
- 4 GLM-5.1 ↓1 0.99
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 0.98
- 6 Kimi K2.5 ↑5 0.98
- 7
Qwen3.5-122B-A10B ↑1 0.98
- 8
Qwen3.6 27B ↓2 0.98
- 9
Qwen3.5-27B ↓2 0.97
- 10 GPT-5.2 ↑7 0.97
ruMMLU acc.college_physics
102 models we track have a score here.
- 1
Qwen3-235B-A22B-Thinking Jul 2025 ↑3 0.99
- 2 Kimi K2.5 ↑9 0.99
- 3 Claude Opus 4.6 ↓2 0.99
- 4
Qwen3.5-27B ↑3 0.99
- 5 GPT-5.4 ↓3 0.99
- 6
Qwen3.6 27B 0.99
- 7 Claude Opus 4.5 ↑2 0.98
- 8
Qwen3.5-122B-A10B 0.98
- 9
Qwen3.5-35B-A3B ↑4 0.98
- 10
Qwen3.5-9B ↑13 0.98
ruMMLU acc.human_sexuality
102 models we track have a score here.
- 1 Claude Opus 4.6 0.95
- 2 GLM-5.1 ↑1 0.94
- 3 Claude Opus 4.5 ↑6 0.92
- 4 Claude 3.7 Sonnet ↑17 0.91
- 5
Qwen3.5-122B-A10B ↑3 0.91
- 6
Qwen3.6 27B 0.91
- 7
Qwen3-235B-A22B-Thinking Jul 2025 ↓3 0.90
- 8 Kimi K2.5 ↑3 0.90
- 9
Qwen3.5-27B ↓2 0.90
- 10
Qwen3.5-35B-A3B ↑3 0.90
ruMMLU acc.moral_scenarios
102 models we track have a score here.
- 1 Claude Opus 4.6 0.90
- 2 GPT-5.4 0.87
- 3 Claude Opus 4.5 ↑6 0.87
- 4 GLM-5.1 ↓1 0.84
- 5
Qwen3.5-27B ↑2 0.83
- 6
Qwen3.5-122B-A10B ↑2 0.82
- 7
Qwen3.6 27B ↓1 0.82
- 8 GPT-5.2 ↑9 0.81
- 9 Claude 3.7 Sonnet ↑12 0.80
- 10
Qwen3.5-35B-A3B ↑3 0.80
ruMMLU acc.world_religions
102 models we track have a score here.
- 1
Qwen3.5-27B ↑6 0.92
- 2 GPT-5.4 0.92
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 0.91
- 4 Claude Opus 4.6 ↓3 0.91
- 5
Gemma 4 31B IT ↑10 0.91
- 6
Qwen3.6 27B 0.91
- 7 Kimi K2.5 ↑4 0.90
- 8 GLM-5.1 ↓5 0.90
- 9 Claude 3.7 Sonnet ↑12 0.89
- 10
DeepSeek-V3 Mar 2025 ↑14 0.89
ruMMLU acc.abstract_algebra
102 models we track have a score here.
- 1 Kimi K2.5 ↑10 0.96
- 2 GLM-5.1 ↑1 0.96
- 3
Qwen3.6 27B ↑3 0.96
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.95
- 5 Claude Opus 4.6 ↓4 0.95
- 6
Qwen3.5-27B ↑1 0.95
- 7 GPT-5.4 ↓5 0.95
- 8
Qwen 3.6 35B-A3B ↓3 0.95
- 9
Qwen3-30B-A3B-Thinking Jul 2025 ↑1 0.94
- 10
Qwen3.5-35B-A3B ↑3 0.94
ruMMLU acc.college_medicine
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.91
- 2 Claude Opus 4.6 ↓1 0.91
- 3 Claude Opus 4.5 ↑6 0.90
- 4
Qwen3.6 27B ↑2 0.90
- 5 GLM-5.1 ↓2 0.89
- 6
Qwen3.5-122B-A10B ↑2 0.88
- 7
Qwen3.5-27B 0.88
- 8 GPT-5.2 ↑9 0.87
- 9
Qwen3.5-35B-A3B ↑4 0.87
- 10
MiniMax-M2.7 ↑8 0.86
ruMMLU acc.machine_learning
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.94
- 2 GLM-5.1 ↑1 0.93
- 3 GPT-5.2 ↑14 0.91
- 4 Kimi K2.5 ↑7 0.91
- 5 Claude Opus 4.6 ↓4 0.89
- 6
Qwen3.5-27B ↑1 0.89
- 7
MiniMax-M2.7 ↑11 0.89
- 8
Qwen3.5-122B-A10B 0.88
- 9
Qwen3.6 27B ↓3 0.88
- 10
Qwen 3.6 35B-A3B ↓5 0.87
ruMMLU acc.medical_genetics
102 models we track have a score here.
- 1 Kimi K2.5 ↑10 1.00
- 2 Claude Opus 4.6 ↓1 1.00
- 3
Qwen3.5-122B-A10B ↑5 1.00
- 4
Qwen3.5-35B-A3B ↑9 1.00
- 5 GPT-5.4 ↓3 1.00
- 6 GLM-5.1 ↓3 1.00
- 7 GPT-5.2 ↑10 0.99
- 8
Qwen3.5-27B ↓1 0.99
- 9
Qwen3.6 27B ↓3 0.99
- 10 Claude 3.7 Sonnet ↑11 0.98
ruMMLU acc.professional_law
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.86
- 2 Claude Opus 4.6 ↓1 0.84
- 3 GLM-5.1 0.81
- 4 Claude Opus 4.5 ↑5 0.80
- 5 GPT-5.2 ↑12 0.79
- 6 Kimi K2.5 ↑5 0.77
- 7
Qwen3.5-27B 0.76
- 8
Qwen3.5-122B-A10B 0.75
- 9
Qwen3.6 27B ↓3 0.75
- 10
Qwen3.5-35B-A3B ↑3 0.72
ruMMLU acc.public_relations
102 models we track have a score here.
- 1 Claude Opus 4.6 0.81
- 2
Qwen3.5-27B ↑5 0.81
- 3 Claude Opus 4.5 ↑6 0.79
- 4 GPT-5.4 ↓2 0.79
- 5
Gemini 2.0 Flash Feb 2025 ↑17 0.78
- 6
Qwen3.5-122B-A10B ↑2 0.77
- 7 Claude 3.7 Sonnet ↑14 0.76
- 8
DeepSeek-V3 Mar 2025 ↑16 0.75
- 9 GPT-4o Mar 2025 ↑21 0.75
- 10 Kimi K2.5 ↑1 0.75
ruMMLU acc.security_studies
102 models we track have a score here.
- 1 Claude Opus 4.6 0.87
- 2 Claude Opus 4.5 ↑7 0.87
- 3 Claude 3.7 Sonnet ↑18 0.84
- 4 GPT-5.4 ↓2 0.84
- 5 Kimi K2.5 ↑6 0.84
- 6
Qwen3.5-27B ↑1 0.84
- 7 GLM-5.1 ↓4 0.84
- 8
Gemini 1.5 Pro Sep 2024 ↑18 0.83
- 9
Qwen3.5-35B-A3B ↑4 0.83
- 10
Qwen3.6 27B ↓4 0.83
ruMMLU acc.college_chemistry
102 models we track have a score here.
- 1
Qwen3.5-35B-A3B ↑12 0.78
- 2 GPT-5.4 0.78
- 3
Qwen3.6 27B ↑3 0.78
- 4 Claude Opus 4.6 ↓3 0.77
- 5 GLM-5.1 ↓2 0.77
- 6
Qwen3.5-122B-A10B ↑2 0.76
- 7
Qwen3.5-27B 0.76
- 8 Kimi K2.5 ↑3 0.75
- 9
Qwen3.5-9B ↑14 0.75
- 10
MiniMax-M2.7 ↑8 0.75
ruMMLU acc.computer_security
102 models we track have a score here.
- 1 Kimi K2.5 ↑10 0.92
- 2 Claude Opus 4.6 ↓1 0.91
- 3 GPT-5.4 ↓1 0.91
- 4 GLM-5.1 ↓1 0.91
- 5 Claude Opus 4.5 ↑4 0.90
- 6
Qwen3.6 27B 0.90
- 7
Qwen3.5-27B 0.88
- 8
Qwen3.5-35B-A3B ↑5 0.88
- 9
MiniMax-M2.7 ↑9 0.87
- 10 Claude 3.7 Sonnet ↑11 0.86
ruMMLU acc.international_law
102 models we track have a score here.
- 1
Qwen3.5-27B ↑6 0.97
- 2 Kimi K2.5 ↑9 0.96
- 3 Claude 3.7 Sonnet ↑18 0.95
- 4
Qwen3.5-122B-A10B ↑4 0.94
- 5
Llama-3.1-Nemotron-70B-Instruct ↑43 0.93
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.93
- 7
Qwen3-Next-80B-A3B ↑7 0.93
- 8 Claude Opus 4.6 ↓7 0.93
- 9
DeepSeek-V4-Flash ↑16 0.93
- 10
DeepSeek-V4-Pro ↑10 0.93
ruMMLU acc.logical_fallacies
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.92
- 2 Kimi K2.5 ↑9 0.91
- 3
Qwen3.6 27B ↑3 0.91
- 4 Claude Opus 4.5 ↑5 0.90
- 5 Claude Opus 4.6 ↓4 0.90
- 6
Qwen3.5-27B ↑1 0.90
- 7 GLM-5.1 ↓4 0.88
- 8
Qwen3.5-122B-A10B 0.87
- 9 Claude 3.7 Sonnet ↑12 0.87
- 10
Qwen 3.6 35B-A3B ↓5 0.87
ruMMLU acc.us_foreign_policy
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2 Claude 3.7 Sonnet ↑19 0.96
- 3
Gemini 2.0 Flash Feb 2025 ↑19 0.95
- 4 Claude Opus 4.5 ↑5 0.95
- 5 GPT-5.4 ↓3 0.95
- 6
Qwen2-72B ↑33 0.94
- 7 GLM-5.1 ↓4 0.94
- 8
Qwen2.5-72B ↑25 0.93
- 9
Llama-3.1-Nemotron-70B-Instruct ↑39 0.93
- 10
Qwen3.5-122B-A10B ↓2 0.93
ruMMLU acc.clinical_knowledge
102 models we track have a score here.
- 1 Claude Opus 4.6 0.94
- 2 GLM-5.1 ↑1 0.94
- 3 Claude Opus 4.5 ↑6 0.93
- 4 GPT-5.4 ↓2 0.93
- 5
Qwen 3.6 35B-A3B 0.92
- 6 GPT-5.2 ↑11 0.92
- 7
Qwen3.5-122B-A10B ↑1 0.91
- 8 Kimi K2.5 ↑3 0.91
- 9
Qwen3.6 27B ↓3 0.91
- 10
Qwen3.5-35B-A3B ↑3 0.91
ruMMLU acc.conceptual_physics
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2 Kimi K2.5 ↑9 0.96
- 3 GPT-5.4 ↓1 0.96
- 4 Claude Opus 4.5 ↑5 0.96
- 5
Qwen3.5-122B-A10B ↑3 0.96
- 6
Qwen3.5-27B ↑1 0.96
- 7
Qwen 3.6 35B-A3B ↓2 0.96
- 8
Qwen3.6 27B ↓2 0.96
- 9
Qwen3-32B ↑3 0.95
- 10
Qwen3.5-9B ↑13 0.94
ruMMLU acc.college_mathematics
102 models we track have a score here.
- 1 Claude Opus 4.6 0.99
- 2 GPT-5.4 0.98
- 3
Qwen3.6 27B ↑3 0.98
- 4 Kimi K2.5 ↑7 0.97
- 5
Qwen3.5-122B-A10B ↑3 0.97
- 6
Qwen3.5-27B ↑1 0.97
- 7 GLM-5.1 ↓4 0.97
- 8
Qwen 3.6 35B-A3B ↓3 0.97
- 9
Qwen3.5-35B-A3B ↑4 0.96
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 0.95
ruMMLU acc.high_school_biology
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.97
- 2 Claude Opus 4.6 ↓1 0.97
- 3 GLM-5.1 0.97
- 4
Qwen3.5-27B ↑3 0.96
- 5
Qwen 3.6 35B-A3B 0.96
- 6
Qwen3.6 27B 0.96
- 7 GPT-5.4 ↓5 0.96
- 8
Qwen3.5-122B-A10B 0.95
- 9
Qwen2.5-Max ↑19 0.95
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 0.95
ruMMLU acc.high_school_physics
102 models we track have a score here.
- 1 Claude Opus 4.6 0.96
- 2 GPT-5.4 0.95
- 3
Qwen3.5-122B-A10B ↑5 0.95
- 4 GLM-5.1 ↓1 0.95
- 5 Kimi K2.5 ↑6 0.94
- 6
Qwen3.5-27B ↑1 0.94
- 7
Qwen 3.6 35B-A3B ↓2 0.94
- 8
Qwen3.6 27B ↓2 0.94
- 9 Claude Opus 4.5 0.93
- 10
Qwen3.5-35B-A3B ↑3 0.93
ruMMLU acc.high_school_chemistry
102 models we track have a score here.
- 1
Qwen3.5-122B-A10B ↑7 0.97
- 2 Claude Opus 4.6 ↓1 0.96
- 3 GLM-5.1 0.96
- 4
Qwen3.5-27B ↑3 0.96
- 5 GPT-5.4 ↓3 0.96
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.95
- 7 Kimi K2.5 ↑4 0.95
- 8
Qwen3.6 27B ↓2 0.95
- 9
Qwen3-32B ↑3 0.94
- 10
Qwen3.5-35B-A3B ↑3 0.94
ruMMLU acc.high_school_geography
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.94
- 2 Claude Opus 4.6 ↓1 0.94
- 3 GLM-5.1 0.94
- 4 Claude 3.7 Sonnet ↑17 0.93
- 5 GPT-5.4 ↓3 0.93
- 6
Qwen2-72B ↑33 0.92
- 7
Qwen3.5-27B 0.92
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.91
- 9
Qwen3.5-122B-A10B ↓1 0.91
- 10 GPT-5.2 ↑7 0.91
ruMMLU acc.professional_medicine
102 models we track have a score here.
- 1 GPT-5.2 ↑16 0.97
- 2 Claude Opus 4.6 ↓1 0.97
- 3 GPT-5.4 ↓1 0.97
- 4 GLM-5.1 ↓1 0.97
- 5 Claude Opus 4.5 ↑4 0.97
- 6
Qwen3.5-27B ↑1 0.96
- 7
Qwen 3.6 35B-A3B ↓2 0.96
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.95
- 9
Qwen3.6 27B ↓3 0.95
- 10 Claude 3.7 Sonnet ↑11 0.94
ruMMLU acc.electrical_engineering
102 models we track have a score here.
- 1 Claude Opus 4.6 0.89
- 2 GPT-5.4 0.89
- 3 GLM-5.1 0.89
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.88
- 5 Kimi K2.5 ↑6 0.88
- 6
Llama 4 Maverick ↑30 0.86
- 7
Qwen3.5-122B-A10B ↑1 0.86
- 8 GPT-5.2 ↑9 0.86
- 9 Claude Opus 4.5 0.85
- 10
Qwen3.5-27B ↓3 0.85
ruMMLU acc.elementary_mathematics
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 0.97
- 3 GPT-5.4 ↓1 0.97
- 4 GLM-5.1 ↓1 0.97
- 5 Kimi K2.5 ↑6 0.97
- 6
Qwen3.5-122B-A10B ↑2 0.97
- 7
Qwen3.5-27B 0.96
- 8
Qwen3.6 27B ↓2 0.96
- 9
Qwen3-Next-80B-A3B ↑5 0.96
- 10
Qwen3-32B ↑2 0.96
ruMMLU acc.high_school_psychology
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2 GPT-5.4 0.97
- 3 GLM-5.1 0.96
- 4 Claude Opus 4.5 ↑5 0.96
- 5 GPT-5.2 ↑12 0.96
- 6
Qwen3.6 27B 0.96
- 7 Kimi K2.5 ↑4 0.95
- 8
Qwen3.5-27B ↓1 0.95
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.95
- 10 Claude 3.7 Sonnet ↑11 0.95
ruMMLU acc.high_school_statistics
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2 GPT-5.4 0.94
- 3
Qwen3.5-27B ↑4 0.94
- 4 Kimi K2.5 ↑7 0.93
- 5
Qwen3.6 27B ↑1 0.93
- 6
Qwen3.5-122B-A10B ↑2 0.92
- 7
Qwen3.5-35B-A3B ↑6 0.92
- 8 GLM-5.1 ↓5 0.91
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.91
- 10 Claude Opus 4.5 ↓1 0.91
ruMMLU acc.high_school_us_history
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.96
- 2 Claude Opus 4.6 ↓1 0.96
- 3 Kimi K2.5 ↑8 0.95
- 4 GPT-5.4 ↓2 0.95
- 5
Gemini 2.0 Flash Feb 2025 ↑17 0.95
- 6 GPT-5.2 ↑11 0.95
- 7
Qwen3.5-122B-A10B ↑1 0.95
- 8 GLM-5.1 ↓5 0.95
- 9 Claude 3.7 Sonnet ↑12 0.94
- 10
Qwen3.6 27B ↓4 0.94
ruMMLU acc.high_school_mathematics
102 models we track have a score here.
- 1 GLM-5.1 ↑2 0.99
- 2 Kimi K2.5 ↑9 0.99
- 3 Claude Opus 4.6 ↓2 0.99
- 4
Qwen3.5-27B ↑3 0.99
- 5
Qwen3-30B-A3B-Thinking Jul 2025 ↑5 0.99
- 6 GPT-5.4 ↓4 0.99
- 7
Qwen3.6 27B ↓1 0.99
- 8
Qwen3.5-122B-A10B 0.98
- 9
Qwen 3.6 35B-A3B ↓4 0.98
- 10
Qwen3.5-35B-A3B ↑3 0.98
ruMMLU acc.professional_accounting
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2 GPT-5.4 0.96
- 3 GLM-5.1 0.94
- 4
Qwen3.5-122B-A10B ↑4 0.94
- 5
Qwen3.5-27B ↑2 0.93
- 6
Qwen3.6 27B 0.92
- 7 Claude Opus 4.5 ↑2 0.91
- 8 Kimi K2.5 ↑3 0.91
- 9
Qwen 3.6 35B-A3B ↓4 0.91
- 10 GPT-5.2 ↑7 0.90
ruMMLU acc.professional_psychology
102 models we track have a score here.
- 1 Claude Opus 4.6 0.93
- 2 GPT-5.4 0.92
- 3 Claude Opus 4.5 ↑6 0.92
- 4 GLM-5.1 ↓1 0.91
- 5 Kimi K2.5 ↑6 0.90
- 6 GPT-5.2 ↑11 0.90
- 7
Qwen3.5-122B-A10B ↑1 0.89
- 8
Qwen3.5-27B ↓1 0.88
- 9
Qwen3.6 27B ↓3 0.88
- 10 Claude 3.7 Sonnet ↑11 0.87
ruMMLU acc.college_computer_science
102 models we track have a score here.
- 1 Kimi K2.5 ↑10 0.98
- 2 GPT-5.4 0.98
- 3
Qwen3.6 27B ↑3 0.98
- 4 Claude Opus 4.6 ↓3 0.97
- 5
Qwen3.5-27B ↑2 0.97
- 6 GLM-5.1 ↓3 0.97
- 7
Qwen3-235B-A22B-Thinking Jul 2025 ↓3 0.95
- 8
Qwen3.5-122B-A10B 0.95
- 9
Qwen3-Next-80B-A3B ↑5 0.94
- 10
MiniMax-M2.7 ↑8 0.94
ruMMLU acc.high_school_world_history
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2 Claude Opus 4.5 ↑7 0.97
- 3 GPT-5.4 ↓1 0.96
- 4
Qwen3.5-27B ↑3 0.95
- 5 Claude 3.7 Sonnet ↑16 0.94
- 6
Qwen3.6 27B 0.94
- 7
Qwen3.5-122B-A10B ↑1 0.93
- 8 Kimi K2.5 ↑3 0.93
- 9
Gemma 4 31B IT ↑6 0.93
- 10
DeepSeek-V4-Flash ↑15 0.92
ruMMLU acc.high_school_macroeconomics
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2 GLM-5.1 ↑1 0.96
- 3 GPT-5.4 ↓1 0.96
- 4 Claude Opus 4.5 ↑5 0.96
- 5
Qwen3.5-122B-A10B ↑3 0.95
- 6
Qwen3.5-27B ↑1 0.95
- 7 Kimi K2.5 ↑4 0.95
- 8 GPT-5.2 ↑9 0.94
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.94
- 10
Qwen3.6 27B ↓4 0.94
ruMMLU acc.high_school_microeconomics
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.99
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 0.98
- 3 Kimi K2.5 ↑8 0.98
- 4 Claude Opus 4.6 ↓3 0.98
- 5 GLM-5.1 ↓2 0.98
- 6
Qwen3.5-122B-A10B ↑2 0.98
- 7
Qwen3.5-27B 0.97
- 8 Claude Opus 4.5 ↑1 0.97
- 9
DeepSeek-V3 Mar 2025 ↑15 0.97
- 10
Qwen3-30B-A3B-Thinking Jul 2025 0.97
ruMMLU acc.high_school_computer_science
102 models we track have a score here.
- 1
Qwen3.5-122B-A10B ↑7 0.98
- 2 GLM-5.1 ↑1 0.98
- 3
Qwen3.6 27B ↑3 0.98
- 4
Gemini 1.5 Pro Sep 2024 ↑22 0.97
- 5 Claude Opus 4.6 ↓4 0.97
- 6
Qwen3.5-27B ↑1 0.97
- 7
Qwen3.5-35B-A3B ↑6 0.97
- 8 GPT-5.4 ↓6 0.97
- 9
MiniMax-M2.7 ↑9 0.97
- 10
Qwen 3.6 35B-A3B ↓5 0.97
ruMMLU acc.high_school_european_history
102 models we track have a score here.
- 1 Claude Opus 4.6 0.93
- 2 Claude Opus 4.5 ↑7 0.92
- 3
Qwen3.5-122B-A10B ↑5 0.90
- 4
Qwen3.5-27B ↑3 0.90
- 5 GPT-5.4 ↓3 0.90
- 6
Qwen3.6 27B 0.90
- 7 GLM-5.1 ↓4 0.90
- 8 GPT-4o Mar 2025 ↑22 0.88
- 9 GPT-5.2 ↑8 0.88
- 10
Qwen2.5-72B ↑23 0.88
ruMMLU acc.high_school_government_and_politics
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.99
- 2 Claude Opus 4.6 ↓1 0.98
- 3 GPT-5.4 ↓1 0.98
- 4
Qwen3.5-122B-A10B ↑4 0.97
- 5
Qwen3.5-27B ↑2 0.97
- 6 GLM-5.1 ↓3 0.97
- 7 Claude 3.7 Sonnet ↑14 0.96
- 8
Qwen3.6 27B ↓2 0.96
- 9
Gemini 2.0 Flash Feb 2025 ↑13 0.96
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 0.96
CheGeKa em
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.66
- 2 Claude Opus 4.6 ↓1 0.63
- 3 GLM-5.1 0.57
- 4 Claude Opus 4.5 ↑5 0.56
- 5 GPT-5.2 ↑12 0.54
- 6
Gemini 1.5 Pro Sep 2024 ↑20 0.53
- 7 Claude 3.7 Sonnet ↑14 0.53
- 8
Gemini 2.0 Flash Feb 2025 ↑14 0.50
- 9
DeepSeek-V4-Flash ↑16 0.48
- 10
DeepSeek-V4-Pro ↑10 0.48
CheGeKa f1
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.75
- 2 Claude Opus 4.6 ↓1 0.74
- 3 Claude Opus 4.5 ↑6 0.68
- 4 GLM-5.1 ↓1 0.68
- 5
Gemini 1.5 Pro Sep 2024 ↑21 0.63
- 6 Claude 3.7 Sonnet ↑15 0.63
- 7 GPT-5.2 ↑10 0.62
- 8 Kimi K2.5 ↑3 0.59
- 9
Gemini 2.0 Flash Feb 2025 ↑13 0.59
- 10
DeepSeek-V4-Flash ↑15 0.58
ruDetox j
101 models we track have a score here.
- 1
Gemma 4 31B IT ↑14 0.41
- 2 Claude Opus 4.6 ↓1 0.39
- 3 Claude Opus 4.5 ↑6 0.39
- 4 GPT-5.4 ↓2 0.39
- 5 Kimi K2.5 ↑6 0.38
- 6
Gemma 4 26B A4B ↑31 0.38
- 7 GPT-4o Mar 2025 ↑23 0.38
- 8
DeepSeek-V3 Mar 2025 ↑16 0.38
- 9 Claude 3.7 Sonnet ↑12 0.37
- 10
Qwen2.5-72B ↑23 0.37
ruDetox fl
101 models we track have a score here.
- 1
Gemini 1.5 Flash 8B ↑46 0.79
- 2
Gemma 3 27B ↑39 0.79
- 3
Gemini 1.5 Flash Sep 2024 ↑29 0.78
- 4
Qwen2-72B ↑35 0.78
- 5
Gemma 2 2B IT ↑78 0.78
- 6
Gemini 1.5 Pro Sep 2024 ↑20 0.78
- 7
Gemma 4 26B A4B ↑30 0.77
- 8
DeepSeek-V3 Mar 2025 ↑16 0.77
- 9
Gemma 4 31B IT ↑6 0.76
- 10
Command R (03-2024) ↑54 0.75
ruDetox sim
101 models we track have a score here.
- 1
Mistral 7B v0.1 ↑75 0.88
- 2
Mistral 7B v0.3 ↑77 0.87
- 3
Llama 3-8B ↑66 0.87
- 4
Llama 3.2 3B ↑83 0.86
- 5
Llama 3.1-8B ↑62 0.86
- 6
Llama 3-70B ↑37 0.86
- 7
Mixtral 8x7B ↑48 0.85
- 8
Qwen1.5-4B-Chat ↑87 0.84
- 9
Yi 6B ↑84 0.84
- 10 GLM-5.1 ↓7 0.84
ruDetox sta
101 models we track have a score here.
- 1
Qwen3.5-4B ↑41 0.90
- 2
Qwen3.5-9B ↑21 0.87
- 3
Qwen3.5-35B-A3B ↑10 0.82
- 4 Claude 3.5 Haiku ↑42 0.82
- 5
Qwen3-Next-80B-A3B ↑9 0.81
- 6
Qwen1.5-7B-Chat ↑80 0.81
- 7
Gemini 1.5 Flash Sep 2024 ↑25 0.80
- 8
Gemma 3 27B ↑33 0.80
- 9 GPT-5.2 ↑8 0.80
- 10
phi-3-small 7.4B ↑52 0.80
ruModAr acc
37 models we track have a score here.
- 1
Qwen3.6 27B ↑5 1.00
- 2
Qwen3.5-27B ↑5 1.00
- 3
Qwen3.5-122B-A10B ↑5 1.00
- 4 GPT-5.4 ↓2 1.00
- 5 GLM-5.1 ↓2 1.00
- 6
Qwen3-30B-A3B-Thinking Jul 2025 ↑4 1.00
- 7
Qwen3-Next-80B-A3B ↑7 1.00
- 8
Qwen3.5-35B-A3B ↑5 1.00
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 1.00
- 10
Qwen3.5-9B ↑13 1.00
ruModAr em
65 models we track have a score here.
- 1 gpt-oss-20b ↑34 0.94
- 2 Claude 3.7 Sonnet ↑19 0.92
- 3
Gemma 2 27B ↑47 0.87
- 4
Qwen2.5-72B ↑29 0.85
- 5
Llama 3.1-70B ↑33 0.83
- 6
DeepSeek-R1-Distill-Qwen-32B ↑28 0.83
- 7
Llama 3-70B ↑36 0.83
- 8
Gemini 2.0 Flash Feb 2025 ↑14 0.82
- 9
DeepSeek-V3 Mar 2025 ↑15 0.81
- 10
Qwen2.5-Max ↑18 0.79
MaMuRAMu acc
102 models we track have a score here.
- 1 Claude Opus 4.6 0.93
- 2 GPT-5.4 0.92
- 3 Claude Opus 4.5 ↑6 0.92
- 4 GLM-5.1 ↓1 0.91
- 5 GPT-5.2 ↑12 0.90
- 6
Qwen3.5-122B-A10B ↑2 0.90
- 7 Kimi K2.5 ↑4 0.90
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.89
- 9
Qwen3.5-27B ↓2 0.89
- 10 Claude 3.7 Sonnet ↑11 0.89
MaMuRAMu acc.anatomy
102 models we track have a score here.
- 1 Claude Opus 4.6 0.96
- 2 Claude Opus 4.5 ↑7 0.93
- 3
Qwen3.5-122B-A10B ↑5 0.91
- 4 GPT-5.4 ↓2 0.91
- 5
DeepSeek-V3 Mar 2025 ↑19 0.89
- 6 GPT-4o Mar 2025 ↑24 0.89
- 7 Kimi K2.5 ↑4 0.89
- 8
DeepSeek-V4-Flash ↑17 0.89
- 9
DeepSeek-V4-Pro ↑11 0.89
- 10
Gemini 2.0 Flash Feb 2025 ↑12 0.87
MaMuRAMu acc.virology
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2 GPT-5.4 0.97
- 3
DeepSeek-V3 Mar 2025 ↑21 0.96
- 4 GLM-5.1 ↓1 0.96
- 5 Claude Opus 4.5 ↑4 0.95
- 6 GPT-5.2 ↑11 0.95
- 7
Qwen2-72B ↑32 0.94
- 8 GPT-4o Mar 2025 ↑22 0.94
- 9
Qwen3.5-122B-A10B ↓1 0.94
- 10
Qwen3.5-27B ↓3 0.94
MaMuRAMu acc.astronomy
102 models we track have a score here.
- 1 GLM-5.1 ↑2 0.93
- 2
Gemini 1.5 Pro Sep 2024 ↑24 0.88
- 3
Qwen2.5-Max ↑25 0.88
- 4 Claude Opus 4.5 ↑5 0.88
- 5 Claude Opus 4.6 ↓4 0.88
- 6
Qwen3.5-122B-A10B ↑2 0.88
- 7
DeepSeek-V4-Flash ↑18 0.88
- 8
DeepSeek-V4-Pro ↑12 0.88
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.87
- 10 Kimi K2.5 ↑1 0.87
MaMuRAMu acc.marketing
102 models we track have a score here.
- 1
Qwen3.5-27B ↑6 0.84
- 2 Claude Opus 4.6 ↓1 0.83
- 3
Qwen2.5-72B ↑30 0.81
- 4 Kimi K2.5 ↑7 0.81
- 5 GPT-5.4 ↓3 0.81
- 6 GLM-5.1 ↓3 0.81
- 7
Qwen3.6 27B ↓1 0.81
- 8 Claude Opus 4.5 ↑1 0.81
- 9
Qwen3.5-35B-A3B ↑4 0.81
- 10
Qwen3.5-122B-A10B ↓2 0.80
MaMuRAMu acc.nutrition
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.99
- 2 GLM-5.1 ↑1 0.99
- 3 Claude Opus 4.5 ↑6 0.97
- 4
Qwen3-32B ↑8 0.96
- 5 GPT-5.2 ↑12 0.96
- 6 Kimi K2.5 ↑5 0.96
- 7 Claude Opus 4.6 ↓6 0.96
- 8
Qwen3.5-27B ↓1 0.96
- 9
Qwen3.6 27B ↓3 0.96
- 10
Gemini 1.5 Pro Sep 2024 ↑16 0.95
MaMuRAMu acc.sociology
102 models we track have a score here.
- 1 GPT-5.2 ↑16 0.93
- 2 Claude Opus 4.6 ↓1 0.93
- 3 GPT-5.4 ↓1 0.93
- 4
Qwen2.5-72B ↑29 0.91
- 5
Qwen2-72B ↑34 0.90
- 6
Gemini 2.0 Flash-Lite ↑25 0.90
- 7 Claude Opus 4.5 ↑2 0.90
- 8 Kimi K2.5 ↑3 0.90
- 9
Qwen2.5-Max ↑19 0.88
- 10 Claude 3.7 Sonnet ↑11 0.88
MaMuRAMu acc.management
102 models we track have a score here.
- 1
DeepSeek-V3 Mar 2025 ↑23 0.88
- 2 Claude 3.7 Sonnet ↑19 0.86
- 3
Qwen3.5-27B ↑4 0.86
- 4 GPT-5.4 ↓2 0.86
- 5 GLM-5.1 ↓2 0.86
- 6 Claude Opus 4.5 ↑3 0.84
- 7 Claude Opus 4.6 ↓6 0.84
- 8
Qwen3.5-122B-A10B 0.84
- 9
Qwen3.5-35B-A3B ↑4 0.84
- 10
Qwen3.6 27B ↓4 0.84
MaMuRAMu acc.philosophy
102 models we track have a score here.
- 1 Kimi K2.5 ↑10 0.84
- 2
Gemini 2.0 Flash Feb 2025 ↑20 0.82
- 3 GPT-5.4 ↓1 0.82
- 4 GLM-5.1 ↓1 0.82
- 5 Claude Opus 4.5 ↑4 0.81
- 6 Claude Opus 4.6 ↓5 0.81
- 7 Claude 3.7 Sonnet ↑14 0.79
- 8
Gemma 3 27B ↑33 0.79
- 9
DeepSeek-V3 Mar 2025 ↑15 0.79
- 10 GPT-5.2 ↑7 0.79
MaMuRAMu acc.prehistory
102 models we track have a score here.
- 1
Qwen3-235B-A22B-Thinking Jul 2025 ↑3 0.98
- 2 Claude Opus 4.5 ↑7 0.98
- 3
Qwen3.5-27B ↑4 0.98
- 4 GLM-5.1 ↓1 0.98
- 5
DeepSeek-V4-Flash ↑20 0.98
- 6
DeepSeek-V4-Pro ↑14 0.98
- 7 GPT-5.2 ↑10 0.96
- 8 Kimi K2.5 ↑3 0.96
- 9 Claude Opus 4.6 ↓8 0.96
- 10 GPT-5.4 ↓8 0.96
MaMuRAMu acc.human_aging
102 models we track have a score here.
- 1
Qwen2.5-Max ↑27 0.85
- 2
DeepSeek-V3 Mar 2025 ↑22 0.85
- 3
Llama 4 Maverick ↑33 0.85
- 4 Claude Opus 4.6 ↓3 0.85
- 5
Qwen2-72B ↑34 0.83
- 6
Qwen2.5-72B ↑27 0.83
- 7
Gemini 1.5 Pro Sep 2024 ↑19 0.83
- 8 Claude 3.7 Sonnet ↑13 0.83
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.83
- 10 GPT-5.2 ↑7 0.83
MaMuRAMu acc.econometrics
102 models we track have a score here.
- 1
Qwen2.5-Math-72B-Instruct ↑52 0.88
- 2
Qwen3-32B ↑10 0.88
- 3 Claude 3.7 Sonnet ↑18 0.87
- 4 Claude Opus 4.5 ↑5 0.87
- 5
Qwen2-72B ↑34 0.86
- 6
Qwen2.5-72B ↑27 0.86
- 7
Qwen 3.6 35B-A3B ↓2 0.86
- 8 GPT-5.4 ↓6 0.85
- 9
Gemma 4 31B IT ↑6 0.85
- 10 GLM-5.1 ↓7 0.85
MaMuRAMu acc.formal_logic
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.89
- 2 GPT-5.4 0.89
- 3 GLM-5.1 0.89
- 4 Kimi K2.5 ↑7 0.88
- 5 Claude Opus 4.6 ↓4 0.88
- 6
Qwen3.6 27B 0.88
- 7
Qwen2.5-Max ↑21 0.87
- 8 Claude 3.7 Sonnet ↑13 0.87
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.87
- 10
Qwen3.5-122B-A10B ↓2 0.87
MaMuRAMu acc.global_facts
102 models we track have a score here.
- 1 Claude Opus 4.6 0.76
- 2 GPT-5.4 0.74
- 3 Claude Opus 4.5 ↑6 0.73
- 4 GPT-5.2 ↑13 0.72
- 5 GLM-5.1 ↓2 0.72
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.71
- 7 Kimi K2.5 ↑4 0.71
- 8
MiniMax-M2.7 ↑10 0.69
- 9
Qwen3.5-35B-A3B ↑4 0.68
- 10
Gemini 1.5 Pro Sep 2024 ↑16 0.68
MaMuRAMu acc.jurisprudence
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2
DeepSeek-V4-Flash ↑23 0.97
- 3
DeepSeek-V4-Pro ↑17 0.97
- 4 GPT-5.4 ↓2 0.96
- 5 Claude Opus 4.5 ↑4 0.95
- 6
Qwen3.5-27B ↑1 0.95
- 7 GLM-5.1 ↓4 0.95
- 8 GPT-5.2 ↑9 0.94
- 9 Claude 3.7 Sonnet ↑12 0.93
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 0.93
MaMuRAMu acc.miscellaneous
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.96
- 2 Claude Opus 4.5 ↑7 0.95
- 3 Claude Opus 4.6 ↓2 0.95
- 4 GPT-5.2 ↑13 0.93
- 5 GLM-5.1 ↓2 0.93
- 6 Kimi K2.5 ↑5 0.92
- 7
Gemini 2.0 Flash Feb 2025 ↑15 0.92
- 8
DeepSeek-V3 Mar 2025 ↑16 0.92
- 9
MiniMax-M2.7 ↑9 0.92
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 0.91
MaMuRAMu acc.foreign_policy
102 models we track have a score here.
- 1
DeepSeek-V3 Mar 2025 ↑23 0.98
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 0.98
- 3 Claude Opus 4.5 ↑6 0.98
- 4 GPT-5.4 ↓2 0.98
- 5
Qwen2-72B ↑34 0.96
- 6 GPT-4o Mar 2025 ↑24 0.96
- 7
Qwen3-Next-80B-A3B ↑7 0.96
- 8 Claude Opus 4.6 ↓7 0.96
- 9
Qwen3.5-27B ↓2 0.96
- 10
Qwen 3.6 35B-A3B ↓5 0.96
MaMuRAMu acc.moral_disputes
102 models we track have a score here.
- 1 GLM-5.1 ↑2 0.88
- 2 Claude Opus 4.6 ↓1 0.86
- 3 GPT-5.4 ↓1 0.86
- 4 Kimi K2.5 ↑7 0.85
- 5
Qwen3.5-122B-A10B ↑3 0.85
- 6
Qwen2.5-Max ↑22 0.83
- 7
Qwen3-Next-80B-A3B ↑7 0.83
- 8 Claude Opus 4.5 ↑1 0.83
- 9
Qwen 3.6 35B-A3B ↓4 0.83
- 10
DeepSeek-V4-Flash ↑15 0.83
MaMuRAMu acc.business_ethics
102 models we track have a score here.
- 1 GPT-5.2 ↑16 0.88
- 2 Claude Opus 4.6 ↓1 0.86
- 3
Qwen2-72B ↑36 0.85
- 4
Qwen3.5-122B-A10B ↑4 0.85
- 5
Qwen3.5-27B ↑2 0.85
- 6 Claude 3.7 Sonnet ↑15 0.84
- 7
DeepSeek-V3 Mar 2025 ↑17 0.84
- 8
Qwen3-32B ↑4 0.83
- 9
Qwen 3.6 35B-A3B ↓4 0.83
- 10
Qwen3.6 27B ↓4 0.83
MaMuRAMu acc.college_biology
102 models we track have a score here.
- 1
Gemini 2.0 Flash-Lite ↑30 0.87
- 2
DeepSeek-V3 Mar 2025 ↑22 0.87
- 3
Qwen3-8B ↑26 0.87
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.87
- 5
Qwen3.5-27B ↑2 0.87
- 6
Qwen2-72B ↑33 0.84
- 7
Gemini 1.5 Flash Sep 2024 ↑25 0.84
- 8
Gemini 1.5 Pro Sep 2024 ↑18 0.84
- 9
Qwen2.5-Max ↑19 0.84
- 10
Gemma 3 27B ↑31 0.84
MaMuRAMu acc.college_physics
102 models we track have a score here.
- 1
DeepSeek-V3 Mar 2025 ↑23 0.91
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 0.91
- 3
Qwen3-Next-80B-A3B ↑11 0.91
- 4 GPT-5.2 ↑13 0.91
- 5
Qwen3.5-27B ↑2 0.91
- 6
Qwen3.5-35B-A3B ↑7 0.91
- 7
Qwen3.5-4B ↑35 0.91
- 8 GPT-5.4 ↓6 0.91
- 9 GLM-5.1 ↓6 0.91
- 10
Qwen2.5-Max ↑18 0.89
MaMuRAMu acc.human_sexuality
102 models we track have a score here.
- 1 GPT-4o Mar 2025 ↑29 0.89
- 2
Mixtral 8x7B ↑53 0.88
- 3
Llama 3-70B ↑40 0.88
- 4 GPT-4o mini ↑36 0.88
- 5
Llama 3.1-70B ↑33 0.88
- 6
Gemini 2.0 Flash Feb 2025 ↑16 0.88
- 7
Gemma 3 27B ↑34 0.88
- 8
DeepSeek-V3 Mar 2025 ↑16 0.88
- 9
Qwen3-14B ↑10 0.88
- 10
Qwen3-32B ↑2 0.88
MaMuRAMu acc.moral_scenarios
102 models we track have a score here.
- 1 Claude Opus 4.6 0.96
- 2
Qwen3.5-27B ↑5 0.96
- 3 Claude Opus 4.5 ↑6 0.95
- 4
Qwen3.5-122B-A10B ↑4 0.95
- 5 GPT-5.4 ↓3 0.95
- 6 GLM-5.1 ↓3 0.95
- 7 Claude 3.7 Sonnet ↑14 0.93
- 8 GPT-4o Mar 2025 ↑22 0.93
- 9
Qwen3.6 27B ↓3 0.93
- 10
Qwen2-72B ↑29 0.91
MaMuRAMu acc.world_religions
102 models we track have a score here.
- 1
Gemini 2.0 Flash-Lite ↑30 0.98
- 2 Claude 3.7 Sonnet ↑19 0.98
- 3 Claude Opus 4.5 ↑6 0.98
- 4 Kimi K2.5 ↑7 0.98
- 5 Claude Opus 4.6 ↓4 0.98
- 6 GLM-5.1 ↓3 0.98
- 7
Qwen 3.6 35B-A3B ↓2 0.98
- 8
Qwen2.5-72B ↑25 0.97
- 9
DeepSeek-V3 Mar 2025 ↑15 0.97
- 10 GPT-5.2 ↑7 0.97
MaMuRAMu acc.abstract_algebra
102 models we track have a score here.
- 1
Qwen2-72B ↑38 0.96
- 2
Qwen3-30B-A3B-Thinking Jul 2025 ↑8 0.96
- 3
Qwen3-Next-80B-A3B ↑11 0.96
- 4 Kimi K2.5 ↑7 0.96
- 5
Qwen3.5-122B-A10B ↑3 0.96
- 6
Qwen3.5-27B ↑1 0.96
- 7 GPT-5.4 ↓5 0.96
- 8
MiniMax-M2.7 ↑10 0.96
- 9 GLM-5.1 ↓6 0.96
- 10
DeepSeek-V3 Mar 2025 ↑14 0.93
MaMuRAMu acc.college_medicine
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.98
- 2 Claude Opus 4.6 ↓1 0.97
- 3 GPT-5.4 ↓1 0.97
- 4 GPT-5.2 ↑13 0.96
- 5
Qwen3.5-122B-A10B ↑3 0.96
- 6 GLM-5.1 ↓3 0.96
- 7 Claude 3.7 Sonnet ↑14 0.95
- 8
DeepSeek-V3 Mar 2025 ↑16 0.95
- 9
MiniMax-M2.7 ↑9 0.95
- 10
Qwen3-30B-A3B-Thinking Jul 2025 0.95
MaMuRAMu acc.machine_learning
102 models we track have a score here.
- 1
Qwen3-32B ↑11 0.98
- 2 Claude Opus 4.5 ↑7 0.98
- 3
Qwen2.5-Max ↑25 0.96
- 4 Claude 3.7 Sonnet ↑17 0.96
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 0.96
- 6
Qwen3-30B-A3B-Thinking Jul 2025 ↑4 0.96
- 7
Qwen3-Next-80B-A3B ↑7 0.96
- 8 Kimi K2.5 ↑3 0.96
- 9 Claude Opus 4.6 ↓8 0.96
- 10
Qwen3.5-122B-A10B ↓2 0.96
MaMuRAMu acc.medical_genetics
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.98
- 2 Claude Opus 4.5 ↑7 0.97
- 3 GPT-5.2 ↑14 0.97
- 4
Gemma 4 31B IT ↑11 0.97
- 5 GLM-5.1 ↓2 0.97
- 6
Qwen3.6 27B 0.97
- 7 Claude Opus 4.6 ↓6 0.95
- 8
Qwen3.5-122B-A10B 0.95
- 9
Qwen3.5-27B ↓2 0.95
- 10
Qwen3.5-35B-A3B ↑3 0.95
MaMuRAMu acc.professional_law
102 models we track have a score here.
- 1
Qwen3.5-122B-A10B ↑7 0.94
- 2
Gemini 1.5 Pro Sep 2024 ↑24 0.92
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 0.92
- 4 GPT-5.2 ↑13 0.92
- 5
Gemini 2.0 Flash Feb 2025 ↑17 0.91
- 6
Gemini 2.0 Flash-Lite ↑25 0.91
- 7 Claude 3.7 Sonnet ↑14 0.91
- 8
DeepSeek-V3 Mar 2025 ↑16 0.91
- 9 Claude Opus 4.6 ↓8 0.91
- 10 GPT-5.4 ↓8 0.91
MaMuRAMu acc.public_relations
102 models we track have a score here.
- 1 Claude Opus 4.6 0.93
- 2 GPT-5.4 0.88
- 3 Claude Opus 4.5 ↑6 0.86
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.84
- 5
Gemini 1.5 Pro Sep 2024 ↑21 0.82
- 6 Claude 3.7 Sonnet ↑15 0.82
- 7
DeepSeek-V3 Mar 2025 ↑17 0.82
- 8 GPT-5.2 ↑9 0.82
- 9 Kimi K2.5 ↑2 0.82
- 10
Qwen2-72B ↑29 0.81
MaMuRAMu acc.security_studies
102 models we track have a score here.
- 1
Qwen-Turbo ↑44 0.98
- 2 Claude 3.5 Haiku ↑44 0.96
- 3
Gemma 3 27B ↑38 0.96
- 4
Mixtral 8x7B ↑51 0.95
- 5
Gemini 1.5 Flash Sep 2024 ↑27 0.95
- 6
Qwen2.5-72B ↑27 0.95
- 7
Gemini 1.5 Pro Sep 2024 ↑19 0.95
- 8
Gemini 2.0 Flash Feb 2025 ↑14 0.95
- 9 GPT-4o Mar 2025 ↑21 0.95
- 10
Qwen3-32B ↑2 0.95
MaMuRAMu acc.college_chemistry
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.98
- 2
Qwen3.5-35B-A3B ↑11 0.98
- 3 GLM-5.1 0.98
- 4
Qwen3.6 27B ↑2 0.98
- 5
Qwen2.5-Max ↑23 0.96
- 6 GPT-4o Mar 2025 ↑24 0.96
- 7
Qwen3-235B-A22B-Thinking Jul 2025 ↓3 0.96
- 8 GPT-5.2 ↑9 0.96
- 9 Claude Opus 4.6 ↓8 0.96
- 10
Qwen3.5-122B-A10B ↓2 0.96
MaMuRAMu acc.computer_security
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.96
- 2 Claude Opus 4.6 ↓1 0.96
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 0.93
- 4 GPT-5.4 ↓2 0.93
- 5
Gemma 4 31B IT ↑10 0.93
- 6 GLM-5.1 ↓3 0.93
- 7
Gemini 2.0 Flash Feb 2025 ↑15 0.91
- 8
Llama 4 Maverick ↑28 0.91
- 9 GPT-5.2 ↑8 0.91
- 10 Kimi K2.5 ↑1 0.91
MaMuRAMu acc.international_law
102 models we track have a score here.
- 1 Claude 3.7 Sonnet ↑20 0.96
- 2 GPT-5.2 ↑15 0.96
- 3 GLM-5.1 0.96
- 4 Claude Opus 4.6 ↓3 0.95
- 5
Qwen3.5-122B-A10B ↑3 0.95
- 6 GPT-5.4 ↓4 0.95
- 7
Gemini 1.5 Pro Sep 2024 ↑19 0.94
- 8
Qwen2.5-Max ↑20 0.94
- 9
Gemini 2.0 Flash Feb 2025 ↑13 0.94
- 10
DeepSeek-V3 Mar 2025 ↑14 0.94
MaMuRAMu acc.logical_fallacies
102 models we track have a score here.
- 1 Claude Opus 4.6 0.96
- 2 Kimi K2.5 ↑9 0.95
- 3 GLM-5.1 0.94
- 4 Claude Opus 4.5 ↑5 0.93
- 5
Qwen3.5-122B-A10B ↑3 0.93
- 6
Qwen 3.6 35B-A3B ↓1 0.93
- 7
Qwen2.5-Max ↑21 0.92
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.92
- 9 GPT-5.4 ↓7 0.92
- 10
Qwen3.6 27B ↓4 0.91
MaMuRAMu acc.clinical_knowledge
102 models we track have a score here.
- 1
Qwen 3.6 35B-A3B ↑4 0.91
- 2 Claude Opus 4.5 ↑7 0.89
- 3 GPT-5.2 ↑14 0.89
- 4 Claude Opus 4.6 ↓3 0.89
- 5 GPT-5.4 ↓3 0.89
- 6 GLM-5.1 ↓3 0.89
- 7
Qwen3.5-122B-A10B ↑1 0.88
- 8
Qwen3-Next-80B-A3B ↑6 0.86
- 9
Qwen3.5-35B-A3B ↑4 0.85
- 10
Qwen2.5-Max ↑18 0.83
MaMuRAMu acc.conceptual_physics
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.96
- 2 GLM-5.1 ↑1 0.96
- 3
Qwen2.5-Max ↑25 0.95
- 4 Claude 3.7 Sonnet ↑17 0.95
- 5
DeepSeek-V3 Mar 2025 ↑19 0.95
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.95
- 7 Claude Opus 4.6 ↓6 0.95
- 8 GPT-5.4 ↓6 0.95
- 9
Qwen3.6 27B ↓3 0.95
- 10
DeepSeek-V4-Flash ↑15 0.95
MaMuRAMu acc.college_mathematics
102 models we track have a score here.
- 1
DeepSeek-V3 Mar 2025 ↑23 0.98
- 2 gpt-oss-20b ↑33 0.98
- 3 GLM-5.1 0.98
- 4 Kimi K2.5 ↑7 0.96
- 5 Claude Opus 4.6 ↓4 0.96
- 6
Qwen3.5-122B-A10B ↑2 0.96
- 7
Qwen3.5-27B 0.96
- 8
Qwen3.5-35B-A3B ↑5 0.96
- 9 GPT-5.4 ↓7 0.96
- 10
MiniMax-M2.7 ↑8 0.96
MaMuRAMu acc.high_school_biology
102 models we track have a score here.
- 1
DeepSeek-V3 Mar 2025 ↑23 0.93
- 2 GPT-5.2 ↑15 0.93
- 3 Kimi K2.5 ↑8 0.93
- 4
Gemini 1.5 Pro Sep 2024 ↑22 0.91
- 5
Gemini 2.0 Flash-Lite ↑26 0.91
- 6 Claude 3.7 Sonnet ↑15 0.91
- 7 GPT-4o Mar 2025 ↑23 0.91
- 8
Llama 4 Scout ↑41 0.91
- 9
Qwen3-32B ↑3 0.91
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 0.91
MaMuRAMu acc.high_school_history
102 models we track have a score here.
- 1
Qwen1.5-32B ↑56 0.95
- 2
Qwen2.5-72B ↑31 0.95
- 3
Gemini 2.0 Flash Feb 2025 ↑19 0.95
- 4
Gemini 2.0 Flash-Lite ↑27 0.95
- 5 Claude 3.7 Sonnet ↑16 0.95
- 6
DeepSeek-V3 Mar 2025 ↑18 0.95
- 7 GPT-4o Mar 2025 ↑23 0.95
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.95
- 9
Qwen3-Next-80B-A3B ↑5 0.95
- 10 GPT-5.2 ↑7 0.95
MaMuRAMu acc.high_school_physics
102 models we track have a score here.
- 1 Claude Opus 4.6 0.95
- 2
Qwen3-4B-Thinking-2507 ↑14 0.93
- 3 Claude Opus 4.5 ↑6 0.93
- 4
Qwen3.5-9B ↑19 0.93
- 5 GPT-5.4 ↓3 0.93
- 6 GLM-5.1 ↓3 0.93
- 7
Qwen 3.6 35B-A3B ↓2 0.93
- 8
Qwen3.6 27B ↓2 0.93
- 9
Qwen3-32B ↑3 0.91
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 0.91
MaMuRAMu acc.high_school_chemistry
102 models we track have a score here.
- 1 Claude Opus 4.6 0.97
- 2
Qwen3.5-122B-A10B ↑6 0.97
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 0.95
- 4
Qwen3-30B-A3B-Thinking Jul 2025 ↑6 0.95
- 5 Kimi K2.5 ↑6 0.95
- 6
Qwen3.5-27B ↑1 0.95
- 7 GLM-5.1 ↓4 0.95
- 8
Qwen 3.6 35B-A3B ↓3 0.95
- 9
Qwen3-4B-Thinking-2507 ↑7 0.94
- 10 Claude Opus 4.5 ↓1 0.94
MaMuRAMu acc.high_school_geography
102 models we track have a score here.
- 1 GLM-5.1 ↑2 0.99
- 2 Claude Opus 4.6 ↓1 0.99
- 3 GPT-5.4 ↓1 0.98
- 4 GPT-5.2 ↑13 0.98
- 5
Qwen3.5-122B-A10B ↑3 0.98
- 6 Claude Opus 4.5 ↑3 0.97
- 7
Qwen3.5-27B 0.97
- 8
Gemini 2.0 Flash-Lite ↑23 0.96
- 9 Kimi K2.5 ↑2 0.96
- 10
Qwen 3.6 35B-A3B ↓5 0.96
MaMuRAMu acc.professional_medicine
102 models we track have a score here.
- 1 GPT-4o Mar 2025 ↑29 0.98
- 2 GPT-5.2 ↑15 0.98
- 3 Claude Opus 4.6 ↓2 0.98
- 4 Claude 3.7 Sonnet ↑17 0.97
- 5
Qwen3.5-122B-A10B ↑3 0.97
- 6 GPT-5.4 ↓4 0.97
- 7
MiniMax-M2.7 ↑11 0.97
- 8
Gemini 2.0 Flash Feb 2025 ↑14 0.95
- 9
DeepSeek-V3 Mar 2025 ↑15 0.95
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 0.95
MaMuRAMu acc.electrical_engineering
102 models we track have a score here.
- 1
DeepSeek-V3 Mar 2025 ↑23 0.91
- 2
Llama 4 Scout ↑47 0.91
- 3 Claude Opus 4.5 ↑6 0.91
- 4 GPT-5.2 ↑13 0.91
- 5 Claude Opus 4.6 ↓4 0.91
- 6
Gemma 4 31B IT ↑9 0.91
- 7
Qwen2-72B ↑32 0.89
- 8 Claude 3.5 Haiku ↑38 0.89
- 9
Qwen2.5-Max ↑19 0.89
- 10
Gemini 2.0 Flash-Lite ↑21 0.89
MaMuRAMu acc.elementary_mathematics
102 models we track have a score here.
- 1
Qwen2.5-72B ↑32 1.00
- 2
DeepSeek-R1-Distill-Qwen-32B ↑32 1.00
- 3
Qwen2.5-Max ↑25 1.00
- 4
Gemini 2.0 Flash-Lite ↑27 1.00
- 5 Claude 3.7 Sonnet ↑16 1.00
- 6
DeepSeek-V3 Mar 2025 ↑18 1.00
- 7
Llama 4 Maverick ↑29 1.00
- 8
Qwen3-32B ↑4 1.00
- 9
Qwen3-4B ↑18 1.00
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 1.00
MaMuRAMu acc.high_school_psychology
102 models we track have a score here.
- 1
DeepSeek-V3 Mar 2025 ↑23 0.95
- 2
Llama 4 Maverick ↑34 0.95
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 0.95
- 4 gpt-oss-20b ↑31 0.95
- 5 Claude Opus 4.5 ↑4 0.95
- 6
Gemma 4 26B A4B ↑31 0.95
- 7
Qwen 3.6 35B-A3B ↓2 0.95
- 8
phi-3-medium 14B ↑43 0.93
- 9
Phi-3-medium-128k-instruct ↑43 0.93
- 10
Phi-3.5-MoE ↑44 0.93
MaMuRAMu acc.high_school_statistics
102 models we track have a score here.
- 1
Gemini 2.0 Flash-Lite ↑30 0.93
- 2 Claude Opus 4.5 ↑7 0.93
- 3
Phi-3-medium-128k-instruct ↑49 0.91
- 4
Qwen2-72B ↑35 0.91
- 5
Qwen2.5-72B ↑28 0.91
- 6
Gemini 1.5 Pro Sep 2024 ↑20 0.91
- 7
Qwen-Turbo ↑38 0.91
- 8
Llama 3.3 70B Instruct ↑36 0.91
- 9
Gemini 2.0 Flash Feb 2025 ↑13 0.91
- 10 Claude 3.7 Sonnet ↑11 0.91
MaMuRAMu acc.high_school_mathematics
102 models we track have a score here.
- 1
Qwen2.5-72B ↑32 0.95
- 2 GPT-4o Mar 2025 ↑28 0.95
- 3
Gemma 4 31B IT ↑12 0.95
- 4
Qwen2.5-Math-72B-Instruct ↑49 0.93
- 5
Gemini 2.0 Flash Feb 2025 ↑17 0.93
- 6
Gemini 2.0 Flash-Lite ↑25 0.93
- 7
DeepSeek-V3 Mar 2025 ↑17 0.93
- 8
Llama 4 Maverick ↑28 0.93
- 9
Qwen3-32B ↑3 0.93
- 10
Qwen3-4B ↑17 0.93
MaMuRAMu acc.professional_accounting
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.98
- 2 GPT-5.2 ↑15 0.98
- 3 Claude Opus 4.6 ↓2 0.98
- 4 GPT-5.4 ↓2 0.98
- 5 GLM-5.1 ↓2 0.97
- 6 Claude 3.7 Sonnet ↑15 0.95
- 7
Qwen3.5-35B-A3B ↑6 0.95
- 8
MiniMax-M2.7 ↑10 0.95
- 9
Gemini 2.0 Flash Feb 2025 ↑13 0.94
- 10
DeepSeek-V3 Mar 2025 ↑14 0.94
MaMuRAMu acc.professional_psychology
102 models we track have a score here.
- 1
Qwen3-Next-80B-A3B ↑13 1.00
- 2 Claude Opus 4.5 ↑7 1.00
- 3
Qwen2.5-Max ↑25 0.98
- 4 Claude 3.7 Sonnet ↑17 0.98
- 5
Qwen3-30B-A3B-Thinking Jul 2025 ↑5 0.98
- 6
Qwen3.5-122B-A10B ↑2 0.98
- 7 GPT-5.4 ↓5 0.98
- 8 GLM-5.1 ↓5 0.98
- 9
Llama 3-70B ↑34 0.96
- 10
Qwen2.5-72B ↑23 0.96
MaMuRAMu acc.college_computer_science
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.98
- 2 Claude Opus 4.6 ↓1 0.98
- 3 GPT-5.4 ↓1 0.98
- 4 Kimi K2.5 ↑7 0.96
- 5
Qwen3.5-27B ↑2 0.96
- 6 GLM-5.1 ↓3 0.96
- 7
Qwen3.6 27B ↓1 0.96
- 8
Gemini 2.0 Flash Feb 2025 ↑14 0.93
- 9 Claude 3.7 Sonnet ↑12 0.93
- 10
Qwen3-32B ↑2 0.93
MaMuRAMu acc.high_school_world_history
102 models we track have a score here.
- 1 GPT-5.4 ↑1 1.00
- 2 Claude Opus 4.5 ↑7 0.99
- 3 GPT-5.2 ↑14 0.99
- 4 Claude Opus 4.6 ↓3 0.99
- 5
Qwen3.5-122B-A10B ↑3 0.99
- 6
Qwen3.5-27B ↑1 0.99
- 7 Claude 3.7 Sonnet ↑14 0.97
- 8 GPT-4o Mar 2025 ↑22 0.97
- 9 Kimi K2.5 ↑2 0.97
- 10
Gemma 4 31B IT ↑5 0.97
MaMuRAMu acc.high_school_macroeconomics
102 models we track have a score here.
- 1 GPT-5.4 ↑1 0.92
- 2 Claude Opus 4.6 ↓1 0.90
- 3
Qwen3.5-27B ↑4 0.90
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.89
- 5 Claude Opus 4.5 ↑4 0.89
- 6
Qwen3.5-122B-A10B ↑2 0.89
- 7
Qwen3.5-35B-A3B ↑6 0.89
- 8 GLM-5.1 ↓5 0.89
- 9
Qwen 3.6 35B-A3B ↓4 0.89
- 10
Qwen2-7B ↑55 0.87
MaMuRAMu acc.high_school_microeconomics
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.87
- 2 Claude Opus 4.6 ↓1 0.87
- 3
Qwen3.5-27B ↑4 0.86
- 4 GPT-5.4 ↓2 0.84
- 5
Gemma 4 31B IT ↑10 0.84
- 6 GLM-5.1 ↓3 0.84
- 7
Qwen2.5-Max ↑21 0.83
- 8
Llama 4 Maverick ↑28 0.83
- 9
Qwen3-235B-A22B-Thinking Jul 2025 ↓5 0.83
- 10 Claude 3.7 Sonnet ↑11 0.82
MaMuRAMu acc.high_school_computer_science
102 models we track have a score here.
- 1 GLM-5.1 ↑2 0.86
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 0.84
- 3
Qwen3-Next-80B-A3B ↑11 0.84
- 4 Kimi K2.5 ↑7 0.84
- 5 Claude Opus 4.6 ↓4 0.84
- 6
Qwen3.5-35B-A3B ↑7 0.84
- 7 GPT-5.4 ↓5 0.84
- 8
Qwen3.6 27B ↓2 0.84
- 9
Qwen3-30B-A3B-Thinking Jul 2025 ↑1 0.81
- 10
Qwen3.5-122B-A10B ↓2 0.81
MaMuRAMu acc.high_school_european_history
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 0.97
- 2 Claude Opus 4.6 ↓1 0.97
- 3 GPT-5.4 ↓1 0.95
- 4 Claude 3.7 Sonnet ↑17 0.92
- 5 GPT-5.2 ↑12 0.92
- 6 Kimi K2.5 ↑5 0.92
- 7 GLM-5.1 ↓4 0.92
- 8 GPT-4o Mar 2025 ↑22 0.91
- 9
Qwen3.5-122B-A10B ↓1 0.91
- 10
DeepSeek-V4-Flash ↑15 0.91
MaMuRAMu acc.high_school_government_and_politics
102 models we track have a score here.
- 1 GPT-4o Mar 2025 ↑29 0.97
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 0.97
- 3 Claude Opus 4.5 ↑6 0.97
- 4 GPT-5.2 ↑13 0.97
- 5 Claude Opus 4.6 ↓4 0.97
- 6 GPT-5.4 ↓4 0.97
- 7 GLM-5.1 ↓4 0.97
- 8
Gemini 1.5 Pro Sep 2024 ↑18 0.96
- 9 Claude 3.7 Sonnet ↑12 0.96
- 10
Qwen3.5-122B-A10B ↓2 0.96
SimpleAr acc
37 models we track have a score here.
- 1
Qwen3-235B-A22B-Thinking Jul 2025 ↑3 1.00
- 2
Qwen3-30B-A3B-Thinking Jul 2025 ↑8 1.00
- 3
Qwen3-4B-Thinking-2507 ↑13 1.00
- 4 Claude Opus 4.5 ↑5 1.00
- 5 GPT-5.2 ↑12 1.00
- 6 Claude Opus 4.6 ↓5 1.00
- 7
Qwen3.5-122B-A10B ↑1 1.00
- 8
Qwen3.5-27B ↓1 1.00
- 9
Gemma 4 26B A4B ↑28 1.00
- 10
Gemma 4 31B IT ↑5 1.00
SimpleAr em
64 models we track have a score here.
- 1
Qwen2.5-Max ↑27 1.00
- 2
Gemini 2.0 Flash Feb 2025 ↑20 1.00
- 3 Claude 3.7 Sonnet ↑18 1.00
- 4
DeepSeek-V3 Mar 2025 ↑20 1.00
- 5 Claude 3.5 Haiku ↑41 1.00
- 6
Gemini 2.0 Flash-Lite ↑25 1.00
- 7 GPT-4o Mar 2025 ↑23 1.00
- 8
Qwen2.5-72B ↑25 1.00
- 9
Qwen2.5-Math-72B-Instruct ↑44 1.00
- 10
Gemma 3 27B ↑31 1.00
ruEthics good.law
101 models we track have a score here.
- 1
Qwen3-4B-Thinking-2507 ↑15 0.56
- 2
Qwen2-72B ↑37 0.54
- 3
Qwen3-30B-A3B-Thinking Jul 2025 ↑7 0.53
- 4
DeepSeek-R1-Distill-Qwen-32B ↑30 0.52
- 5
MiniMax-M2.7 ↑13 0.52
- 6
Qwen3-32B ↑6 0.51
- 7
Qwen2.5-Math-72B-Instruct ↑46 0.50
- 8 GPT-4o Mar 2025 ↑22 0.49
- 9
phi-3-small 7.4B ↑53 0.49
- 10
Qwen3-Next-80B-A3B ↑4 0.49
ruEthics good.moral
101 models we track have a score here.
- 1
Qwen3-4B-Thinking-2507 ↑15 0.61
- 2
Qwen2-72B ↑37 0.60
- 3
Qwen3-32B ↑9 0.58
- 4
DeepSeek-R1-Distill-Qwen-32B ↑30 0.57
- 5
MiniMax-M2.7 ↑13 0.56
- 6
Qwen3-30B-A3B-Thinking Jul 2025 ↑4 0.55
- 7 GPT-4o Mar 2025 ↑23 0.54
- 8
Qwen3-Next-80B-A3B ↑6 0.52
- 9 GPT-5.2 ↑8 0.51
- 10
Gemini 1.5 Pro Sep 2024 ↑16 0.51
ruEthics correct.law
101 models we track have a score here.
- 1
Qwen3-30B-A3B-Thinking Jul 2025 ↑9 0.63
- 2
Qwen2-72B ↑37 0.58
- 3 GPT-4o Mar 2025 ↑27 0.57
- 4
Qwen3-32B ↑8 0.57
- 5 Claude Opus 4.6 ↓4 0.55
- 6
DeepSeek-R1-Distill-Qwen-32B ↑28 0.55
- 7
Qwen3-235B-A22B-Thinking Jul 2025 ↓3 0.54
- 8
MiniMax-M2.7 ↑10 0.52
- 9
Gemini 1.5 Pro Sep 2024 ↑17 0.50
- 10 Claude Opus 4.5 ↓1 0.50
ruEthics ethical.law
101 models we track have a score here.
- 1 GPT-5.2 ↑16 0.73
- 2
Qwen2-72B ↑37 0.71
- 3 Claude Opus 4.6 ↓2 0.70
- 4 Claude Opus 4.5 ↑5 0.68
- 5
Qwen3.6 27B ↑1 0.68
- 6 GPT-5.4 ↓4 0.66
- 7
MiniMax-M2.7 ↑11 0.65
- 8
Gemini 1.5 Pro Sep 2024 ↑18 0.64
- 9
Qwen3-30B-A3B-Thinking Jul 2025 ↑1 0.64
- 10
Qwen3.5-27B ↓3 0.64
ruEthics good.virtue
101 models we track have a score here.
- 1
Qwen3-4B-Thinking-2507 ↑15 0.56
- 2
Qwen2-72B ↑37 0.55
- 3
Qwen3-32B ↑9 0.55
- 4
DeepSeek-R1-Distill-Qwen-32B ↑30 0.54
- 5
MiniMax-M2.7 ↑13 0.52
- 6
Qwen3-30B-A3B-Thinking Jul 2025 ↑4 0.52
- 7 GPT-4o Mar 2025 ↑23 0.50
- 8 GPT-5.2 ↑9 0.49
- 9
Gemini 1.5 Pro Sep 2024 ↑17 0.49
- 10
Qwen2.5-Math-72B-Instruct ↑43 0.48
ruEthics good.justice
101 models we track have a score here.
- 1
Qwen3-4B-Thinking-2507 ↑15 0.51
- 2
Qwen2-72B ↑37 0.50
- 3
Qwen3-30B-A3B-Thinking Jul 2025 ↑7 0.49
- 4
Qwen3-32B ↑8 0.48
- 5
DeepSeek-R1-Distill-Qwen-32B ↑29 0.48
- 6
MiniMax-M2.7 ↑12 0.47
- 7
Qwen2.5-Math-72B-Instruct ↑46 0.46
- 8
Qwen3-Next-80B-A3B ↑6 0.45
- 9 GPT-4o Mar 2025 ↑21 0.44
- 10 GPT-5.2 ↑7 0.43
ruEthics correct.moral
101 models we track have a score here.
- 1
Qwen3-30B-A3B-Thinking Jul 2025 ↑9 0.64
- 2
Qwen2-72B ↑37 0.62
- 3 GPT-4o Mar 2025 ↑27 0.61
- 4
Qwen3-32B ↑8 0.60
- 5
DeepSeek-R1-Distill-Qwen-32B ↑29 0.58
- 6 Claude Opus 4.6 ↓5 0.57
- 7
MiniMax-M2.7 ↑11 0.56
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.55
- 9
Gemini 1.5 Pro Sep 2024 ↑17 0.55
- 10
Qwen2.5-Max ↑18 0.54
ruEthics ethical.moral
101 models we track have a score here.
- 1 GPT-5.2 ↑16 0.77
- 2
Qwen2-72B ↑37 0.77
- 3 Claude Opus 4.5 ↑6 0.76
- 4 Claude Opus 4.6 ↓3 0.74
- 5 GPT-5.4 ↓3 0.72
- 6
Qwen3.6 27B 0.71
- 7
MiniMax-M2.7 ↑11 0.70
- 8
Gemini 1.5 Pro Sep 2024 ↑18 0.69
- 9
Qwen3-30B-A3B-Thinking Jul 2025 ↑1 0.69
- 10
Qwen3.5-27B ↓3 0.67
ruEthics correct.virtue
101 models we track have a score here.
- 1
Qwen3-30B-A3B-Thinking Jul 2025 ↑9 0.59
- 2 GPT-4o Mar 2025 ↑28 0.57
- 3
Qwen3-32B ↑9 0.56
- 4
Qwen2-72B ↑35 0.55
- 5
DeepSeek-R1-Distill-Qwen-32B ↑29 0.54
- 6 Claude Opus 4.6 ↓5 0.53
- 7
Gemini 1.5 Pro Sep 2024 ↑19 0.52
- 8
MiniMax-M2.7 ↑10 0.50
- 9
Qwen2.5-Max ↑19 0.50
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 0.49
ruEthics ethical.virtue
101 models we track have a score here.
- 1 GPT-5.2 ↑16 0.72
- 2 Claude Opus 4.5 ↑7 0.71
- 3
Qwen2-72B ↑36 0.70
- 4 Claude Opus 4.6 ↓3 0.70
- 5 GPT-5.4 ↓3 0.68
- 6
Gemini 1.5 Pro Sep 2024 ↑20 0.67
- 7
MiniMax-M2.7 ↑11 0.66
- 8
Qwen3-30B-A3B-Thinking Jul 2025 ↑2 0.66
- 9
Qwen3.6 27B ↓3 0.65
- 10
Qwen3-32B ↑2 0.64
ruEthics correct.justice
101 models we track have a score here.
- 1
Qwen3-30B-A3B-Thinking Jul 2025 ↑9 0.55
- 2 GPT-4o Mar 2025 ↑28 0.52
- 3
Qwen3-32B ↑9 0.51
- 4
Qwen2-72B ↑35 0.50
- 5
DeepSeek-R1-Distill-Qwen-32B ↑29 0.48
- 6
MiniMax-M2.7 ↑12 0.48
- 7 Claude Opus 4.6 ↓6 0.47
- 8
Qwen2.5-Max ↑20 0.46
- 9
Gemini 1.5 Pro Sep 2024 ↑17 0.46
- 10
Qwen3-235B-A22B-Thinking Jul 2025 ↓6 0.46
ruEthics ethical.justice
101 models we track have a score here.
- 1
Qwen2-72B ↑38 0.66
- 2 GPT-5.2 ↑15 0.64
- 3 Claude Opus 4.5 ↑6 0.63
- 4 Claude Opus 4.6 ↓3 0.63
- 5
MiniMax-M2.7 ↑13 0.62
- 6
Qwen3-30B-A3B-Thinking Jul 2025 ↑4 0.61
- 7 GPT-5.4 ↓5 0.61
- 8
Qwen3.6 27B ↓2 0.60
- 9
Gemini 1.5 Pro Sep 2024 ↑17 0.59
- 10
Gemma 4 31B IT ↑5 0.58
ruEthics good.utilitarianism
101 models we track have a score here.
- 1
Qwen3-4B-Thinking-2507 ↑15 0.49
- 2
Qwen3-32B ↑10 0.48
- 3
Qwen2-72B ↑36 0.47
- 4
MiniMax-M2.7 ↑14 0.46
- 5
DeepSeek-R1-Distill-Qwen-32B ↑29 0.45
- 6
Qwen3-30B-A3B-Thinking Jul 2025 ↑4 0.44
- 7 GPT-5.2 ↑10 0.44
- 8
Gemini 1.5 Pro Sep 2024 ↑18 0.43
- 9 GPT-4o Mar 2025 ↑21 0.42
- 10
Qwen3-Next-80B-A3B ↑4 0.42
ruEthics correct.utilitarianism
101 models we track have a score here.
- 1
Qwen3-30B-A3B-Thinking Jul 2025 ↑9 0.52
- 2 GPT-4o Mar 2025 ↑28 0.49
- 3
Qwen3-32B ↑9 0.49
- 4
Qwen2-72B ↑35 0.48
- 5 Claude Opus 4.6 ↓4 0.46
- 6
MiniMax-M2.7 ↑12 0.45
- 7
Qwen3-4B-Thinking-2507 ↑9 0.44
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.43
- 9
Qwen2.5-Max ↑19 0.43
- 10
DeepSeek-R1-Distill-Qwen-32B ↑24 0.43
ruEthics ethical.utilitarianism
101 models we track have a score here.
- 1 GPT-5.2 ↑16 0.61
- 2
Qwen2-72B ↑37 0.60
- 3 Claude Opus 4.5 ↑6 0.59
- 4 Claude Opus 4.6 ↓3 0.58
- 5 GPT-5.4 ↓3 0.57
- 6
Gemini 1.5 Pro Sep 2024 ↑20 0.56
- 7
Qwen3.6 27B ↓1 0.55
- 8
Gemma 4 31B IT ↑7 0.55
- 9
Qwen3-30B-A3B-Thinking Jul 2025 ↑1 0.54
- 10
Qwen3.5-27B ↓3 0.54
ruMultiAr acc
37 models we track have a score here.
- 1
Qwen3-30B-A3B-Thinking Jul 2025 ↑9 1.00
- 2
Qwen3-Next-80B-A3B ↑12 1.00
- 3
Qwen3.5-27B ↑4 1.00
- 4 GPT-5.4 ↓2 1.00
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 1.00
- 6 GLM-5.1 ↓3 1.00
- 7 Claude Opus 4.6 ↓6 1.00
- 8
Qwen3.5-122B-A10B 1.00
- 9
Qwen3-4B-Thinking-2507 ↑7 1.00
- 10
Qwen3-14B ↑9 0.99
ruMultiAr em
65 models we track have a score here.
- 1 gpt-oss-20b ↑34 1.00
- 2
DeepSeek-R1-Distill-Qwen-32B ↑32 0.91
- 3
Qwen2.5-Max ↑25 0.59
- 4
DeepSeek-V3 Mar 2025 ↑20 0.51
- 5
Gemini 1.5 Pro Sep 2024 ↑21 0.51
- 6
Llama 4 Maverick ↑30 0.49
- 7
Gemini 2.0 Flash Feb 2025 ↑15 0.48
- 8 Claude 3.7 Sonnet ↑13 0.48
- 9
Gemini 2.0 Flash-Lite ↑22 0.45
- 10
Qwen2.5-72B ↑23 0.43
ruCodeEval pass@1
102 models we track have a score here.
- 1
Qwen3.6 27B ↑5 0.77
- 2
Gemini 2.0 Flash Feb 2025 ↑20 0.77
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 0.72
- 4
Qwen3-4B-Thinking-2507 ↑12 0.72
- 5
Gemma 4 31B IT ↑10 0.72
- 6 GPT-4o mini ↑34 0.70
- 7
Qwen 3.6 35B-A3B ↓2 0.70
- 8
Gemini 2.0 Flash-Lite ↑23 0.67
- 9
Qwen3-4B ↑18 0.65
- 10
Gemini 1.5 Flash Sep 2024 ↑22 0.63
ruCodeEval pass@5
102 models we track have a score here.
- 1
Qwen3.6 27B ↑5 0.88
- 2
Qwen 3.6 35B-A3B ↑3 0.87
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 0.86
- 4
Qwen3-4B-Thinking-2507 ↑12 0.84
- 5
Qwen3.5-27B ↑2 0.82
- 6
Qwen3-32B ↑6 0.80
- 7
Gemini 2.0 Flash Feb 2025 ↑15 0.79
- 8
Qwen3.5-122B-A10B 0.78
- 9
Qwen3-Next-80B-A3B ↑5 0.77
- 10 GPT-4o mini ↑30 0.75
ruCodeEval pass@10
102 models we track have a score here.
- 1
Qwen3.5-27B ↑6 0.89
- 2
Qwen 3.6 35B-A3B ↑3 0.89
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 0.88
- 4
Qwen3.6 27B ↑2 0.88
- 5
Qwen3-4B-Thinking-2507 ↑11 0.86
- 6
Qwen3.5-122B-A10B ↑2 0.86
- 7
Qwen3-32B ↑5 0.84
- 8
Qwen3-Next-80B-A3B ↑6 0.83
- 9
DeepSeek-V3 Mar 2025 ↑15 0.82
- 10
Gemini 2.0 Flash Feb 2025 ↑12 0.79
MathLogicQA acc
102 models we track have a score here.
- 1 Claude Opus 4.6 1.00
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 1.00
- 3
Qwen3-30B-A3B-Thinking Jul 2025 ↑7 1.00
- 4 GLM-5.1 ↓1 1.00
- 5
Qwen3.6 27B ↑1 1.00
- 6 Kimi K2.5 ↑5 1.00
- 7
Qwen3.5-27B 0.99
- 8
Qwen3-4B ↑19 0.99
- 9
Qwen3.5-122B-A10B ↓1 0.99
- 10
MiniMax-M2.7 ↑8 0.99
ruHumanEval pass@1
101 models we track have a score here.
- 1
Qwen3.6 27B ↑5 0.80
- 2
Qwen3-4B-Thinking-2507 ↑14 0.77
- 3
Qwen 3.6 35B-A3B ↑2 0.77
- 4
Qwen3-4B ↑23 0.76
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 0.76
- 6
Gemini 2.0 Flash Feb 2025 ↑16 0.74
- 7 GPT-4o mini ↑33 0.70
- 8
Qwen3-14B ↑11 0.64
- 9
Qwen3-32B ↑3 0.63
- 10
Qwen3-30B-A3B-Thinking Jul 2025 0.62
ruHumanEval pass@5
101 models we track have a score here.
- 1
Qwen3.6 27B ↑5 0.87
- 2
Qwen 3.6 35B-A3B ↑3 0.87
- 3
Qwen3-4B-Thinking-2507 ↑13 0.85
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.84
- 5
Qwen3-4B ↑22 0.84
- 6
Qwen3-Next-80B-A3B ↑8 0.84
- 7
Qwen3.5-27B 0.83
- 8
Qwen3-32B ↑4 0.82
- 9
Qwen3.5-122B-A10B ↓1 0.81
- 10
DeepSeek-V3 Mar 2025 ↑14 0.79
ruHumanEval pass@10
101 models we track have a score here.
- 1
Qwen3.5-122B-A10B ↑7 0.88
- 2
Qwen3.5-27B ↑5 0.88
- 3
Qwen3.6 27B ↑3 0.88
- 4
Qwen 3.6 35B-A3B ↑1 0.88
- 5
Qwen3-Next-80B-A3B ↑9 0.87
- 6
Qwen3-4B ↑21 0.86
- 7
Qwen3-4B-Thinking-2507 ↑9 0.86
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.85
- 9
Qwen3-32B ↑3 0.85
- 10
DeepSeek-V3 Mar 2025 ↑14 0.82
ruWorldTree acc
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 1.00
- 2 GPT-5.2 ↑15 1.00
- 3 Claude Opus 4.6 ↓2 1.00
- 4
Qwen3.5-27B ↑3 1.00
- 5 GLM-5.1 ↓2 1.00
- 6
Gemini 2.0 Flash Feb 2025 ↑16 0.99
- 7 Claude 3.7 Sonnet ↑14 0.99
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.99
- 9
Qwen3.5-122B-A10B ↓1 0.99
- 10
Qwen2.5-72B ↑23 0.99
ruWorldTree f1_macro
102 models we track have a score here.
- 1 Claude Opus 4.5 ↑8 1.00
- 2 GPT-5.2 ↑15 1.00
- 3 Claude Opus 4.6 ↓2 1.00
- 4
Qwen3.5-27B ↑3 1.00
- 5 GLM-5.1 ↓2 1.00
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 0.99
- 7
Gemini 2.0 Flash Feb 2025 ↑15 0.99
- 8 Claude 3.7 Sonnet ↑13 0.99
- 9 GPT-5.4 ↓7 0.99
- 10
Qwen2.5-72B ↑23 0.99
ruHateSpeech acc
101 models we track have a score here.
- 1 GPT-5.4 ↑1 0.95
- 2 GLM-5.1 ↑1 0.94
- 3
Qwen3.6 27B ↑3 0.93
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.93
- 5
Qwen3.5-122B-A10B ↑3 0.93
- 6
Qwen3.5-27B ↑1 0.93
- 7
Qwen3-30B-A3B-Thinking Jul 2025 ↑3 0.92
- 8
Qwen 3.6 35B-A3B ↓3 0.92
- 9 Claude Opus 4.6 ↓8 0.91
- 10 Kimi K2.5 ↑1 0.91
ruHateSpeech acc.лгбт
101 models we track have a score here.
- 1
Qwen3-14B ↑18 1.00
- 2
Qwen3.5-122B-A10B ↑6 1.00
- 3
Qwen3.5-27B ↑4 1.00
- 4
MiniMax-M2.7 ↑14 1.00
- 5
phi-3-medium 14B ↑46 0.94
- 6
phi-3-small 7.4B ↑56 0.94
- 7
Qwen2.5-Math-72B-Instruct ↑46 0.94
- 8
DeepSeek-R1-Distill-Qwen-32B ↑26 0.94
- 9 Claude 3.7 Sonnet ↑12 0.94
- 10 GPT-4o Mar 2025 ↑20 0.94
ruHateSpeech acc.другое
101 models we track have a score here.
- 1
Qwen3-14B ↑18 0.95
- 2
Qwen3.5-122B-A10B ↑6 0.95
- 3 GPT-5.4 ↓1 0.95
- 4
Qwen3.6 27B ↑2 0.95
- 5
Qwen2.5-72B ↑28 0.93
- 6 GPT-4o Mar 2025 ↑24 0.93
- 7
Qwen3-30B-A3B-Thinking Jul 2025 ↑3 0.93
- 8
Qwen3-4B-Thinking-2507 ↑8 0.93
- 9 GPT-5.2 ↑8 0.93
- 10
Qwen3.5-27B ↓3 0.93
ruHateSpeech acc.женщины
101 models we track have a score here.
- 1 GPT-5.4 ↑1 1.00
- 2 Kimi K2.5 ↑9 0.98
- 3 GLM-5.1 0.98
- 4
Qwen3-235B-A22B-Thinking Jul 2025 0.96
- 5 GPT-5.2 ↑12 0.96
- 6
Qwen3.5-27B ↑1 0.96
- 7
Qwen3.6 27B ↓1 0.96
- 8 Claude Opus 4.6 ↓7 0.95
- 9
Qwen3.5-122B-A10B ↓1 0.95
- 10
Qwen3-30B-A3B-Thinking Jul 2025 0.94
ruHateSpeech acc.мужчины
101 models we track have a score here.
- 1
DeepSeek-V4-Flash ↑24 0.86
- 2
DeepSeek-V4-Pro ↑18 0.86
- 3
Qwen2.5-72B ↑30 0.83
- 4
Qwen3-32B ↑8 0.83
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 0.83
- 6 GPT-5.4 ↓4 0.83
- 7
Gemini 2.0 Flash-Lite ↑24 0.80
- 8
Qwen3-8B ↑21 0.80
- 9
Qwen3-30B-A3B-Thinking Jul 2025 ↑1 0.80
- 10 GLM-5.1 ↓7 0.80
ruHateSpeech acc.мигранты
101 models we track have a score here.
- 1
Qwen1.5-7B-Chat ↑85 1.00
- 2
DeepSeek-R1-Distill-Qwen-32B ↑32 1.00
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 1.00
- 4
Qwen3-Next-80B-A3B ↑10 1.00
- 5 Claude Opus 4.5 ↑4 1.00
- 6 GPT-5.2 ↑11 1.00
- 7 Kimi K2.5 ↑4 1.00
- 8 Claude Opus 4.6 ↓7 1.00
- 9
Qwen3.5-122B-A10B ↓1 1.00
- 10
Qwen3.5-27B ↓3 1.00
ruHateSpeech acc.национальность
101 models we track have a score here.
- 1
Qwen3-235B-A22B-Thinking Jul 2025 ↑3 0.97
- 2
Qwen3.5-122B-A10B ↑6 0.97
- 3
Qwen3.5-27B ↑4 0.97
- 4 GLM-5.1 ↓1 0.97
- 5
Qwen 3.6 35B-A3B 0.97
- 6
Qwen3.6 27B 0.97
- 7
Qwen3-14B ↑12 0.95
- 8
Qwen3-32B ↑4 0.95
- 9
Qwen3-30B-A3B-Thinking Jul 2025 ↑1 0.95
- 10 GPT-5.2 ↑7 0.95
ruOpenBookQA acc
102 models we track have a score here.
- 1 Claude Opus 4.6 0.98
- 2 Claude Opus 4.5 ↑7 0.97
- 3 GPT-5.4 ↓1 0.97
- 4
Qwen3.5-27B ↑3 0.97
- 5 GLM-5.1 ↓2 0.96
- 6
Qwen3.6 27B 0.96
- 7
Qwen3-32B ↑5 0.96
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.96
- 9 GPT-5.2 ↑8 0.96
- 10 Kimi K2.5 ↑1 0.96
ruOpenBookQA f1_macro
102 models we track have a score here.
- 1 Claude Opus 4.6 0.98
- 2 Claude Opus 4.5 ↑7 0.97
- 3 GPT-5.4 ↓1 0.97
- 4
Qwen3.5-27B ↑3 0.97
- 5 GLM-5.1 ↓2 0.97
- 6
Qwen3.6 27B 0.97
- 7 Kimi K2.5 ↑4 0.96
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 0.96
- 9
Qwen3-32B ↑3 0.96
- 10 Claude 3.7 Sonnet ↑11 0.95
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: Official MERA leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| Claude Opus 4.6 | generative-scoring, multi-turn, chat template, system prompt | 0.86 | |||
| GPT-5.4 | generative-scoring, multi-turn, chat template, system prompt | 0.82 | |||
| GLM-5.1 | generative-scoring, multi-turn, chat template, system prompt | 0.80 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.80 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.79 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.79 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.78 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.77 | ||||
| Claude Opus 4.5 | generative-scoring, multi-turn, chat template, system prompt | 0.76 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.75 | ||||
| Kimi K2.5 | generative-scoring, multi-turn, chat template, system prompt | 0.75 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.73 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.73 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.71 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.71 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.71 | ||||
| GPT-5.2 | generative-scoring, multi-turn, chat template, system prompt | 0.71 | |||
| generative-scoring, multi-turn | 0.69 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.69 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.69 | ||||
| Claude 3.7 Sonnet | generative-scoring, multi-turn, chat template, system prompt | 0.68 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.68 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.68 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.68 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.68 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.68 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.67 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.67 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.66 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.65 | ||||
| GPT-4o Mar 2025 | generative-scoring, multi-turn, chat template, system prompt | 0.64 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.64 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.63 | ||||
| log-likelihood, system prompt | 0.62 | ||||
| generative-scoring, chat template, system prompt | 0.62 | ||||
| gpt-oss-20b | generative-scoring, multi-turn, chat template, system prompt | 0.62 | |||
| generative-scoring, system prompt | 0.61 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.61 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.60 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.60 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.60 | ||||
| generative-scoring, chat template, system prompt | 0.59 | ||||
| log-likelihood, chat template, system prompt | 0.59 | ||||
| log-likelihood, system prompt | 0.59 | ||||
| generative-scoring, system prompt | 0.58 | ||||
| log-likelihood | 0.57 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.57 | ||||
| GPT-4o mini | generative-scoring, multi-turn, chat template, system prompt | 0.57 | |||
| generative-scoring | 0.57 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.57 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.57 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.57 | ||||
| log-likelihood | 0.57 | ||||
| generative-scoring | 0.56 | ||||
| log-likelihood | 0.56 | ||||
| generative-scoring | 0.56 | ||||
| log-likelihood | 0.56 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.56 | ||||
| generative-scoring | 0.55 | ||||
| log-likelihood, chat template | 0.55 | ||||
| generative-scoring, chat template | 0.55 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.55 | ||||
| Claude 3.5 Haiku | generative-scoring, multi-turn, chat template, system prompt | 0.55 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.54 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.53 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.53 | ||||
| log-likelihood | 0.52 | ||||
| log-likelihood | 0.51 | ||||
| generative-scoring | 0.51 | ||||
| log-likelihood | 0.51 | ||||
| log-likelihood, system prompt | 0.51 | ||||
| generative-scoring | 0.51 | ||||
| log-likelihood | 0.50 | ||||
| generative-scoring, system prompt | 0.49 | ||||
| log-likelihood | 0.49 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.49 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.49 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.49 | ||||
| log-likelihood | 0.48 | ||||
| log-likelihood, chat template, system prompt | 0.48 | ||||
| generative-scoring | 0.48 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.48 | ||||
| generative-scoring | 0.48 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.48 | ||||
| generative-scoring, chat template, system prompt | 0.48 | ||||
| log-likelihood | 0.47 | ||||
| log-likelihood, multi-turn, chat template | 0.47 | ||||
| log-likelihood | 0.47 | ||||
| qwen3-4b-instruct-2507 | generative-scoring, multi-turn, chat template, system prompt | 0.47 | |||
| log-likelihood, multi-turn, chat template, system prompt | 0.47 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.47 | ||||
| log-likelihood | 0.47 | ||||
| generative-scoring | 0.47 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.46 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.46 | ||||
| log-likelihood, multi-turn, chat template | 0.46 | ||||
| generative-scoring | 0.46 | ||||
| generative-scoring | 0.46 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.46 | ||||
| generative-scoring, multi-turn, chat template | 0.46 | ||||
| generative-scoring | 0.46 | ||||
| log-likelihood, system prompt | 0.46 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.46 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.46 | ||||
| log-likelihood | 0.46 | ||||
| generative-scoring | 0.46 | ||||
| log-likelihood, multi-turn, chat template | 0.46 | ||||
| log-likelihood | 0.45 | ||||
| log-likelihood, chat template | 0.45 | ||||
| generative-scoring | 0.45 | ||||
| log-likelihood | 0.45 | ||||
| generative-scoring, multi-turn, chat template | 0.45 | ||||
| generative-scoring, chat template | 0.45 | ||||
| log-likelihood | 0.45 | ||||
| generative-scoring | 0.45 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.45 | ||||
| generative-scoring | 0.45 | ||||
| generative-scoring, system prompt | 0.45 | ||||
| generative-scoring | 0.44 | ||||
| log-likelihood, chat template, system prompt | 0.44 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.44 | ||||
| log-likelihood | 0.44 | ||||
| log-likelihood, chat template | 0.44 | ||||
| log-likelihood, chat template | 0.44 | ||||
| log-likelihood, chat template, system prompt | 0.44 | ||||
| generative-scoring, chat template, system prompt | 0.44 | ||||
| GPT-3.5 Turbo Jan 2024 | generative-scoring, multi-turn, chat template | 0.44 | |||
| generative-scoring, chat template | 0.44 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.44 | ||||
| generative-scoring, chat template | 0.44 | ||||
| generative-scoring, chat template | 0.44 | ||||
| generative-scoring | 0.44 | ||||
| generative-scoring, chat template | 0.44 | ||||
| log-likelihood, chat template, system prompt | 0.44 | ||||
| generative-scoring, multi-turn, chat template | 0.44 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.43 | ||||
| generative-scoring, chat template, system prompt | 0.43 | ||||
| generative-scoring, chat template, system prompt | 0.42 | ||||
| log-likelihood, chat template | 0.42 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.42 | ||||
| log-likelihood, chat template | 0.42 | ||||
| log-likelihood, chat template | 0.42 | ||||
| log-likelihood, chat template, system prompt | 0.42 | ||||
| generative-scoring, chat template | 0.41 | ||||
| log-likelihood, system prompt | 0.41 | ||||
| log-likelihood | 0.41 | ||||
| log-likelihood | 0.40 | ||||
| generative-scoring, system prompt | 0.40 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.40 | ||||
| log-likelihood | 0.40 | ||||
| log-likelihood | 0.40 | ||||
| log-likelihood | 0.40 | ||||
| generative-scoring | 0.39 | ||||
| generative-scoring | 0.39 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.39 | ||||
| log-likelihood | 0.39 | ||||
| generative-scoring | 0.39 | ||||
| generative-scoring | 0.39 | ||||
| generative-scoring | 0.39 | ||||
| log-likelihood | 0.39 | ||||
| generative-scoring | 0.39 | ||||
| generative-scoring | 0.38 | ||||
| log-likelihood | 0.37 | ||||
| log-likelihood | 0.37 | ||||
| log-likelihood | 0.37 | ||||
| log-likelihood | 0.37 | ||||
| generative-scoring | 0.37 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.36 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.36 | ||||
| log-likelihood, system prompt | 0.36 | ||||
| log-likelihood, multi-turn, chat template | 0.36 | ||||
| generative-scoring, chat template | 0.35 | ||||
| log-likelihood, chat template, system prompt | 0.35 | ||||
| generative-scoring, chat template, system prompt | 0.35 | ||||
| generative-scoring, chat template | 0.35 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.35 | ||||
| generative-scoring, system prompt | 0.35 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.35 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.35 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.35 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.35 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.35 | ||||
| generative-scoring, chat template, system prompt | 0.35 | ||||
| generative-scoring | 0.34 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.34 | ||||
| log-likelihood, chat template, system prompt | 0.34 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.34 | ||||
| log-likelihood, chat template | 0.34 | ||||
| log-likelihood | 0.34 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.34 | ||||
| log-likelihood, chat template | 0.34 | ||||
| generative-scoring, multi-turn, chat template | 0.34 | ||||
| generative-scoring, chat template | 0.34 | ||||
| log-likelihood | 0.33 | ||||
| log-likelihood, chat template | 0.33 | ||||
| generative-scoring, chat template, system prompt | 0.33 | ||||
| generative-scoring | 0.33 | ||||
| generative-scoring | 0.33 | ||||
| generative-scoring, chat template | 0.33 | ||||
| log-likelihood, chat template | 0.33 | ||||
| generative-scoring, chat template | 0.33 | ||||
| log-likelihood | 0.33 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.32 | ||||
| log-likelihood, chat template, system prompt | 0.32 | ||||
| generative-scoring | 0.32 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.32 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.32 | ||||
| generative-scoring | 0.32 | ||||
| log-likelihood, chat template | 0.32 | ||||
| log-likelihood, system prompt | 0.32 | ||||
| log-likelihood, chat template | 0.32 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.31 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.31 | ||||
| log-likelihood, chat template | 0.31 | ||||
| log-likelihood, multi-turn, chat template | 0.31 | ||||
| generative-scoring, chat template | 0.31 | ||||
| generative-scoring, multi-turn, chat template | 0.31 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.31 | ||||
| generative-scoring, chat template | 0.31 | ||||
| generative-scoring, chat template | 0.30 | ||||
| generative-scoring, chat template | 0.30 | ||||
| log-likelihood, chat template | 0.30 | ||||
| generative-scoring | 0.30 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.30 | ||||
| log-likelihood, chat template | 0.30 | ||||
| log-likelihood, chat template | 0.30 | ||||
| log-likelihood | 0.30 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.30 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.30 | ||||
| generative-scoring, system prompt | 0.30 | ||||
| generative-scoring, multi-turn, chat template | 0.29 | ||||
| log-likelihood, chat template | 0.29 | ||||
| log-likelihood, chat template, system prompt | 0.29 | ||||
| generative-scoring, chat template, system prompt | 0.29 | ||||
| generative-scoring, chat template | 0.28 | ||||
| generative-scoring, chat template | 0.28 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.28 | ||||
| log-likelihood, chat template | 0.28 | ||||
| generative-scoring, chat template | 0.28 | ||||
| LFM2-8B-A1B | generative-scoring, multi-turn, chat template, system prompt | 0.28 | |||
| generative-scoring, chat template | 0.27 | ||||
| generative-scoring | 0.27 | ||||
| generative-scoring, chat template | 0.27 | ||||
| log-likelihood, chat template | 0.27 | ||||
| log-likelihood | 0.26 | ||||
| log-likelihood, multi-turn, chat template | 0.25 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.25 | ||||
| log-likelihood, system prompt | 0.25 | ||||
| log-likelihood | 0.25 | ||||
| generative-scoring, multi-turn, chat template | 0.25 | ||||
| generative-scoring | 0.24 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.24 | ||||
| generative-scoring | 0.24 | ||||
| generative-scoring, system prompt | 0.23 | ||||
| log-likelihood, chat template | 0.23 | ||||
| log-likelihood, chat template | 0.23 | ||||
| log-likelihood, chat template | 0.23 | ||||
| log-likelihood, chat template | 0.23 | ||||
| generative-scoring, chat template | 0.22 | ||||
| generative-scoring, chat template | 0.22 | ||||
| log-likelihood, chat template, system prompt | 0.22 | ||||
| generative-scoring, chat template, system prompt | 0.22 | ||||
| log-likelihood, system prompt | 0.21 | ||||
| LFM2.5-1.2B-Instruct | generative-scoring, multi-turn, chat template, system prompt | 0.20 | |||
| generative-scoring, chat template | 0.19 | ||||
| generative-scoring, chat template | 0.19 | ||||
| log-likelihood, chat template | 0.19 | ||||
| log-likelihood, chat template | 0.19 | ||||
| generative-scoring, system prompt | 0.19 | ||||
| generative-scoring, chat template | 0.19 | ||||
| log-likelihood, chat template | 0.18 | ||||
| log-likelihood, chat template | 0.18 | ||||
| generative-scoring, chat template, system prompt | 0.17 | ||||
| generative-scoring, chat template | 0.10 | ||||
| generative-scoring, chat template | 0.10 | ||||
| generative-scoring, chat template | 0.06 | ||||
| generative-scoring, chat template | 0.06 |
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: Official MERA leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| Claude Opus 4.6 | generative-scoring, multi-turn, chat template, system prompt | 0.86 | |||
| GPT-5.4 | generative-scoring, multi-turn, chat template, system prompt | 0.82 | |||
| GLM-5.1 | generative-scoring, multi-turn, chat template, system prompt | 0.80 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.80 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.79 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.79 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.78 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.77 | ||||
| Claude Opus 4.5 | generative-scoring, multi-turn, chat template, system prompt | 0.76 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.75 | ||||
| Kimi K2.5 | generative-scoring, multi-turn, chat template, system prompt | 0.75 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.73 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.73 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.71 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.71 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.71 | ||||
| GPT-5.2 | generative-scoring, multi-turn, chat template, system prompt | 0.71 | |||
| generative-scoring, multi-turn | 0.69 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.69 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.69 | ||||
| Claude 3.7 Sonnet | generative-scoring, multi-turn, chat template, system prompt | 0.68 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.68 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.68 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.68 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.68 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.68 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.67 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.67 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.66 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.65 | ||||
| GPT-4o Mar 2025 | generative-scoring, multi-turn, chat template, system prompt | 0.64 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.64 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.63 | ||||
| log-likelihood, system prompt | 0.62 | ||||
| generative-scoring, chat template, system prompt | 0.62 | ||||
| gpt-oss-20b | generative-scoring, multi-turn, chat template, system prompt | 0.62 | |||
| generative-scoring, system prompt | 0.61 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.61 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.60 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.60 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.60 | ||||
| generative-scoring, chat template, system prompt | 0.59 | ||||
| log-likelihood, chat template, system prompt | 0.59 | ||||
| log-likelihood, system prompt | 0.59 | ||||
| generative-scoring, system prompt | 0.58 | ||||
| log-likelihood | 0.57 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.57 | ||||
| GPT-4o mini | generative-scoring, multi-turn, chat template, system prompt | 0.57 | |||
| generative-scoring | 0.57 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.57 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.57 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.57 | ||||
| log-likelihood | 0.57 | ||||
| generative-scoring | 0.56 | ||||
| log-likelihood | 0.56 | ||||
| generative-scoring | 0.56 | ||||
| log-likelihood | 0.56 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.56 | ||||
| generative-scoring | 0.55 | ||||
| log-likelihood, chat template | 0.55 | ||||
| generative-scoring, chat template | 0.55 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.55 | ||||
| Claude 3.5 Haiku | generative-scoring, multi-turn, chat template, system prompt | 0.55 | |||
| generative-scoring, multi-turn, chat template, system prompt | 0.54 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.53 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.53 | ||||
| log-likelihood | 0.52 | ||||
| log-likelihood | 0.51 | ||||
| generative-scoring | 0.51 | ||||
| log-likelihood | 0.51 | ||||
| log-likelihood, system prompt | 0.51 | ||||
| generative-scoring | 0.51 | ||||
| log-likelihood | 0.50 | ||||
| generative-scoring, system prompt | 0.49 | ||||
| log-likelihood | 0.49 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.49 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.49 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.49 | ||||
| log-likelihood | 0.48 | ||||
| log-likelihood, chat template, system prompt | 0.48 | ||||
| generative-scoring | 0.48 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.48 | ||||
| generative-scoring | 0.48 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.48 | ||||
| generative-scoring, chat template, system prompt | 0.48 | ||||
| log-likelihood | 0.47 | ||||
| log-likelihood, multi-turn, chat template | 0.47 | ||||
| log-likelihood | 0.47 | ||||
| qwen3-4b-instruct-2507 | generative-scoring, multi-turn, chat template, system prompt | 0.47 | |||
| log-likelihood, multi-turn, chat template, system prompt | 0.47 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.47 | ||||
| log-likelihood | 0.47 | ||||
| generative-scoring | 0.47 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.46 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.46 | ||||
| log-likelihood, multi-turn, chat template | 0.46 | ||||
| generative-scoring | 0.46 | ||||
| generative-scoring | 0.46 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.46 | ||||
| generative-scoring, multi-turn, chat template | 0.46 | ||||
| generative-scoring | 0.46 | ||||
| log-likelihood, system prompt | 0.46 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.46 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.46 | ||||
| log-likelihood | 0.46 | ||||
| generative-scoring | 0.46 | ||||
| log-likelihood, multi-turn, chat template | 0.46 | ||||
| log-likelihood | 0.45 | ||||
| log-likelihood, chat template | 0.45 | ||||
| generative-scoring | 0.45 | ||||
| log-likelihood | 0.45 | ||||
| generative-scoring, multi-turn, chat template | 0.45 | ||||
| generative-scoring, chat template | 0.45 | ||||
| log-likelihood | 0.45 | ||||
| generative-scoring | 0.45 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.45 | ||||
| generative-scoring | 0.45 | ||||
| generative-scoring, system prompt | 0.45 | ||||
| generative-scoring | 0.44 | ||||
| log-likelihood, chat template, system prompt | 0.44 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.44 | ||||
| log-likelihood | 0.44 | ||||
| log-likelihood, chat template | 0.44 | ||||
| log-likelihood, chat template | 0.44 | ||||
| log-likelihood, chat template, system prompt | 0.44 | ||||
| generative-scoring, chat template, system prompt | 0.44 | ||||
| GPT-3.5 Turbo Jan 2024 | generative-scoring, multi-turn, chat template | 0.44 | |||
| generative-scoring, chat template | 0.44 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.44 | ||||
| generative-scoring, chat template | 0.44 | ||||
| generative-scoring, chat template | 0.44 | ||||
| generative-scoring | 0.44 | ||||
| generative-scoring, chat template | 0.44 | ||||
| log-likelihood, chat template, system prompt | 0.44 | ||||
| generative-scoring, multi-turn, chat template | 0.44 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.43 | ||||
| generative-scoring, chat template, system prompt | 0.43 | ||||
| generative-scoring, chat template, system prompt | 0.42 | ||||
| log-likelihood, chat template | 0.42 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.42 | ||||
| log-likelihood, chat template | 0.42 | ||||
| log-likelihood, chat template | 0.42 | ||||
| log-likelihood, chat template, system prompt | 0.42 | ||||
| generative-scoring, chat template | 0.41 | ||||
| log-likelihood, system prompt | 0.41 | ||||
| log-likelihood | 0.41 | ||||
| log-likelihood | 0.40 | ||||
| generative-scoring, system prompt | 0.40 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.40 | ||||
| log-likelihood | 0.40 | ||||
| log-likelihood | 0.40 | ||||
| log-likelihood | 0.40 | ||||
| generative-scoring | 0.39 | ||||
| generative-scoring | 0.39 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.39 | ||||
| log-likelihood | 0.39 | ||||
| generative-scoring | 0.39 | ||||
| generative-scoring | 0.39 | ||||
| generative-scoring | 0.39 | ||||
| log-likelihood | 0.39 | ||||
| generative-scoring | 0.39 | ||||
| generative-scoring | 0.38 | ||||
| log-likelihood | 0.37 | ||||
| log-likelihood | 0.37 | ||||
| log-likelihood | 0.37 | ||||
| log-likelihood | 0.37 | ||||
| generative-scoring | 0.37 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.36 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.36 | ||||
| log-likelihood, system prompt | 0.36 | ||||
| log-likelihood, multi-turn, chat template | 0.36 | ||||
| generative-scoring, chat template | 0.35 | ||||
| log-likelihood, chat template, system prompt | 0.35 | ||||
| generative-scoring, chat template, system prompt | 0.35 | ||||
| generative-scoring, chat template | 0.35 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.35 | ||||
| generative-scoring, system prompt | 0.35 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.35 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.35 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.35 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.35 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.35 | ||||
| generative-scoring, chat template, system prompt | 0.35 | ||||
| generative-scoring | 0.34 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.34 | ||||
| log-likelihood, chat template, system prompt | 0.34 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.34 | ||||
| log-likelihood, chat template | 0.34 | ||||
| log-likelihood | 0.34 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.34 | ||||
| log-likelihood, chat template | 0.34 | ||||
| generative-scoring, multi-turn, chat template | 0.34 | ||||
| generative-scoring, chat template | 0.34 | ||||
| log-likelihood | 0.33 | ||||
| log-likelihood, chat template | 0.33 | ||||
| generative-scoring, chat template, system prompt | 0.33 | ||||
| generative-scoring | 0.33 | ||||
| generative-scoring | 0.33 | ||||
| generative-scoring, chat template | 0.33 | ||||
| log-likelihood, chat template | 0.33 | ||||
| generative-scoring, chat template | 0.33 | ||||
| log-likelihood | 0.33 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.32 | ||||
| log-likelihood, chat template, system prompt | 0.32 | ||||
| generative-scoring | 0.32 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.32 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.32 | ||||
| generative-scoring | 0.32 | ||||
| log-likelihood, chat template | 0.32 | ||||
| log-likelihood, system prompt | 0.32 | ||||
| log-likelihood, chat template | 0.32 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.31 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.31 | ||||
| log-likelihood, chat template | 0.31 | ||||
| log-likelihood, multi-turn, chat template | 0.31 | ||||
| generative-scoring, chat template | 0.31 | ||||
| generative-scoring, multi-turn, chat template | 0.31 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.31 | ||||
| generative-scoring, chat template | 0.31 | ||||
| generative-scoring, chat template | 0.30 | ||||
| generative-scoring, chat template | 0.30 | ||||
| log-likelihood, chat template | 0.30 | ||||
| generative-scoring | 0.30 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.30 | ||||
| log-likelihood, chat template | 0.30 | ||||
| log-likelihood, chat template | 0.30 | ||||
| log-likelihood | 0.30 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.30 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.30 | ||||
| generative-scoring, system prompt | 0.30 | ||||
| generative-scoring, multi-turn, chat template | 0.29 | ||||
| log-likelihood, chat template | 0.29 | ||||
| log-likelihood, chat template, system prompt | 0.29 | ||||
| generative-scoring, chat template, system prompt | 0.29 | ||||
| generative-scoring, chat template | 0.28 | ||||
| generative-scoring, chat template | 0.28 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.28 | ||||
| log-likelihood, chat template | 0.28 | ||||
| generative-scoring, chat template | 0.28 | ||||
| LFM2-8B-A1B | generative-scoring, multi-turn, chat template, system prompt | 0.28 | |||
| generative-scoring, chat template | 0.27 | ||||
| generative-scoring | 0.27 | ||||
| generative-scoring, chat template | 0.27 | ||||
| log-likelihood, chat template | 0.27 | ||||
| log-likelihood | 0.26 | ||||
| log-likelihood, multi-turn, chat template | 0.25 | ||||
| log-likelihood, multi-turn, chat template, system prompt | 0.25 | ||||
| log-likelihood, system prompt | 0.25 | ||||
| log-likelihood | 0.25 | ||||
| generative-scoring, multi-turn, chat template | 0.25 | ||||
| generative-scoring | 0.24 | ||||
| generative-scoring, multi-turn, chat template, system prompt | 0.24 | ||||
| generative-scoring | 0.24 | ||||
| generative-scoring, system prompt | 0.23 | ||||
| log-likelihood, chat template | 0.23 | ||||
| log-likelihood, chat template | 0.23 | ||||
| log-likelihood, chat template | 0.23 | ||||
| log-likelihood, chat template | 0.23 | ||||
| generative-scoring, chat template | 0.22 | ||||
| generative-scoring, chat template | 0.22 | ||||
| log-likelihood, chat template, system prompt | 0.22 | ||||
| generative-scoring, chat template, system prompt | 0.22 | ||||
| log-likelihood, system prompt | 0.21 | ||||
| LFM2.5-1.2B-Instruct | generative-scoring, multi-turn, chat template, system prompt | 0.20 | |||
| generative-scoring, chat template | 0.19 | ||||
| generative-scoring, chat template | 0.19 | ||||
| log-likelihood, chat template | 0.19 | ||||
| log-likelihood, chat template | 0.19 | ||||
| generative-scoring, system prompt | 0.19 | ||||
| generative-scoring, chat template | 0.19 | ||||
| log-likelihood, chat template | 0.18 | ||||
| log-likelihood, chat template | 0.18 | ||||
| generative-scoring, chat template, system prompt | 0.17 | ||||
| generative-scoring, chat template | 0.10 | ||||
| generative-scoring, chat template | 0.10 | ||||
| generative-scoring, chat template | 0.06 | ||||
| generative-scoring, chat template | 0.06 |
13 models have held number one on the official leaderboard since its release in November 2023
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official MERA leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official MERA leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5%
- MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87%
- CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7%
- Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
- MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%
Data audit
8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (209 entries).
- Pass Models scored 102 models we track, plus 107 other entries on its leaderboard.
- Warning Newest score The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash.
- Note Frontier models None of the top 10 has a score. The strongest tested is GPT-5.4 (#28).
- Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on mera.a-ai.ru: 277 rows for models we track and 107 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.