BenCzechMark
Models complete Czech language tasks spanning knowledge, comprehension, and reasoning.
The benchmark ranks models by statistically significant wins across tasks.
- Made by
- Czech Language Community
- Released
- October 2024
- Kind
- Knowledge in many languages
- Level
- University
- Scored
- Checked automatically
- Answers
- Prediction
- Tasks
- All public
- Measures
- Capability
- Subject
- Languages and literature
- Language
- Czech
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
-
DeepSeek V4 Flash 0731 #52 on the capability index
- Last model added
- August 2026
- Scores refreshed
- November 20th, 2026
DeepSeek-V3 leads with 82.9 among models released by August 2026 DeepSeek-V3 leads with 82.9 among models released by August 2026
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
The newest model with a score here came out in August 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 82.9
DeepSeek-V3
- 82.3
Llama 3.1-405B
- 81.0
DeepSeek V4 Flash 0731
- 80.6
DeepSeek-R1
- 77.2
Qwen2.5-72B
- 77.0
Cohere Command A
- 72.7
Mistral Large 2
- 72.6
Llama 4 Maverick
- 71.8
Gemma 4 31B IT
- 70.8
Llama 3.3 70B Instruct
- 70.8 anex.sh/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8
- 70.3
MiniMax-M1-80k
- 70.2
Cogito v1 Preview 70B
- 70.0 anex.sh/mistralai/Mistral-Small-4-119B-2603-NVFP4
- 69.7
Qwen3.6 27B
- 68.4
Llama 4 Scout
- 68.3 meta-llama/Llama-3.1-70B-Instruct
- 66.0
Qwen 3.6 35B-A3B
- 65.6 CRA/nvidia/Qwen3.6-35B-A3B-NVFP4
- 65.4
Muse Glimmer
- 64.6 Qwen/Qwen2.5-32B-Instruct
- 64.2
Llama 3.1-70B
- 63.6 CRA/unsloth/Qwen3.8-27B-NVFP4
- 62.6 anex.sh/Qwen/Qwen3.8-27B-FP8
- 61.4
Qwen2-72B
- 61.0
Gemma 4 26B A4B
- 57.8 mistralai/Mixtral-8x22B-Instruct-v0.1
- 57.2
Aya Expanse 32B
- 56.6
Gemma 3 27B
- 54.4 AMead10/c4ai-command-r-08-2024-awq
- 52.7
Phi-4
- 50.5 google/gemma-2-27b-it
- 46.4
Qwen2.5-7B
- 44.4 Qwen/Qwen2.5-7B-Instruct
- 44.1
Gemma 4 E4B
- 42.6 speakleash/Bielik-11B-v2.3-Instruct
- 42.5 CohereForAI/aya-23-35B
- 42.0 meta-llama/Llama-3.1-8B-Instruct
- 41.8 mistralai/Mistral-Nemo-Instruct-2407
- 40.6
Apertus 8B Instruct
- 40.5 mistralai/Mixtral-8x7B-Instruct-v0.1
- 37.3 NousResearch/Hermes-3-Llama-3.1-8B
- 36.7
Aya Expanse 8B
- 36.3 meta-llama/Llama-3-8B-Instruct
- 35.9
Gemma 2 27B
- 35.3
Qwen2-7B
- 34.6
Qwen2-7B-Instruct
- 34.6 mistralai/Ministral-8B-Instruct-2410
- 33.8
Gemma 2 9B
- 33.0
Llama 3.1-8B
- 32.4 google/gemma-2-9b-it
- 30.5
Gemma 4 E2B
- 29.6 Granite-3.0-8B-Base
- 28.9 utter-project/EuroLLM-22B-Instruct-Preview
- 28.6 CohereForAI/aya-23-8B
- 25.7 mistralai/Mistral-7B-Instruct-v0.3
- 24.0 meta-llama/Llama-3.2-3B-Instruct
- 23.8 BSC-LT/ALIA-40b
- 22.5 Granite-3.0-8B-Instruct
- 21.2 utter-project/EuroLLM-9B-Instruct
- 20.7
Llama 3.2 3B
- 19.2 BSC-LT/salamandra-7b-instruct
- 19.0
Gemma 2 2B IT
- 18.5 internlm/internlm2_5-7b-chat
- 17.8 utter-project/EuroLLM-9B
- 15.8 ibm-granite/granite-3.0-3b-a800m-instruct
- 14.3 ibm-granite/granite-3.0-3b-a800m-base
- 10.5
phi-3-mini 3.8B
- 8.67 meta-llama/Llama-3.2-1B-Instruct
- 8.12 google/gemma-2-2b
- 8.02
Llama 3.2 1B
- 6.48
OLMo-7B-Instruct
- 6.09
MPT-7B
- 5.55 BUT-FIT/csmpt7b
- 5.28 utter-project/EuroLLM-1.7B-Instruct
- 3.59 utter-project/EuroLLM-1.7B
- 3.53 anex.sh/LiquidAI/LFM2-24B-A2B
- 1.96 BUT-FIT/CSTinyLlama-1.2B
- 1
DeepSeek-V3 Mar 2025, 8bit, 3-shot, max input 2048 82.9
- 2
Llama 3.1-405B bfloat16, 3-shot, max input 2048 82.3
- 3
DeepSeek V4 Flash 0731 8bit, 3-shot, max input 2048 81.0
- 4
DeepSeek-R1 May 2025, 8bit, 3-shot, max input 2048 80.6
- 5
Qwen2.5-72B bfloat16, 3-shot, max input 2048 77.2
- 6
Cohere Command A 8bit, 3-shot, max input 2048 77.0
- 7
Mistral Large 2 Nov 2024, bfloat16, 3-shot, max input 2048 72.7
- 8
Llama 4 Maverick 8bit, 3-shot, max input 2048 72.6
- 9
Gemma 4 31B IT bfloat16, 3-shot, max input 2048, chat template, multiturn 71.8
- 10
Llama 3.3 70B Instruct bfloat16, 3-shot, max input 2048 70.8
- 11 anex.sh/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 8bit, 3-shot, max input 2048 70.8
- 12
MiniMax-M1-80k 8bit, 3-shot, max input 2048 70.3
- 13
Cogito v1 Preview 70B 8bit, 3-shot, max input 2048 70.2
- 14 anex.sh/mistralai/Mistral-Small-4-119B-2603-NVFP4 4bit, 3-shot, max input 2048 70.0
- 15
Qwen3.6 27B bfloat16, 3-shot, max input 2048 69.7
- 16
Llama 4 Scout 8bit, 3-shot, max input 2048 68.4
- 17 meta-llama/Llama-3.1-70B-Instruct bfloat16, 3-shot, max input 2048 68.3
- 18
Qwen 3.6 35B-A3B 8bit, 3-shot, max input 2048 66.0
- 19 CRA/nvidia/Qwen3.6-35B-A3B-NVFP4 4bit, 3-shot, max input 2048 65.6
- 20
Muse Glimmer bfloat16, 3-shot, max input 2048, chat template, multiturn 65.4
- 21 Qwen/Qwen2.5-32B-Instruct bfloat16, 3-shot, max input 2048 64.6
- 22
Llama 3.1-70B bfloat16, 3-shot, max input 2048 64.2
- 23 CRA/unsloth/Qwen3.8-27B-NVFP4 4bit, 3-shot, max input 2048 63.6
- 24 anex.sh/Qwen/Qwen3.8-27B-FP8 8bit, 3-shot, max input 2048 62.6
- 25
Qwen2-72B bfloat16, 3-shot, max input 2048 61.4
- 26
Gemma 4 26B A4B bfloat16, 3-shot, max input 2048 61.0
- 27 mistralai/Mixtral-8x22B-Instruct-v0.1 bfloat16, 3-shot, max input 2048 57.8
- 28
Aya Expanse 32B bfloat16, 3-shot, max input 2048 57.2
- 29
Gemma 3 27B bfloat16, 3-shot, max input 2048 56.6
- 30 AMead10/c4ai-command-r-08-2024-awq 4bit, 3-shot, max input 2048 54.4
- 31
Phi-4 bfloat16, 3-shot, max input 2048 52.7
- 32 google/gemma-2-27b-it bfloat16, 3-shot, max input 2048 50.5
- 33
Qwen2.5-7B bfloat16, 3-shot, max input 2048 46.4
- 34 Qwen/Qwen2.5-7B-Instruct bfloat16, 3-shot, max input 2048 44.4
- 35
Gemma 4 E4B bfloat16, 3-shot, max input 2048, chat template, multiturn 44.1
- 36 speakleash/Bielik-11B-v2.3-Instruct bfloat16, 3-shot, max input 2048 42.6
- 37 CohereForAI/aya-23-35B bfloat16, 3-shot, max input 2048 42.5
- 38 meta-llama/Llama-3.1-8B-Instruct bfloat16, 3-shot, max input 1792 42.0
- 39 mistralai/Mistral-Nemo-Instruct-2407 bfloat16, 3-shot, max input 2048 41.8
- 40
Apertus 8B Instruct bfloat16, 3-shot, max input 2048 40.6
- 41 mistralai/Mixtral-8x7B-Instruct-v0.1 bfloat16, 3-shot, max input 2048 40.5
- 42 NousResearch/Hermes-3-Llama-3.1-8B bfloat16, 3-shot, max input 2048 37.3
- 43
Aya Expanse 8B bfloat16, 3-shot, max input 2048 36.7
- 44 meta-llama/Llama-3-8B-Instruct bfloat16, 3-shot, max input 2048 36.3
- 45
Gemma 2 27B bfloat16, 3-shot, max input 2048 35.9
- 46
Qwen2-7B bfloat16, 3-shot, max input 2048 35.3
- 47
Qwen2-7B-Instruct bfloat16, 3-shot, max input 1792 34.6
- 48 mistralai/Ministral-8B-Instruct-2410 bfloat16, 3-shot, max input 2048 34.6
- 49
Gemma 2 9B bfloat16, 3-shot, max input 2048 33.8
- 50
Llama 3.1-8B bfloat16, 3-shot, max input 2048 33.0
- 51 google/gemma-2-9b-it bfloat16, 3-shot, max input 2048 32.4
- 52
Gemma 4 E2B bfloat16, 3-shot, max input 2048, chat template, multiturn 30.5
- 53 Granite-3.0-8B-Base bfloat16, 3-shot, max input 2048 29.6
- 54 utter-project/EuroLLM-22B-Instruct-Preview 8bit, 3-shot, max input 2048 28.9
- 55 CohereForAI/aya-23-8B bfloat16, 3-shot, max input 2048 28.6
- 56 mistralai/Mistral-7B-Instruct-v0.3 bfloat16, 3-shot, max input 2048 25.7
- 57 meta-llama/Llama-3.2-3B-Instruct bfloat16, 3-shot, max input 2048 24.0
- 58 BSC-LT/ALIA-40b bfloat16, 3-shot, max input 2048 23.8
- 59 Granite-3.0-8B-Instruct bfloat16, 3-shot, max input 2048 22.5
- 60 utter-project/EuroLLM-9B-Instruct bfloat16, 3-shot, max input 2048 21.2
- 61
Llama 3.2 3B bfloat16, 3-shot, max input 2048 20.7
- 62 BSC-LT/salamandra-7b-instruct bfloat16, 3-shot, max input 2048 19.2
- 63
Gemma 2 2B IT bfloat16, 3-shot, max input 2048 19.0
- 64 internlm/internlm2_5-7b-chat bfloat16, 3-shot, max input 2048 18.5
- 65 utter-project/EuroLLM-9B bfloat16, 3-shot, max input 2048 17.8
- 66 ibm-granite/granite-3.0-3b-a800m-instruct bfloat16, 3-shot, max input 2048 15.8
- 67 ibm-granite/granite-3.0-3b-a800m-base bfloat16, 3-shot, max input 2048 14.3
- 68
phi-3-mini 3.8B bfloat16, 3-shot, max input 2048 10.5
- 69 meta-llama/Llama-3.2-1B-Instruct bfloat16, 3-shot, max input 2048 8.67
- 70 google/gemma-2-2b bfloat16, 3-shot, max input 2048 8.12
- 71
Llama 3.2 1B bfloat16, 3-shot, max input 2048 8.02
- 72
OLMo-7B-Instruct bfloat16, 3-shot, max input 2048 6.48
- 73
MPT-7B bfloat16, 3-shot, max input 2048 6.09
- 74 BUT-FIT/csmpt7b bfloat16, 3-shot, max input 2048 5.55
- 75 utter-project/EuroLLM-1.7B-Instruct bfloat16, 3-shot, max input 2048 5.28
- 76 utter-project/EuroLLM-1.7B bfloat16, 3-shot, max input 2048 3.59
- 77 anex.sh/LiquidAI/LFM2-24B-A2B bfloat16, 3-shot, max input 2048 3.53
- 78 BUT-FIT/CSTinyLlama-1.2B bfloat16, 3-shot, max input 2048 1.96
Source: Official BenCzechMark leaderboard
- 82.9
DeepSeek-V3
- 82.3
Llama 3.1-405B
- 81.0
DeepSeek V4 Flash 0731
- 80.6
DeepSeek-R1
- 77.2
Qwen2.5-72B
- 77.0
Cohere Command A
- 72.7
Mistral Large 2
- 72.6
Llama 4 Maverick
- 71.8
Gemma 4 31B IT
- 70.8
Llama 3.3 70B Instruct
- 70.3
MiniMax-M1-80k
- 70.2
Cogito v1 Preview 70B
- 69.7
Qwen3.6 27B
- 68.4
Llama 4 Scout
- 66.0
Qwen 3.6 35B-A3B
- 65.4
Muse Glimmer
- 64.2
Llama 3.1-70B
- 61.4
Qwen2-72B
- 61.0
Gemma 4 26B A4B
- 57.2
Aya Expanse 32B
- 56.6
Gemma 3 27B
- 52.7
Phi-4
- 46.4
Qwen2.5-7B
- 44.1
Gemma 4 E4B
- 40.6
Apertus 8B Instruct
- 36.7
Aya Expanse 8B
- 35.9
Gemma 2 27B
- 35.3
Qwen2-7B
- 34.6
Qwen2-7B-Instruct
- 33.8
Gemma 2 9B
- 33.0
Llama 3.1-8B
- 30.5
Gemma 4 E2B
- 29.6 Granite-3.0-8B-Base
- 22.5 Granite-3.0-8B-Instruct
- 20.7
Llama 3.2 3B
- 19.0
Gemma 2 2B IT
- 10.5
phi-3-mini 3.8B
- 8.02
Llama 3.2 1B
- 6.48
OLMo-7B-Instruct
- 6.09
MPT-7B
- 1
DeepSeek-V3 Mar 2025, 8bit, 3-shot, max input 2048 82.9
- 2
Llama 3.1-405B bfloat16, 3-shot, max input 2048 82.3
- 3
DeepSeek V4 Flash 0731 8bit, 3-shot, max input 2048 81.0
- 4
DeepSeek-R1 May 2025, 8bit, 3-shot, max input 2048 80.6
- 5
Qwen2.5-72B bfloat16, 3-shot, max input 2048 77.2
- 6
Cohere Command A 8bit, 3-shot, max input 2048 77.0
- 7
Mistral Large 2 Nov 2024, bfloat16, 3-shot, max input 2048 72.7
- 8
Llama 4 Maverick 8bit, 3-shot, max input 2048 72.6
- 9
Gemma 4 31B IT bfloat16, 3-shot, max input 2048, chat template, multiturn 71.8
- 10
Llama 3.3 70B Instruct bfloat16, 3-shot, max input 2048 70.8
- 11
MiniMax-M1-80k 8bit, 3-shot, max input 2048 70.3
- 12
Cogito v1 Preview 70B 8bit, 3-shot, max input 2048 70.2
- 13
Qwen3.6 27B bfloat16, 3-shot, max input 2048 69.7
- 14
Llama 4 Scout 8bit, 3-shot, max input 2048 68.4
- 15
Qwen 3.6 35B-A3B 8bit, 3-shot, max input 2048 66.0
- 16
Muse Glimmer bfloat16, 3-shot, max input 2048, chat template, multiturn 65.4
- 17
Llama 3.1-70B bfloat16, 3-shot, max input 2048 64.2
- 18
Qwen2-72B bfloat16, 3-shot, max input 2048 61.4
- 19
Gemma 4 26B A4B bfloat16, 3-shot, max input 2048 61.0
- 20
Aya Expanse 32B bfloat16, 3-shot, max input 2048 57.2
- 21
Gemma 3 27B bfloat16, 3-shot, max input 2048 56.6
- 22
Phi-4 bfloat16, 3-shot, max input 2048 52.7
- 23
Qwen2.5-7B bfloat16, 3-shot, max input 2048 46.4
- 24
Gemma 4 E4B bfloat16, 3-shot, max input 2048, chat template, multiturn 44.1
- 25
Apertus 8B Instruct bfloat16, 3-shot, max input 2048 40.6
- 26
Aya Expanse 8B bfloat16, 3-shot, max input 2048 36.7
- 27
Gemma 2 27B bfloat16, 3-shot, max input 2048 35.9
- 28
Qwen2-7B bfloat16, 3-shot, max input 2048 35.3
- 29
Qwen2-7B-Instruct bfloat16, 3-shot, max input 1792 34.6
- 30
Gemma 2 9B bfloat16, 3-shot, max input 2048 33.8
- 31
Llama 3.1-8B bfloat16, 3-shot, max input 2048 33.0
- 32
Gemma 4 E2B bfloat16, 3-shot, max input 2048, chat template, multiturn 30.5
- 33 Granite-3.0-8B-Base bfloat16, 3-shot, max input 2048 29.6
- 34 Granite-3.0-8B-Instruct bfloat16, 3-shot, max input 2048 22.5
- 35
Llama 3.2 3B bfloat16, 3-shot, max input 2048 20.7
- 36
Gemma 2 2B IT bfloat16, 3-shot, max input 2048 19.0
- 37
phi-3-mini 3.8B bfloat16, 3-shot, max input 2048 10.5
- 38
Llama 3.2 1B bfloat16, 3-shot, max input 2048 8.02
- 39
OLMo-7B-Instruct bfloat16, 3-shot, max input 2048 6.48
- 40
MPT-7B bfloat16, 3-shot, max input 2048 6.09
Source: Official BenCzechMark leaderboard
| # | Entry | Score | Setting | Released | Run on | Link | Source |
|---|---|---|---|---|---|---|---|
| 1 | | 82.9 | 8bit, 3-shot, max input 2048 | Mar 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 2 | | 82.3 | bfloat16, 3-shot, max input 2048 | Jul 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 3 | | 81.0 | 8bit, 3-shot, max input 2048 | Jul 2026 | Aug 2026 | huggingface.co | Official leaderboard |
| 4 | | 80.6 | 8bit, 3-shot, max input 2048 | May 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 5 | | 77.2 | bfloat16, 3-shot, max input 2048 | Sep 2024 | Nov 2024 | huggingface.co | Official leaderboard |
| 6 | | 77.0 | 8bit, 3-shot, max input 2048 | Mar 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 7 | | 72.7 | bfloat16, 3-shot, max input 2048 | Nov 2024 | Jan 2025 | huggingface.co | Official leaderboard |
| 8 | | 72.6 | 8bit, 3-shot, max input 2048 | Apr 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 9 | | 71.8 | bfloat16, 3-shot, max input 2048, chat template, multiturn | Apr 2026 | May 2026 | huggingface.co | Official leaderboard |
| 10 | | 70.8 | bfloat16, 3-shot, max input 2048 | Dec 2024 | Dec 2024 | huggingface.co | Official leaderboard |
| 11 | anex.sh/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 27.8B | 70.8 | 8bit, 3-shot, max input 2048 | Aug 2026 | huggingface.co | Official leaderboard | |
| 12 | | 70.3 | 8bit, 3-shot, max input 2048 | Jun 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 13 | | 70.2 | 8bit, 3-shot, max input 2048 | Apr 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 14 | anex.sh/mistralai/Mistral-Small-4-119B-2603-NVFP4 119B | 70.0 | 4bit, 3-shot, max input 2048 | Aug 2026 | huggingface.co | Official leaderboard | |
| 15 | | 69.7 | bfloat16, 3-shot, max input 2048 | Apr 2026 | Aug 2026 | huggingface.co | Official leaderboard |
| 16 | | 68.4 | 8bit, 3-shot, max input 2048 | Apr 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 17 | meta-llama/Llama-3.1-70B-Instruct 70.6B | 68.3 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 18 | | 66.0 | 8bit, 3-shot, max input 2048 | Apr 2026 | Aug 2026 | huggingface.co | Official leaderboard |
| 19 | CRA/nvidia/Qwen3.6-35B-A3B-NVFP4 27.8B | 65.6 | 4bit, 3-shot, max input 2048 | Aug 2026 | huggingface.co | Official leaderboard | |
| 20 | | 65.4 | bfloat16, 3-shot, max input 2048, chat template, multiturn | Aug 2026 | Oct 2026 | huggingface.co | Official leaderboard |
| 21 | Qwen/Qwen2.5-32B-Instruct 32.8B | 64.6 | bfloat16, 3-shot, max input 2048 | Nov 2024 | huggingface.co | Official leaderboard | |
| 22 | | 64.2 | bfloat16, 3-shot, max input 2048 | Jul 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 23 | CRA/unsloth/Qwen3.8-27B-NVFP4 27B | 63.6 | 4bit, 3-shot, max input 2048 | Sep 2026 | huggingface.co | Official leaderboard | |
| 24 | anex.sh/Qwen/Qwen3.8-27B-FP8 27.8B | 62.6 | 8bit, 3-shot, max input 2048 | Aug 2026 | huggingface.co | Official leaderboard | |
| 25 | | 61.4 | bfloat16, 3-shot, max input 2048 | Jun 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 26 | | 61.0 | bfloat16, 3-shot, max input 2048 | Apr 2026 | May 2026 | huggingface.co | Official leaderboard |
| 27 | mistralai/Mixtral-8x22B-Instruct-v0.1 141B | 57.8 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 28 | | 57.2 | bfloat16, 3-shot, max input 2048 | Oct 2024 | Dec 2024 | huggingface.co | Official leaderboard |
| 29 | | 56.6 | bfloat16, 3-shot, max input 2048 | Mar 2025 | May 2025 | huggingface.co | Official leaderboard |
| 30 | AMead10/c4ai-command-r-08-2024-awq 32.3B | 54.4 | 4bit, 3-shot, max input 2048 | Oct 2024 | huggingface.co | Official leaderboard | |
| 31 | | 52.7 | bfloat16, 3-shot, max input 2048 | Dec 2024 | Jan 2025 | huggingface.co | Official leaderboard |
| 32 | google/gemma-2-27b-it 27.2B | 50.5 | bfloat16, 3-shot, max input 2048 | Dec 2024 | huggingface.co | Official leaderboard | |
| 33 | | 46.4 | bfloat16, 3-shot, max input 2048 | Sep 2024 | Nov 2024 | huggingface.co | Official leaderboard |
| 34 | Qwen/Qwen2.5-7B-Instruct 7.6B | 44.4 | bfloat16, 3-shot, max input 2048 | Nov 2024 | huggingface.co | Official leaderboard | |
| 35 | | 44.1 | bfloat16, 3-shot, max input 2048, chat template, multiturn | Mar 2026 | May 2026 | huggingface.co | Official leaderboard |
| 36 | speakleash/Bielik-11B-v2.3-Instruct 11.2B | 42.6 | bfloat16, 3-shot, max input 2048 | Oct 2024 | huggingface.co | Official leaderboard | |
| 37 | CohereForAI/aya-23-35B 35B | 42.5 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 38 | meta-llama/Llama-3.1-8B-Instruct 8B | 42.0 | bfloat16, 3-shot, max input 1792 | Sep 2024 | huggingface.co | Official leaderboard | |
| 39 | mistralai/Mistral-Nemo-Instruct-2407 12.2B | 41.8 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 40 | | 40.6 | bfloat16, 3-shot, max input 2048 | Sep 2025 | Aug 2026 | huggingface.co | Official leaderboard |
| 41 | mistralai/Mixtral-8x7B-Instruct-v0.1 46.7B | 40.5 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 42 | NousResearch/Hermes-3-Llama-3.1-8B 8B | 37.3 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 43 | | 36.7 | bfloat16, 3-shot, max input 2048 | Oct 2024 | Dec 2024 | huggingface.co | Official leaderboard |
| 44 | meta-llama/Llama-3-8B-Instruct 8B | 36.3 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 45 | | 35.9 | bfloat16, 3-shot, max input 2048 | Jun 2024 | Dec 2024 | huggingface.co | Official leaderboard |
| 46 | | 35.3 | bfloat16, 3-shot, max input 2048 | Jun 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 47 | | 34.6 | bfloat16, 3-shot, max input 1792 | Jun 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 48 | mistralai/Ministral-8B-Instruct-2410 8B | 34.6 | bfloat16, 3-shot, max input 2048 | Dec 2024 | huggingface.co | Official leaderboard | |
| 49 | | 33.8 | bfloat16, 3-shot, max input 2048 | Jun 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 50 | | 33.0 | bfloat16, 3-shot, max input 2048 | Jul 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 51 | google/gemma-2-9b-it 9.2B | 32.4 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 52 | | 30.5 | bfloat16, 3-shot, max input 2048, chat template, multiturn | Mar 2026 | May 2026 | huggingface.co | Official leaderboard |
| 53 | Granite-3.0-8B-Base | 29.6 | bfloat16, 3-shot, max input 2048 | Oct 2024 | Nov 2024 | huggingface.co | Official leaderboard |
| 54 | utter-project/EuroLLM-22B-Instruct-Preview 22B | 28.9 | 8bit, 3-shot, max input 2048 | Jul 2025 | huggingface.co | Official leaderboard | |
| 55 | CohereForAI/aya-23-8B 8B | 28.6 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 56 | mistralai/Mistral-7B-Instruct-v0.3 7.3B | 25.7 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 57 | meta-llama/Llama-3.2-3B-Instruct 3.2B | 24.0 | bfloat16, 3-shot, max input 2048 | Oct 2024 | huggingface.co | Official leaderboard | |
| 58 | BSC-LT/ALIA-40b 40.4B | 23.8 | bfloat16, 3-shot, max input 2048 | Jan 2025 | huggingface.co | Official leaderboard | |
| 59 | Granite-3.0-8B-Instruct | 22.5 | bfloat16, 3-shot, max input 2048 | Oct 2024 | Nov 2024 | huggingface.co | Official leaderboard |
| 60 | utter-project/EuroLLM-9B-Instruct 9.2B | 21.2 | bfloat16, 3-shot, max input 2048 | Dec 2024 | huggingface.co | Official leaderboard | |
| 61 | | 20.7 | bfloat16, 3-shot, max input 2048 | Sep 2024 | Oct 2024 | huggingface.co | Official leaderboard |
| 62 | BSC-LT/salamandra-7b-instruct 7.8B | 19.2 | bfloat16, 3-shot, max input 2048 | Dec 2024 | huggingface.co | Official leaderboard | |
| 63 | | 19.0 | bfloat16, 3-shot, max input 2048 | Jul 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 64 | internlm/internlm2_5-7b-chat 7.7B | 18.5 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 65 | utter-project/EuroLLM-9B 9.2B | 17.8 | bfloat16, 3-shot, max input 2048 | Dec 2024 | huggingface.co | Official leaderboard | |
| 66 | ibm-granite/granite-3.0-3b-a800m-instruct 3.4B | 15.8 | bfloat16, 3-shot, max input 2048 | Nov 2024 | huggingface.co | Official leaderboard | |
| 67 | ibm-granite/granite-3.0-3b-a800m-base 3.4B | 14.3 | bfloat16, 3-shot, max input 2048 | Nov 2024 | huggingface.co | Official leaderboard | |
| 68 | | 10.5 | bfloat16, 3-shot, max input 2048 | Apr 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 69 | meta-llama/Llama-3.2-1B-Instruct 1.2B | 8.67 | bfloat16, 3-shot, max input 2048 | Oct 2024 | huggingface.co | Official leaderboard | |
| 70 | google/gemma-2-2b 2.6B | 8.12 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 71 | | 8.02 | bfloat16, 3-shot, max input 2048 | Sep 2024 | Oct 2024 | huggingface.co | Official leaderboard |
| 72 | | 6.48 | bfloat16, 3-shot, max input 2048 | Feb 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 73 | | 6.09 | bfloat16, 3-shot, max input 2048 | May 2023 | Sep 2024 | huggingface.co | Official leaderboard |
| 74 | BUT-FIT/csmpt7b 6.7B | 5.55 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard | |
| 75 | utter-project/EuroLLM-1.7B-Instruct 1.7B | 5.28 | bfloat16, 3-shot, max input 2048 | Nov 2024 | huggingface.co | Official leaderboard | |
| 76 | utter-project/EuroLLM-1.7B 1.7B | 3.59 | bfloat16, 3-shot, max input 2048 | Nov 2024 | huggingface.co | Official leaderboard | |
| 77 | anex.sh/LiquidAI/LFM2-24B-A2B 23.8B | 3.53 | bfloat16, 3-shot, max input 2048 | Aug 2026 | huggingface.co | Official leaderboard | |
| 78 | BUT-FIT/CSTinyLlama-1.2B 1.2B | 1.96 | bfloat16, 3-shot, max input 2048 | Sep 2024 | huggingface.co | Official leaderboard |
| # | Model | Score | Setting | Released | Run on | Link | Source |
|---|---|---|---|---|---|---|---|
| 1 | | 82.9 | 8bit, 3-shot, max input 2048 | Mar 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 2 | | 82.3 | bfloat16, 3-shot, max input 2048 | Jul 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 3 | | 81.0 | 8bit, 3-shot, max input 2048 | Jul 2026 | Aug 2026 | huggingface.co | Official leaderboard |
| 4 | | 80.6 | 8bit, 3-shot, max input 2048 | May 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 5 | | 77.2 | bfloat16, 3-shot, max input 2048 | Sep 2024 | Nov 2024 | huggingface.co | Official leaderboard |
| 6 | | 77.0 | 8bit, 3-shot, max input 2048 | Mar 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 7 | | 72.7 | bfloat16, 3-shot, max input 2048 | Nov 2024 | Jan 2025 | huggingface.co | Official leaderboard |
| 8 | | 72.6 | 8bit, 3-shot, max input 2048 | Apr 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 9 | | 71.8 | bfloat16, 3-shot, max input 2048, chat template, multiturn | Apr 2026 | May 2026 | huggingface.co | Official leaderboard |
| 10 | | 70.8 | bfloat16, 3-shot, max input 2048 | Dec 2024 | Dec 2024 | huggingface.co | Official leaderboard |
| 11 | | 70.3 | 8bit, 3-shot, max input 2048 | Jun 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 12 | | 70.2 | 8bit, 3-shot, max input 2048 | Apr 2025 | Jul 2025 | huggingface.co | Official leaderboard |
| 13 | | 69.7 | bfloat16, 3-shot, max input 2048 | Apr 2026 | Aug 2026 | huggingface.co | Official leaderboard |
| 14 | | 68.4 | 8bit, 3-shot, max input 2048 | Apr 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 15 | | 66.0 | 8bit, 3-shot, max input 2048 | Apr 2026 | Aug 2026 | huggingface.co | Official leaderboard |
| 16 | | 65.4 | bfloat16, 3-shot, max input 2048, chat template, multiturn | Aug 2026 | Oct 2026 | huggingface.co | Official leaderboard |
| 17 | | 64.2 | bfloat16, 3-shot, max input 2048 | Jul 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 18 | | 61.4 | bfloat16, 3-shot, max input 2048 | Jun 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 19 | | 61.0 | bfloat16, 3-shot, max input 2048 | Apr 2026 | May 2026 | huggingface.co | Official leaderboard |
| 20 | | 57.2 | bfloat16, 3-shot, max input 2048 | Oct 2024 | Dec 2024 | huggingface.co | Official leaderboard |
| 21 | | 56.6 | bfloat16, 3-shot, max input 2048 | Mar 2025 | May 2025 | huggingface.co | Official leaderboard |
| 22 | | 52.7 | bfloat16, 3-shot, max input 2048 | Dec 2024 | Jan 2025 | huggingface.co | Official leaderboard |
| 23 | | 46.4 | bfloat16, 3-shot, max input 2048 | Sep 2024 | Nov 2024 | huggingface.co | Official leaderboard |
| 24 | | 44.1 | bfloat16, 3-shot, max input 2048, chat template, multiturn | Mar 2026 | May 2026 | huggingface.co | Official leaderboard |
| 25 | | 40.6 | bfloat16, 3-shot, max input 2048 | Sep 2025 | Aug 2026 | huggingface.co | Official leaderboard |
| 26 | | 36.7 | bfloat16, 3-shot, max input 2048 | Oct 2024 | Dec 2024 | huggingface.co | Official leaderboard |
| 27 | | 35.9 | bfloat16, 3-shot, max input 2048 | Jun 2024 | Dec 2024 | huggingface.co | Official leaderboard |
| 28 | | 35.3 | bfloat16, 3-shot, max input 2048 | Jun 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 29 | | 34.6 | bfloat16, 3-shot, max input 1792 | Jun 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 30 | | 33.8 | bfloat16, 3-shot, max input 2048 | Jun 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 31 | | 33.0 | bfloat16, 3-shot, max input 2048 | Jul 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 32 | | 30.5 | bfloat16, 3-shot, max input 2048, chat template, multiturn | Mar 2026 | May 2026 | huggingface.co | Official leaderboard |
| 33 | Granite-3.0-8B-Base | 29.6 | bfloat16, 3-shot, max input 2048 | Oct 2024 | Nov 2024 | huggingface.co | Official leaderboard |
| 34 | Granite-3.0-8B-Instruct | 22.5 | bfloat16, 3-shot, max input 2048 | Oct 2024 | Nov 2024 | huggingface.co | Official leaderboard |
| 35 | | 20.7 | bfloat16, 3-shot, max input 2048 | Sep 2024 | Oct 2024 | huggingface.co | Official leaderboard |
| 36 | | 19.0 | bfloat16, 3-shot, max input 2048 | Jul 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 37 | | 10.5 | bfloat16, 3-shot, max input 2048 | Apr 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 38 | | 8.02 | bfloat16, 3-shot, max input 2048 | Sep 2024 | Oct 2024 | huggingface.co | Official leaderboard |
| 39 | | 6.48 | bfloat16, 3-shot, max input 2048 | Feb 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 40 | | 6.09 | bfloat16, 3-shot, max input 2048 | May 2023 | Sep 2024 | huggingface.co | Official leaderboard |
Rankings by part
We read 7 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as Czech Language Understanding, Czech Math Reasoning and Factual Knowledge. DeepSeek V4 Flash 0731 leads 2 of them, and 5 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
Czech Language Understanding
40 models we track have a score here.
- 1
DeepSeek V4 Flash 0731 ↑2 92.5
- 2
Llama 3.1-405B 91.7
- 3
DeepSeek-V3 Mar 2025 ↓2 88.9
- 4
Gemma 4 31B IT ↑5 88.2
- 5
DeepSeek-R1 May 2025 ↓1 86.5
- 6
Cohere Command A 76.3
- 7
Llama 4 Maverick ↑1 76.3
- 8
Mistral Large 2 Nov 2024 ↓1 71.6
- 9
Llama 3.3 70B Instruct ↑1 71.6
- 10
Gemma 4 26B A4B ↑9 71.4
Czech Math Reasoning
40 models we track have a score here.
- 1
DeepSeek V4 Flash 0731 ↑2 93.6
- 2
DeepSeek-V3 Mar 2025 ↓1 92.6
- 3
Gemma 4 31B IT ↑6 90.7
- 4
DeepSeek-R1 May 2025 87.5
- 5
Llama 4 Maverick ↑3 86.9
- 6
Qwen2.5-72B ↓1 84.6
- 7
Llama 3.1-405B ↓5 76.6
- 8
Gemma 4 26B A4B ↑11 72.1
- 9
Qwen3.6 27B ↑4 71.2
- 10
MiniMax-M1-80k ↑1 69.6
Factual Knowledge
40 models we track have a score here.
- 1
DeepSeek-V3 Mar 2025 88.3
- 2
DeepSeek-R1 May 2025 ↑2 86.1
- 3
Llama 3.1-405B ↓1 77.1
- 4
DeepSeek V4 Flash 0731 ↓1 76.7
- 5
Gemma 4 31B IT ↑4 75.1
- 6
Llama 4 Maverick ↑2 73.1
- 7
Mistral Large 2 Nov 2024 72.3
- 8
MiniMax-M1-80k ↑3 71.4
- 9
Cohere Command A ↓3 69.6
- 10
Qwen2.5-72B ↓5 69.2
NER
40 models we track have a score here.
- 1
Muse Glimmer ↑15 98.1
- 2
Llama 3.1-405B 94.2
- 3
Llama 3.1-70B ↑14 93.6
- 4
Qwen2.5-72B ↑1 92.3
- 5
Llama 3.3 70B Instruct ↑5 87.2
- 6
Llama 4 Scout ↑8 87.2
- 7
DeepSeek-R1 May 2025 ↓3 86.5
- 8
DeepSeek V4 Flash 0731 ↓5 86.5
- 9
DeepSeek-V3 Mar 2025 ↓8 85.9
- 10
Cohere Command A ↓4 85.3
NLI
40 models we track have a score here.
- 1
Llama 3.1-405B ↑1 81.9
- 2
Qwen2.5-72B ↑3 79.5
- 3
DeepSeek V4 Flash 0731 79.5
- 4
Cogito v1 Preview 70B ↑8 78.8
- 5
Gemma 4 31B IT ↑4 77.4
- 6
Qwen3.6 27B ↑7 76.9
- 7
Cohere Command A ↓1 75.3
- 8
Mistral Large 2 Nov 2024 ↓1 75.2
- 9
Llama 3.3 70B Instruct ↑1 74.4
- 10
DeepSeek-R1 May 2025 ↓6 74.4
Reading Comprehension
40 models we track have a score here.
- 1
Qwen3.6 27B ↑12 87.6
- 2
Llama 3.1-405B 85.0
- 3
Qwen2.5-72B ↑2 80.8
- 4
Cohere Command A ↑2 77.8
- 5
MiniMax-M1-80k ↑6 74.8
- 6
Qwen 3.6 35B-A3B ↑9 74.4
- 7
DeepSeek-V3 Mar 2025 ↓6 73.9
- 8
Llama 4 Scout ↑6 73.1
- 9
DeepSeek-R1 May 2025 ↓5 71.8
- 10
Gemma 4 31B IT ↓1 71.4
Sentiment
40 models we track have a score here.
- 1
Cohere Command A ↑5 86.9
- 2
Cogito v1 Preview 70B ↑10 85.6
- 3
Gemma 3 27B ↑18 84.9
- 4
Mistral Large 2 Nov 2024 ↑3 79.8
- 5
Aya Expanse 32B ↑15 77.9
- 6
DeepSeek-V3 Mar 2025 ↓5 76.9
- 7
Llama 3.3 70B Instruct ↑3 75.3
- 8
Qwen2.5-72B ↓3 74.7
- 9
MiniMax-M1-80k ↑2 73.1
- 10
Aya Expanse 8B ↑16 72.4
2 models have held number one on the official leaderboard since its release in October 2024
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official BenCzechMark leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official BenCzechMark leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5%
- MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87%
- CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7%
- Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
- MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%
Data audit
All 9 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (78 entries).
- Pass Models scored 40 models we track, plus 38 other entries on its leaderboard.
- Pass Newest score The newest is from August 2026, for Muse Glimmer.
- Note Frontier models None of the top 10 has a score. The strongest tested is DeepSeek V4 Flash 0731 (#52).
- Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on huggingface.co: 41 rows for models we track and 38 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.