Benchmark Almanac

BenCzechMark

DeepSeek-V3 Mar 2025 82.9 leads since March 2025
Not in model cards

Models complete Czech language tasks spanning knowledge, comprehension, and reasoning.

The benchmark ranks models by statistically significant wins across tasks.

Released
October 2024
Kind
Knowledge in many languages
Level
University
Scored
Checked automatically
Answers
Prediction
Tasks
All public
Measures
Capability
Language
Czech
In and out
Text → Text
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
Strongest model tested
DeepSeek V4 Flash 0731 #52 on the capability index
Last model added
August 2026
Scores refreshed
November 20th, 2026
78
Entries on its own leaderboard, 40 models we track

DeepSeek-V3 leads with 82.9 among models released by August 2026 DeepSeek-V3 leads with 82.9 among models released by August 2026

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

The newest model with a score here came out in August 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

78 entries. Scroll sideways for the rest.

  1. 82.9 DeepSeek-V3
  2. 82.3 Llama 3.1-405B
  3. 81.0 DeepSeek V4 Flash 0731
  4. 80.6 DeepSeek-R1
  5. 77.2 Qwen2.5-72B
  6. 77.0 Cohere Command A
  7. 72.7 Mistral Large 2
  8. 72.6 Llama 4 Maverick
  9. 71.8 Gemma 4 31B IT
  10. 70.8 Llama 3.3 70B Instruct
  11. 70.8 anex.sh/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8
  12. 70.3 MiniMax-M1-80k
  13. 70.2 Cogito v1 Preview 70B
  14. 70.0 anex.sh/mistralai/Mistral-Small-4-119B-2603-NVFP4
  15. 69.7 Qwen3.6 27B
  16. 68.4 Llama 4 Scout
  17. 68.3 meta-llama/Llama-3.1-70B-Instruct
  18. 66.0 Qwen 3.6 35B-A3B
  19. 65.6 CRA/nvidia/Qwen3.6-35B-A3B-NVFP4
  20. 65.4 Muse Glimmer
  21. 64.6 Qwen/Qwen2.5-32B-Instruct
  22. 64.2 Llama 3.1-70B
  23. 63.6 CRA/unsloth/Qwen3.8-27B-NVFP4
  24. 62.6 anex.sh/Qwen/Qwen3.8-27B-FP8
  25. 61.4 Qwen2-72B
  26. 61.0 Gemma 4 26B A4B
  27. 57.8 mistralai/Mixtral-8x22B-Instruct-v0.1
  28. 57.2 Aya Expanse 32B
  29. 56.6 Gemma 3 27B
  30. 54.4 AMead10/c4ai-command-r-08-2024-awq
  31. 52.7 Phi-4
  32. 50.5 google/gemma-2-27b-it
  33. 46.4 Qwen2.5-7B
  34. 44.4 Qwen/Qwen2.5-7B-Instruct
  35. 44.1 Gemma 4 E4B
  36. 42.6 speakleash/Bielik-11B-v2.3-Instruct
  37. 42.5 CohereForAI/aya-23-35B
  38. 42.0 meta-llama/Llama-3.1-8B-Instruct
  39. 41.8 mistralai/Mistral-Nemo-Instruct-2407
  40. 40.6 Apertus 8B Instruct
  41. 40.5 mistralai/Mixtral-8x7B-Instruct-v0.1
  42. 37.3 NousResearch/Hermes-3-Llama-3.1-8B
  43. 36.7 Aya Expanse 8B
  44. 36.3 meta-llama/Llama-3-8B-Instruct
  45. 35.9 Gemma 2 27B
  46. 35.3 Qwen2-7B
  47. 34.6 Qwen2-7B-Instruct
  48. 34.6 mistralai/Ministral-8B-Instruct-2410
  49. 33.8 Gemma 2 9B
  50. 33.0 Llama 3.1-8B
  51. 32.4 google/gemma-2-9b-it
  52. 30.5 Gemma 4 E2B
  53. 29.6 Granite-3.0-8B-Base
  54. 28.9 utter-project/EuroLLM-22B-Instruct-Preview
  55. 28.6 CohereForAI/aya-23-8B
  56. 25.7 mistralai/Mistral-7B-Instruct-v0.3
  57. 24.0 meta-llama/Llama-3.2-3B-Instruct
  58. 23.8 BSC-LT/ALIA-40b
  59. 22.5 Granite-3.0-8B-Instruct
  60. 21.2 utter-project/EuroLLM-9B-Instruct
  61. 20.7 Llama 3.2 3B
  62. 19.2 BSC-LT/salamandra-7b-instruct
  63. 19.0 Gemma 2 2B IT
  64. 18.5 internlm/internlm2_5-7b-chat
  65. 17.8 utter-project/EuroLLM-9B
  66. 15.8 ibm-granite/granite-3.0-3b-a800m-instruct
  67. 14.3 ibm-granite/granite-3.0-3b-a800m-base
  68. 10.5 phi-3-mini 3.8B
  69. 8.67 meta-llama/Llama-3.2-1B-Instruct
  70. 8.12 google/gemma-2-2b
  71. 8.02 Llama 3.2 1B
  72. 6.48 OLMo-7B-Instruct
  73. 6.09 MPT-7B
  74. 5.55 BUT-FIT/csmpt7b
  75. 5.28 utter-project/EuroLLM-1.7B-Instruct
  76. 3.59 utter-project/EuroLLM-1.7B
  77. 3.53 anex.sh/LiquidAI/LFM2-24B-A2B
  78. 1.96 BUT-FIT/CSTinyLlama-1.2B
  1. 1 DeepSeek-V3 Mar 2025, 8bit, 3-shot, max input 2048 82.9
  2. 2 Llama 3.1-405B bfloat16, 3-shot, max input 2048 82.3
  3. 3 DeepSeek V4 Flash 0731 8bit, 3-shot, max input 2048 81.0
  4. 4 DeepSeek-R1 May 2025, 8bit, 3-shot, max input 2048 80.6
  5. 5 Qwen2.5-72B bfloat16, 3-shot, max input 2048 77.2
  6. 6 Cohere Command A 8bit, 3-shot, max input 2048 77.0
  7. 7 Mistral Large 2 Nov 2024, bfloat16, 3-shot, max input 2048 72.7
  8. 8 Llama 4 Maverick 8bit, 3-shot, max input 2048 72.6
  9. 9 Gemma 4 31B IT bfloat16, 3-shot, max input 2048, chat template, multiturn 71.8
  10. 10 Llama 3.3 70B Instruct bfloat16, 3-shot, max input 2048 70.8
  11. 11 anex.sh/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 8bit, 3-shot, max input 2048 70.8
  12. 12 MiniMax-M1-80k 8bit, 3-shot, max input 2048 70.3
  13. 13 Cogito v1 Preview 70B 8bit, 3-shot, max input 2048 70.2
  14. 14 anex.sh/mistralai/Mistral-Small-4-119B-2603-NVFP4 4bit, 3-shot, max input 2048 70.0
  15. 15 Qwen3.6 27B bfloat16, 3-shot, max input 2048 69.7
  16. 16 Llama 4 Scout 8bit, 3-shot, max input 2048 68.4
  17. 17 meta-llama/Llama-3.1-70B-Instruct bfloat16, 3-shot, max input 2048 68.3
  18. 18 Qwen 3.6 35B-A3B 8bit, 3-shot, max input 2048 66.0
  19. 19 CRA/nvidia/Qwen3.6-35B-A3B-NVFP4 4bit, 3-shot, max input 2048 65.6
  20. 20 Muse Glimmer bfloat16, 3-shot, max input 2048, chat template, multiturn 65.4
  21. 21 Qwen/Qwen2.5-32B-Instruct bfloat16, 3-shot, max input 2048 64.6
  22. 22 Llama 3.1-70B bfloat16, 3-shot, max input 2048 64.2
  23. 23 CRA/unsloth/Qwen3.8-27B-NVFP4 4bit, 3-shot, max input 2048 63.6
  24. 24 anex.sh/Qwen/Qwen3.8-27B-FP8 8bit, 3-shot, max input 2048 62.6
  25. 25 Qwen2-72B bfloat16, 3-shot, max input 2048 61.4
  26. 26 Gemma 4 26B A4B bfloat16, 3-shot, max input 2048 61.0
  27. 27 mistralai/Mixtral-8x22B-Instruct-v0.1 bfloat16, 3-shot, max input 2048 57.8
  28. 28 Aya Expanse 32B bfloat16, 3-shot, max input 2048 57.2
  29. 29 Gemma 3 27B bfloat16, 3-shot, max input 2048 56.6
  30. 30 AMead10/c4ai-command-r-08-2024-awq 4bit, 3-shot, max input 2048 54.4
  31. 31 Phi-4 bfloat16, 3-shot, max input 2048 52.7
  32. 32 google/gemma-2-27b-it bfloat16, 3-shot, max input 2048 50.5
  33. 33 Qwen2.5-7B bfloat16, 3-shot, max input 2048 46.4
  34. 34 Qwen/Qwen2.5-7B-Instruct bfloat16, 3-shot, max input 2048 44.4
  35. 35 Gemma 4 E4B bfloat16, 3-shot, max input 2048, chat template, multiturn 44.1
  36. 36 speakleash/Bielik-11B-v2.3-Instruct bfloat16, 3-shot, max input 2048 42.6
  37. 37 CohereForAI/aya-23-35B bfloat16, 3-shot, max input 2048 42.5
  38. 38 meta-llama/Llama-3.1-8B-Instruct bfloat16, 3-shot, max input 1792 42.0
  39. 39 mistralai/Mistral-Nemo-Instruct-2407 bfloat16, 3-shot, max input 2048 41.8
  40. 40 Apertus 8B Instruct bfloat16, 3-shot, max input 2048 40.6
  41. 41 mistralai/Mixtral-8x7B-Instruct-v0.1 bfloat16, 3-shot, max input 2048 40.5
  42. 42 NousResearch/Hermes-3-Llama-3.1-8B bfloat16, 3-shot, max input 2048 37.3
  43. 43 Aya Expanse 8B bfloat16, 3-shot, max input 2048 36.7
  44. 44 meta-llama/Llama-3-8B-Instruct bfloat16, 3-shot, max input 2048 36.3
  45. 45 Gemma 2 27B bfloat16, 3-shot, max input 2048 35.9
  46. 46 Qwen2-7B bfloat16, 3-shot, max input 2048 35.3
  47. 47 Qwen2-7B-Instruct bfloat16, 3-shot, max input 1792 34.6
  48. 48 mistralai/Ministral-8B-Instruct-2410 bfloat16, 3-shot, max input 2048 34.6
  49. 49 Gemma 2 9B bfloat16, 3-shot, max input 2048 33.8
  50. 50 Llama 3.1-8B bfloat16, 3-shot, max input 2048 33.0
  51. 51 google/gemma-2-9b-it bfloat16, 3-shot, max input 2048 32.4
  52. 52 Gemma 4 E2B bfloat16, 3-shot, max input 2048, chat template, multiturn 30.5
  53. 53 Granite-3.0-8B-Base bfloat16, 3-shot, max input 2048 29.6
  54. 54 utter-project/EuroLLM-22B-Instruct-Preview 8bit, 3-shot, max input 2048 28.9
  55. 55 CohereForAI/aya-23-8B bfloat16, 3-shot, max input 2048 28.6
  56. 56 mistralai/Mistral-7B-Instruct-v0.3 bfloat16, 3-shot, max input 2048 25.7
  57. 57 meta-llama/Llama-3.2-3B-Instruct bfloat16, 3-shot, max input 2048 24.0
  58. 58 BSC-LT/ALIA-40b bfloat16, 3-shot, max input 2048 23.8
  59. 59 Granite-3.0-8B-Instruct bfloat16, 3-shot, max input 2048 22.5
  60. 60 utter-project/EuroLLM-9B-Instruct bfloat16, 3-shot, max input 2048 21.2
  61. 61 Llama 3.2 3B bfloat16, 3-shot, max input 2048 20.7
  62. 62 BSC-LT/salamandra-7b-instruct bfloat16, 3-shot, max input 2048 19.2
  63. 63 Gemma 2 2B IT bfloat16, 3-shot, max input 2048 19.0
  64. 64 internlm/internlm2_5-7b-chat bfloat16, 3-shot, max input 2048 18.5
  65. 65 utter-project/EuroLLM-9B bfloat16, 3-shot, max input 2048 17.8
  66. 66 ibm-granite/granite-3.0-3b-a800m-instruct bfloat16, 3-shot, max input 2048 15.8
  67. 67 ibm-granite/granite-3.0-3b-a800m-base bfloat16, 3-shot, max input 2048 14.3
  68. 68 phi-3-mini 3.8B bfloat16, 3-shot, max input 2048 10.5
  69. 69 meta-llama/Llama-3.2-1B-Instruct bfloat16, 3-shot, max input 2048 8.67
  70. 70 google/gemma-2-2b bfloat16, 3-shot, max input 2048 8.12
  71. 71 Llama 3.2 1B bfloat16, 3-shot, max input 2048 8.02
  72. 72 OLMo-7B-Instruct bfloat16, 3-shot, max input 2048 6.48
  73. 73 MPT-7B bfloat16, 3-shot, max input 2048 6.09
  74. 74 BUT-FIT/csmpt7b bfloat16, 3-shot, max input 2048 5.55
  75. 75 utter-project/EuroLLM-1.7B-Instruct bfloat16, 3-shot, max input 2048 5.28
  76. 76 utter-project/EuroLLM-1.7B bfloat16, 3-shot, max input 2048 3.59
  77. 77 anex.sh/LiquidAI/LFM2-24B-A2B bfloat16, 3-shot, max input 2048 3.53
  78. 78 BUT-FIT/CSTinyLlama-1.2B bfloat16, 3-shot, max input 2048 1.96

Source: Official BenCzechMark leaderboard

  1. 1 DeepSeek-V3 Mar 2025, 8bit, 3-shot, max input 2048 82.9
  2. 2 Llama 3.1-405B bfloat16, 3-shot, max input 2048 82.3
  3. 3 DeepSeek V4 Flash 0731 8bit, 3-shot, max input 2048 81.0
  4. 4 DeepSeek-R1 May 2025, 8bit, 3-shot, max input 2048 80.6
  5. 5 Qwen2.5-72B bfloat16, 3-shot, max input 2048 77.2
  6. 6 Cohere Command A 8bit, 3-shot, max input 2048 77.0
  7. 7 Mistral Large 2 Nov 2024, bfloat16, 3-shot, max input 2048 72.7
  8. 8 Llama 4 Maverick 8bit, 3-shot, max input 2048 72.6
  9. 9 Gemma 4 31B IT bfloat16, 3-shot, max input 2048, chat template, multiturn 71.8
  10. 10 Llama 3.3 70B Instruct bfloat16, 3-shot, max input 2048 70.8
  11. 11 MiniMax-M1-80k 8bit, 3-shot, max input 2048 70.3
  12. 12 Cogito v1 Preview 70B 8bit, 3-shot, max input 2048 70.2
  13. 13 Qwen3.6 27B bfloat16, 3-shot, max input 2048 69.7
  14. 14 Llama 4 Scout 8bit, 3-shot, max input 2048 68.4
  15. 15 Qwen 3.6 35B-A3B 8bit, 3-shot, max input 2048 66.0
  16. 16 Muse Glimmer bfloat16, 3-shot, max input 2048, chat template, multiturn 65.4
  17. 17 Llama 3.1-70B bfloat16, 3-shot, max input 2048 64.2
  18. 18 Qwen2-72B bfloat16, 3-shot, max input 2048 61.4
  19. 19 Gemma 4 26B A4B bfloat16, 3-shot, max input 2048 61.0
  20. 20 Aya Expanse 32B bfloat16, 3-shot, max input 2048 57.2
  21. 21 Gemma 3 27B bfloat16, 3-shot, max input 2048 56.6
  22. 22 Phi-4 bfloat16, 3-shot, max input 2048 52.7
  23. 23 Qwen2.5-7B bfloat16, 3-shot, max input 2048 46.4
  24. 24 Gemma 4 E4B bfloat16, 3-shot, max input 2048, chat template, multiturn 44.1
  25. 25 Apertus 8B Instruct bfloat16, 3-shot, max input 2048 40.6
  26. 26 Aya Expanse 8B bfloat16, 3-shot, max input 2048 36.7
  27. 27 Gemma 2 27B bfloat16, 3-shot, max input 2048 35.9
  28. 28 Qwen2-7B bfloat16, 3-shot, max input 2048 35.3
  29. 29 Qwen2-7B-Instruct bfloat16, 3-shot, max input 1792 34.6
  30. 30 Gemma 2 9B bfloat16, 3-shot, max input 2048 33.8
  31. 31 Llama 3.1-8B bfloat16, 3-shot, max input 2048 33.0
  32. 32 Gemma 4 E2B bfloat16, 3-shot, max input 2048, chat template, multiturn 30.5
  33. 33 Granite-3.0-8B-Base bfloat16, 3-shot, max input 2048 29.6
  34. 34 Granite-3.0-8B-Instruct bfloat16, 3-shot, max input 2048 22.5
  35. 35 Llama 3.2 3B bfloat16, 3-shot, max input 2048 20.7
  36. 36 Gemma 2 2B IT bfloat16, 3-shot, max input 2048 19.0
  37. 37 phi-3-mini 3.8B bfloat16, 3-shot, max input 2048 10.5
  38. 38 Llama 3.2 1B bfloat16, 3-shot, max input 2048 8.02
  39. 39 OLMo-7B-Instruct bfloat16, 3-shot, max input 2048 6.48
  40. 40 MPT-7B bfloat16, 3-shot, max input 2048 6.09

Source: Official BenCzechMark leaderboard

#EntryScoreSettingReleasedRun onLinkSource
1 DeepSeek-V3 Mar 2025 82.9 8bit, 3-shot, max input 2048 Mar 2025 Jul 2025 huggingface.co Official leaderboard
2 Llama 3.1-405B 82.3 bfloat16, 3-shot, max input 2048 Jul 2024 Sep 2024 huggingface.co Official leaderboard
3 DeepSeek V4 Flash 0731 81.0 8bit, 3-shot, max input 2048 Jul 2026 Aug 2026 huggingface.co Official leaderboard
4 DeepSeek-R1 May 2025 80.6 8bit, 3-shot, max input 2048 May 2025 Jul 2025 huggingface.co Official leaderboard
5 Qwen2.5-72B 77.2 bfloat16, 3-shot, max input 2048 Sep 2024 Nov 2024 huggingface.co Official leaderboard
6 Cohere Command A 77.0 8bit, 3-shot, max input 2048 Mar 2025 Jul 2025 huggingface.co Official leaderboard
7 Mistral Large 2 Nov 2024 72.7 bfloat16, 3-shot, max input 2048 Nov 2024 Jan 2025 huggingface.co Official leaderboard
8 Llama 4 Maverick 72.6 8bit, 3-shot, max input 2048 Apr 2025 Jul 2025 huggingface.co Official leaderboard
9 Gemma 4 31B IT 71.8 bfloat16, 3-shot, max input 2048, chat template, multiturn Apr 2026 May 2026 huggingface.co Official leaderboard
10 Llama 3.3 70B Instruct 70.8 bfloat16, 3-shot, max input 2048 Dec 2024 Dec 2024 huggingface.co Official leaderboard
11 anex.sh/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 27.8B 70.8 8bit, 3-shot, max input 2048 Aug 2026 huggingface.co Official leaderboard
12 MiniMax-M1-80k 70.3 8bit, 3-shot, max input 2048 Jun 2025 Jul 2025 huggingface.co Official leaderboard
13 Cogito v1 Preview 70B 70.2 8bit, 3-shot, max input 2048 Apr 2025 Jul 2025 huggingface.co Official leaderboard
14 anex.sh/mistralai/Mistral-Small-4-119B-2603-NVFP4 119B 70.0 4bit, 3-shot, max input 2048 Aug 2026 huggingface.co Official leaderboard
15 Qwen3.6 27B 69.7 bfloat16, 3-shot, max input 2048 Apr 2026 Aug 2026 huggingface.co Official leaderboard
16 Llama 4 Scout 68.4 8bit, 3-shot, max input 2048 Apr 2025 Jun 2025 huggingface.co Official leaderboard
17 meta-llama/Llama-3.1-70B-Instruct 70.6B 68.3 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
18 Qwen 3.6 35B-A3B 66.0 8bit, 3-shot, max input 2048 Apr 2026 Aug 2026 huggingface.co Official leaderboard
19 CRA/nvidia/Qwen3.6-35B-A3B-NVFP4 27.8B 65.6 4bit, 3-shot, max input 2048 Aug 2026 huggingface.co Official leaderboard
20 Muse Glimmer 65.4 bfloat16, 3-shot, max input 2048, chat template, multiturn Aug 2026 Oct 2026 huggingface.co Official leaderboard
21 Qwen/Qwen2.5-32B-Instruct 32.8B 64.6 bfloat16, 3-shot, max input 2048 Nov 2024 huggingface.co Official leaderboard
22 Llama 3.1-70B 64.2 bfloat16, 3-shot, max input 2048 Jul 2024 Sep 2024 huggingface.co Official leaderboard
23 CRA/unsloth/Qwen3.8-27B-NVFP4 27B 63.6 4bit, 3-shot, max input 2048 Sep 2026 huggingface.co Official leaderboard
24 anex.sh/Qwen/Qwen3.8-27B-FP8 27.8B 62.6 8bit, 3-shot, max input 2048 Aug 2026 huggingface.co Official leaderboard
25 Qwen2-72B 61.4 bfloat16, 3-shot, max input 2048 Jun 2024 Sep 2024 huggingface.co Official leaderboard
26 Gemma 4 26B A4B 61.0 bfloat16, 3-shot, max input 2048 Apr 2026 May 2026 huggingface.co Official leaderboard
27 mistralai/Mixtral-8x22B-Instruct-v0.1 141B 57.8 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
28 Aya Expanse 32B 57.2 bfloat16, 3-shot, max input 2048 Oct 2024 Dec 2024 huggingface.co Official leaderboard
29 Gemma 3 27B 56.6 bfloat16, 3-shot, max input 2048 Mar 2025 May 2025 huggingface.co Official leaderboard
30 AMead10/c4ai-command-r-08-2024-awq 32.3B 54.4 4bit, 3-shot, max input 2048 Oct 2024 huggingface.co Official leaderboard
31 Phi-4 52.7 bfloat16, 3-shot, max input 2048 Dec 2024 Jan 2025 huggingface.co Official leaderboard
32 google/gemma-2-27b-it 27.2B 50.5 bfloat16, 3-shot, max input 2048 Dec 2024 huggingface.co Official leaderboard
33 Qwen2.5-7B 46.4 bfloat16, 3-shot, max input 2048 Sep 2024 Nov 2024 huggingface.co Official leaderboard
34 Qwen/Qwen2.5-7B-Instruct 7.6B 44.4 bfloat16, 3-shot, max input 2048 Nov 2024 huggingface.co Official leaderboard
35 Gemma 4 E4B 44.1 bfloat16, 3-shot, max input 2048, chat template, multiturn Mar 2026 May 2026 huggingface.co Official leaderboard
36 speakleash/Bielik-11B-v2.3-Instruct 11.2B 42.6 bfloat16, 3-shot, max input 2048 Oct 2024 huggingface.co Official leaderboard
37 CohereForAI/aya-23-35B 35B 42.5 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
38 meta-llama/Llama-3.1-8B-Instruct 8B 42.0 bfloat16, 3-shot, max input 1792 Sep 2024 huggingface.co Official leaderboard
39 mistralai/Mistral-Nemo-Instruct-2407 12.2B 41.8 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
40 Apertus 8B Instruct 40.6 bfloat16, 3-shot, max input 2048 Sep 2025 Aug 2026 huggingface.co Official leaderboard
41 mistralai/Mixtral-8x7B-Instruct-v0.1 46.7B 40.5 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
42 NousResearch/Hermes-3-Llama-3.1-8B 8B 37.3 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
43 Aya Expanse 8B 36.7 bfloat16, 3-shot, max input 2048 Oct 2024 Dec 2024 huggingface.co Official leaderboard
44 meta-llama/Llama-3-8B-Instruct 8B 36.3 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
45 Gemma 2 27B 35.9 bfloat16, 3-shot, max input 2048 Jun 2024 Dec 2024 huggingface.co Official leaderboard
46 Qwen2-7B 35.3 bfloat16, 3-shot, max input 2048 Jun 2024 Sep 2024 huggingface.co Official leaderboard
47 Qwen2-7B-Instruct 34.6 bfloat16, 3-shot, max input 1792 Jun 2024 Sep 2024 huggingface.co Official leaderboard
48 mistralai/Ministral-8B-Instruct-2410 8B 34.6 bfloat16, 3-shot, max input 2048 Dec 2024 huggingface.co Official leaderboard
49 Gemma 2 9B 33.8 bfloat16, 3-shot, max input 2048 Jun 2024 Sep 2024 huggingface.co Official leaderboard
50 Llama 3.1-8B 33.0 bfloat16, 3-shot, max input 2048 Jul 2024 Sep 2024 huggingface.co Official leaderboard
51 google/gemma-2-9b-it 9.2B 32.4 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
52 Gemma 4 E2B 30.5 bfloat16, 3-shot, max input 2048, chat template, multiturn Mar 2026 May 2026 huggingface.co Official leaderboard
53 Granite-3.0-8B-Base 29.6 bfloat16, 3-shot, max input 2048 Oct 2024 Nov 2024 huggingface.co Official leaderboard
54 utter-project/EuroLLM-22B-Instruct-Preview 22B 28.9 8bit, 3-shot, max input 2048 Jul 2025 huggingface.co Official leaderboard
55 CohereForAI/aya-23-8B 8B 28.6 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
56 mistralai/Mistral-7B-Instruct-v0.3 7.3B 25.7 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
57 meta-llama/Llama-3.2-3B-Instruct 3.2B 24.0 bfloat16, 3-shot, max input 2048 Oct 2024 huggingface.co Official leaderboard
58 BSC-LT/ALIA-40b 40.4B 23.8 bfloat16, 3-shot, max input 2048 Jan 2025 huggingface.co Official leaderboard
59 Granite-3.0-8B-Instruct 22.5 bfloat16, 3-shot, max input 2048 Oct 2024 Nov 2024 huggingface.co Official leaderboard
60 utter-project/EuroLLM-9B-Instruct 9.2B 21.2 bfloat16, 3-shot, max input 2048 Dec 2024 huggingface.co Official leaderboard
61 Llama 3.2 3B 20.7 bfloat16, 3-shot, max input 2048 Sep 2024 Oct 2024 huggingface.co Official leaderboard
62 BSC-LT/salamandra-7b-instruct 7.8B 19.2 bfloat16, 3-shot, max input 2048 Dec 2024 huggingface.co Official leaderboard
63 Gemma 2 2B IT 19.0 bfloat16, 3-shot, max input 2048 Jul 2024 Sep 2024 huggingface.co Official leaderboard
64 internlm/internlm2_5-7b-chat 7.7B 18.5 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
65 utter-project/EuroLLM-9B 9.2B 17.8 bfloat16, 3-shot, max input 2048 Dec 2024 huggingface.co Official leaderboard
66 ibm-granite/granite-3.0-3b-a800m-instruct 3.4B 15.8 bfloat16, 3-shot, max input 2048 Nov 2024 huggingface.co Official leaderboard
67 ibm-granite/granite-3.0-3b-a800m-base 3.4B 14.3 bfloat16, 3-shot, max input 2048 Nov 2024 huggingface.co Official leaderboard
68 phi-3-mini 3.8B 10.5 bfloat16, 3-shot, max input 2048 Apr 2024 Sep 2024 huggingface.co Official leaderboard
69 meta-llama/Llama-3.2-1B-Instruct 1.2B 8.67 bfloat16, 3-shot, max input 2048 Oct 2024 huggingface.co Official leaderboard
70 google/gemma-2-2b 2.6B 8.12 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
71 Llama 3.2 1B 8.02 bfloat16, 3-shot, max input 2048 Sep 2024 Oct 2024 huggingface.co Official leaderboard
72 OLMo-7B-Instruct 6.48 bfloat16, 3-shot, max input 2048 Feb 2024 Sep 2024 huggingface.co Official leaderboard
73 MPT-7B 6.09 bfloat16, 3-shot, max input 2048 May 2023 Sep 2024 huggingface.co Official leaderboard
74 BUT-FIT/csmpt7b 6.7B 5.55 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
75 utter-project/EuroLLM-1.7B-Instruct 1.7B 5.28 bfloat16, 3-shot, max input 2048 Nov 2024 huggingface.co Official leaderboard
76 utter-project/EuroLLM-1.7B 1.7B 3.59 bfloat16, 3-shot, max input 2048 Nov 2024 huggingface.co Official leaderboard
77 anex.sh/LiquidAI/LFM2-24B-A2B 23.8B 3.53 bfloat16, 3-shot, max input 2048 Aug 2026 huggingface.co Official leaderboard
78 BUT-FIT/CSTinyLlama-1.2B 1.2B 1.96 bfloat16, 3-shot, max input 2048 Sep 2024 huggingface.co Official leaderboard
#ModelScoreSettingReleasedRun onLinkSource
1 DeepSeek-V3 Mar 2025 82.9 8bit, 3-shot, max input 2048 Mar 2025 Jul 2025 huggingface.co Official leaderboard
2 Llama 3.1-405B 82.3 bfloat16, 3-shot, max input 2048 Jul 2024 Sep 2024 huggingface.co Official leaderboard
3 DeepSeek V4 Flash 0731 81.0 8bit, 3-shot, max input 2048 Jul 2026 Aug 2026 huggingface.co Official leaderboard
4 DeepSeek-R1 May 2025 80.6 8bit, 3-shot, max input 2048 May 2025 Jul 2025 huggingface.co Official leaderboard
5 Qwen2.5-72B 77.2 bfloat16, 3-shot, max input 2048 Sep 2024 Nov 2024 huggingface.co Official leaderboard
6 Cohere Command A 77.0 8bit, 3-shot, max input 2048 Mar 2025 Jul 2025 huggingface.co Official leaderboard
7 Mistral Large 2 Nov 2024 72.7 bfloat16, 3-shot, max input 2048 Nov 2024 Jan 2025 huggingface.co Official leaderboard
8 Llama 4 Maverick 72.6 8bit, 3-shot, max input 2048 Apr 2025 Jul 2025 huggingface.co Official leaderboard
9 Gemma 4 31B IT 71.8 bfloat16, 3-shot, max input 2048, chat template, multiturn Apr 2026 May 2026 huggingface.co Official leaderboard
10 Llama 3.3 70B Instruct 70.8 bfloat16, 3-shot, max input 2048 Dec 2024 Dec 2024 huggingface.co Official leaderboard
11 MiniMax-M1-80k 70.3 8bit, 3-shot, max input 2048 Jun 2025 Jul 2025 huggingface.co Official leaderboard
12 Cogito v1 Preview 70B 70.2 8bit, 3-shot, max input 2048 Apr 2025 Jul 2025 huggingface.co Official leaderboard
13 Qwen3.6 27B 69.7 bfloat16, 3-shot, max input 2048 Apr 2026 Aug 2026 huggingface.co Official leaderboard
14 Llama 4 Scout 68.4 8bit, 3-shot, max input 2048 Apr 2025 Jun 2025 huggingface.co Official leaderboard
15 Qwen 3.6 35B-A3B 66.0 8bit, 3-shot, max input 2048 Apr 2026 Aug 2026 huggingface.co Official leaderboard
16 Muse Glimmer 65.4 bfloat16, 3-shot, max input 2048, chat template, multiturn Aug 2026 Oct 2026 huggingface.co Official leaderboard
17 Llama 3.1-70B 64.2 bfloat16, 3-shot, max input 2048 Jul 2024 Sep 2024 huggingface.co Official leaderboard
18 Qwen2-72B 61.4 bfloat16, 3-shot, max input 2048 Jun 2024 Sep 2024 huggingface.co Official leaderboard
19 Gemma 4 26B A4B 61.0 bfloat16, 3-shot, max input 2048 Apr 2026 May 2026 huggingface.co Official leaderboard
20 Aya Expanse 32B 57.2 bfloat16, 3-shot, max input 2048 Oct 2024 Dec 2024 huggingface.co Official leaderboard
21 Gemma 3 27B 56.6 bfloat16, 3-shot, max input 2048 Mar 2025 May 2025 huggingface.co Official leaderboard
22 Phi-4 52.7 bfloat16, 3-shot, max input 2048 Dec 2024 Jan 2025 huggingface.co Official leaderboard
23 Qwen2.5-7B 46.4 bfloat16, 3-shot, max input 2048 Sep 2024 Nov 2024 huggingface.co Official leaderboard
24 Gemma 4 E4B 44.1 bfloat16, 3-shot, max input 2048, chat template, multiturn Mar 2026 May 2026 huggingface.co Official leaderboard
25 Apertus 8B Instruct 40.6 bfloat16, 3-shot, max input 2048 Sep 2025 Aug 2026 huggingface.co Official leaderboard
26 Aya Expanse 8B 36.7 bfloat16, 3-shot, max input 2048 Oct 2024 Dec 2024 huggingface.co Official leaderboard
27 Gemma 2 27B 35.9 bfloat16, 3-shot, max input 2048 Jun 2024 Dec 2024 huggingface.co Official leaderboard
28 Qwen2-7B 35.3 bfloat16, 3-shot, max input 2048 Jun 2024 Sep 2024 huggingface.co Official leaderboard
29 Qwen2-7B-Instruct 34.6 bfloat16, 3-shot, max input 1792 Jun 2024 Sep 2024 huggingface.co Official leaderboard
30 Gemma 2 9B 33.8 bfloat16, 3-shot, max input 2048 Jun 2024 Sep 2024 huggingface.co Official leaderboard
31 Llama 3.1-8B 33.0 bfloat16, 3-shot, max input 2048 Jul 2024 Sep 2024 huggingface.co Official leaderboard
32 Gemma 4 E2B 30.5 bfloat16, 3-shot, max input 2048, chat template, multiturn Mar 2026 May 2026 huggingface.co Official leaderboard
33 Granite-3.0-8B-Base 29.6 bfloat16, 3-shot, max input 2048 Oct 2024 Nov 2024 huggingface.co Official leaderboard
34 Granite-3.0-8B-Instruct 22.5 bfloat16, 3-shot, max input 2048 Oct 2024 Nov 2024 huggingface.co Official leaderboard
35 Llama 3.2 3B 20.7 bfloat16, 3-shot, max input 2048 Sep 2024 Oct 2024 huggingface.co Official leaderboard
36 Gemma 2 2B IT 19.0 bfloat16, 3-shot, max input 2048 Jul 2024 Sep 2024 huggingface.co Official leaderboard
37 phi-3-mini 3.8B 10.5 bfloat16, 3-shot, max input 2048 Apr 2024 Sep 2024 huggingface.co Official leaderboard
38 Llama 3.2 1B 8.02 bfloat16, 3-shot, max input 2048 Sep 2024 Oct 2024 huggingface.co Official leaderboard
39 OLMo-7B-Instruct 6.48 bfloat16, 3-shot, max input 2048 Feb 2024 Sep 2024 huggingface.co Official leaderboard
40 MPT-7B 6.09 bfloat16, 3-shot, max input 2048 May 2023 Sep 2024 huggingface.co Official leaderboard

Rankings by part

We read 7 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as Czech Language Understanding, Czech Math Reasoning and Factual Knowledge. DeepSeek V4 Flash 0731 leads 2 of them, and 5 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.

Czech Language Understanding

40 models we track have a score here.

  1. 1 DeepSeek V4 Flash 0731 ↑2 92.5
  2. 2 Llama 3.1-405B 91.7
  3. 3 DeepSeek-V3 Mar 2025 ↓2 88.9
  4. 4 Gemma 4 31B IT ↑5 88.2
  5. 5 DeepSeek-R1 May 2025 ↓1 86.5
  6. 6 Cohere Command A 76.3
  7. 7 Llama 4 Maverick ↑1 76.3
  8. 8 Mistral Large 2 Nov 2024 ↓1 71.6
  9. 9 Llama 3.3 70B Instruct ↑1 71.6
  10. 10 Gemma 4 26B A4B ↑9 71.4

2 models have held number one on the official leaderboard since its release in October 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Meta DeepSeek
Own leaderboard Not tested since Aug 2026

Source: Official BenCzechMark leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
0.00 20.0 40.0 60.0 80.0 100.0 2023 2024 2025 2026 0.00 20.0 40.0 60.0 80.0 100.0 2023 2024 2025 2026

Source: Official BenCzechMark leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Multilingual

All 545 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%

Data audit

All 9 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (78 entries).
  • Pass Models scored 40 models we track, plus 38 other entries on its leaderboard.
  • Pass Newest score The newest is from August 2026, for Muse Glimmer.
  • Note Frontier models None of the top 10 has a score. The strongest tested is DeepSeek V4 Flash 0731 (#52).
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on huggingface.co: 41 rows for models we track and 38 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check