Benchmark Almanac

LLMzSzŁ

Qwen2.5-72B 69.1% leads since January 2025
Not in model cards

Models answer Polish academic and professional exam questions drawn from the Central Examination Board archives.

The authors compare models on four kinds of exams spanning 154 domains.

Released
January 2025
Kind
Knowledge in many languages
Level
University
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Measures
Capability
Language
Polish
In and out
Text → Text
Interaction
Single turn
Items from
Exams and contests
Refresh
One fixed set
Run by
Anyone
Strongest model tested
Qwen3-32B #135 on the capability index
Last model added
April 2025
Scores refreshed
November 20th, 2026
98
Entries on its own leaderboard, 32 models we track

Qwen2.5-72B leads with 69.1% among models released by April 2025 Qwen2.5-72B leads with 69.1% among models released by April 2025

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

The newest model with a score here came out in April 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

98 entries. Scroll sideways for the rest.

  1. 69.1 Qwen2.5-72B
  2. 67.2 mistralai/Mistral-Large-Instruct-2407
  3. 67.1 Llama 3.3 70B Instruct
  4. 66.8 Qwen3-32B
  5. 66.6 mistralai/Mistral-Large-Instruct-2411
  6. 66.6 meta-llama/Meta-Llama-3.1-70B-Instruct
  7. 64.6 CYFRAGOVPL/Llama-PLLuM-70B-base
  8. 64.4 CYFRAGOVPL/Llama-PLLuM-70B-chat
  9. 64.0 meta-llama/Meta-Llama-3-70B-Instruct
  10. 63.5 Qwen3-30B-A3B
  11. 63.1 Llama 3.1-70B
  12. 62.2 Llama 3-70B
  13. 61.0 Qwen2.5-32B
  14. 61.0 Qwen3-14B
  15. 60.5 CYFRAGOVPL/PLLuM-8x7B-nc-chat
  16. 59.2 speakleash/Bielik-11B-v2.6-Instruct
  17. 58.9 speakleash/Bielik-11B-v2.5-Instruct
  18. 58.2 mistralai/Mixtral-8x22B-Instruct-v0.1
  19. 57.8 CYFRAGOVPL/PLLuM-8x7B-nc-base
  20. 57.5 speakleash/Bielik-11B-v2.1-Instruct
  21. 57.4 speakleash/Bielik-11B-v2.3-Instruct
  22. 57.4 speakleash/Bielik-11B-v2.2-Instruct
  23. 55.8 DeepSeek-R1-Distill-Qwen-32B
  24. 55.6 speakleash/Bielik-11B-v2.0-Instruct
  25. 55.3 Qwen2.5-14B
  26. 55.3 Qwen3-8B
  27. 55.1 speakleash/Bielik-11B-v2
  28. 54.8 speakleash/Bielik-4.5B-v3.0-Instruct
  29. 54.2 CYFRAGOVPL/PLLuM-12b-nc-chat-250715
  30. 53.4 CYFRAGOVPL/PLLuM-12B-nc-chat
  31. 52.8 CYFRAGOVPL/PLLuM-8x7B-chat
  32. 52.2 CYFRAGOVPL/PLLuM-8x7B-base
  33. 51.8 Phi-4
  34. 51.4 utter-project/EuroLLM-9B-Instruct
  35. 50.5 CYFRAGOVPL/PLLuM-12B-nc-base
  36. 50.2 Qwen3-4B
  37. 49.7 CYFRAGOVPL/PLLuM-12B-chat
  38. 49.5 mistralai/Mixtral-8x7B-Instruct-v0.1
  39. 47.7 Mixtral 8x7B
  40. 47.7 CYFRAGOVPL/Llama-PLLuM-8B-chat
  41. 47.4 meta-llama/Meta-Llama-3.1-8B-Instruct
  42. 46.8 Gemma 7B
  43. 46.8 speakleash/Bielik-1.5B-v3.0-Instruct
  44. 46.8 DeepSeek-R1-Distill-Qwen-14B
  45. 46.5 CYFRAGOVPL/PLLuM-12B-base
  46. 46.4 meta-llama/Llama-2-70b-hf
  47. 46.3 CYFRAGOVPL/Llama-PLLuM-8B-base
  48. 45.6 Qwen2-7B
  49. 45.5 mistralai/Mistral-Nemo-Instruct-2407
  50. 44.8 meta-llama/Meta-Llama-3-8B-Instruct
  51. 44.2 Llama 3.1-8B
  52. 44.2 OLMo 2 32B Instruct
  53. 42.3 mistralai/Ministral-8B-Instruct-2410
  54. 42.2 Mistral-Nemo-Base-2407
  55. 41.8 allenai/OLMo-2-0325-32B
  56. 41.7 mistralai/Mistral-7B-Instruct-v0.3
  57. 41.5 01-ai/Yi-1.5-34B-Chat
  58. 41.4 01-ai/Yi-34B-Chat
  59. 41.4 Llama 3-8B
  60. 41.1 Yi-1.5-34B
  61. 40.8 speakleash/Bielik-7B-Instruct-v0.1
  62. 40.8 Mistral-7B-Instruct-v0.2
  63. 40.3 01-ai/Yi-34B-Chat-4bits
  64. 40.2 Voicelab/trurl-2-13b-8bit
  65. 40.2 Voicelab/trurl-2-13b
  66. 39.9 Qwen3-1.7B
  67. 39.9 meta-llama/Llama-3.2-3B-Instruct
  68. 39.1 speakleash/Bielik-7B-v0.1
  69. 38.2 lucyknada/microsoft_WizardLM-2-7B
  70. 37.8 Mistral 7B v0.1
  71. 37.6 Yi-1.5-9B-Chat
  72. 37.6 01-ai/Yi-34B-200K
  73. 37.1 Mistral 7B v0.3
  74. 37.1 01-ai/Yi-1.5-9B
  75. 36.0 Mistral-7B-Instruct-v0.1
  76. 34.9 Voicelab/trurl-2-13b-academic
  77. 34.9 OPI-PG/Qra-13b
  78. 34.2 Qwen2-1.5B
  79. 33.9 meta-llama/Llama-2-13b-hf
  80. 33.4 phi-3-mini 3.8B
  81. 33.2 allenai/OLMo-2-1124-13B-Instruct
  82. 32.6 allenai/OLMo-2-1124-13B
  83. 32.3 Voicelab/trurl-2-7b
  84. 31.9 Voicelab/trurl-2-7b-8bit
  85. 31.6 meta-llama/Llama-3.2-1B-Instruct
  86. 31.5 allenai/OLMo-2-1124-7B-Instruct
  87. 31.1 allenai/OLMo-2-1124-7B
  88. 29.1 OPI-PG/Qra-7b
  89. 28.0 meta-llama/Llama-2-7b-hf
  90. 25.7 microsoft/phi-1
  91. 25.6 utter-project/EuroLLM-1.7B-Instruct
  92. 25.6 Phi-2
  93. 25.5 OPI-PG/Qra-1b
  94. 24.9 sdadas/polish-gpt2-large
  95. 24.4 sdadas/polish-gpt2-medium
  96. 24.3 Phi-1.5
  97. 24.2 sdadas/polish-gpt2-small
  98. 24.0 sdadas/polish-gpt2-xl
  1. 1 Qwen2.5-72B 69.1%
  2. 2 mistralai/Mistral-Large-Instruct-2407 67.2%
  3. 3 Llama 3.3 70B Instruct 67.1%
  4. 4 Qwen3-32B 66.8%
  5. 5 mistralai/Mistral-Large-Instruct-2411 66.6%
  6. 6 meta-llama/Meta-Llama-3.1-70B-Instruct 66.6%
  7. 7 CYFRAGOVPL/Llama-PLLuM-70B-base 64.6%
  8. 8 CYFRAGOVPL/Llama-PLLuM-70B-chat 64.4%
  9. 9 meta-llama/Meta-Llama-3-70B-Instruct 64%
  10. 10 Qwen3-30B-A3B 63.5%
  11. 11 Llama 3.1-70B 63.1%
  12. 12 Llama 3-70B 62.2%
  13. 13 Qwen2.5-32B 61%
  14. 14 Qwen3-14B 61%
  15. 15 CYFRAGOVPL/PLLuM-8x7B-nc-chat 60.5%
  16. 16 speakleash/Bielik-11B-v2.6-Instruct 59.2%
  17. 17 speakleash/Bielik-11B-v2.5-Instruct 58.9%
  18. 18 mistralai/Mixtral-8x22B-Instruct-v0.1 58.2%
  19. 19 CYFRAGOVPL/PLLuM-8x7B-nc-base 57.8%
  20. 20 speakleash/Bielik-11B-v2.1-Instruct 57.5%
  21. 21 speakleash/Bielik-11B-v2.3-Instruct 57.4%
  22. 22 speakleash/Bielik-11B-v2.2-Instruct 57.4%
  23. 23 DeepSeek-R1-Distill-Qwen-32B 55.8%
  24. 24 speakleash/Bielik-11B-v2.0-Instruct 55.6%
  25. 25 Qwen2.5-14B 55.3%
  26. 26 Qwen3-8B 55.3%
  27. 27 speakleash/Bielik-11B-v2 55.1%
  28. 28 speakleash/Bielik-4.5B-v3.0-Instruct 54.8%
  29. 29 CYFRAGOVPL/PLLuM-12b-nc-chat-250715 54.2%
  30. 30 CYFRAGOVPL/PLLuM-12B-nc-chat 53.4%
  31. 31 CYFRAGOVPL/PLLuM-8x7B-chat 52.8%
  32. 32 CYFRAGOVPL/PLLuM-8x7B-base 52.2%
  33. 33 Phi-4 51.8%
  34. 34 utter-project/EuroLLM-9B-Instruct 51.4%
  35. 35 CYFRAGOVPL/PLLuM-12B-nc-base 50.5%
  36. 36 Qwen3-4B 50.2%
  37. 37 CYFRAGOVPL/PLLuM-12B-chat 49.7%
  38. 38 mistralai/Mixtral-8x7B-Instruct-v0.1 49.5%
  39. 39 Mixtral 8x7B 47.7%
  40. 40 CYFRAGOVPL/Llama-PLLuM-8B-chat 47.7%
  41. 41 meta-llama/Meta-Llama-3.1-8B-Instruct 47.4%
  42. 42 Gemma 7B 46.8%
  43. 43 speakleash/Bielik-1.5B-v3.0-Instruct 46.8%
  44. 44 DeepSeek-R1-Distill-Qwen-14B 46.8%
  45. 45 CYFRAGOVPL/PLLuM-12B-base 46.5%
  46. 46 meta-llama/Llama-2-70b-hf 46.4%
  47. 47 CYFRAGOVPL/Llama-PLLuM-8B-base 46.3%
  48. 48 Qwen2-7B 45.6%
  49. 49 mistralai/Mistral-Nemo-Instruct-2407 45.5%
  50. 50 meta-llama/Meta-Llama-3-8B-Instruct 44.8%
  51. 51 Llama 3.1-8B 44.2%
  52. 52 OLMo 2 32B Instruct 44.2%
  53. 53 mistralai/Ministral-8B-Instruct-2410 42.3%
  54. 54 Mistral-Nemo-Base-2407 42.2%
  55. 55 allenai/OLMo-2-0325-32B 41.8%
  56. 56 mistralai/Mistral-7B-Instruct-v0.3 41.7%
  57. 57 01-ai/Yi-1.5-34B-Chat 41.5%
  58. 58 01-ai/Yi-34B-Chat 41.4%
  59. 59 Llama 3-8B 41.4%
  60. 60 Yi-1.5-34B 41.1%
  61. 61 speakleash/Bielik-7B-Instruct-v0.1 40.8%
  62. 62 Mistral-7B-Instruct-v0.2 40.8%
  63. 63 01-ai/Yi-34B-Chat-4bits 40.3%
  64. 64 Voicelab/trurl-2-13b-8bit 40.2%
  65. 65 Voicelab/trurl-2-13b 40.2%
  66. 66 Qwen3-1.7B 39.9%
  67. 67 meta-llama/Llama-3.2-3B-Instruct 39.9%
  68. 68 speakleash/Bielik-7B-v0.1 39.1%
  69. 69 lucyknada/microsoft_WizardLM-2-7B 38.2%
  70. 70 Mistral 7B v0.1 37.8%
  71. 71 Yi-1.5-9B-Chat 37.6%
  72. 72 01-ai/Yi-34B-200K 37.6%
  73. 73 Mistral 7B v0.3 37.1%
  74. 74 01-ai/Yi-1.5-9B 37.1%
  75. 75 Mistral-7B-Instruct-v0.1 36%
  76. 76 Voicelab/trurl-2-13b-academic 34.9%
  77. 77 OPI-PG/Qra-13b 34.9%
  78. 78 Qwen2-1.5B 34.2%
  79. 79 meta-llama/Llama-2-13b-hf 33.9%
  80. 80 phi-3-mini 3.8B 33.4%
  81. 81 allenai/OLMo-2-1124-13B-Instruct 33.2%
  82. 82 allenai/OLMo-2-1124-13B 32.6%
  83. 83 Voicelab/trurl-2-7b 32.3%
  84. 84 Voicelab/trurl-2-7b-8bit 31.9%
  85. 85 meta-llama/Llama-3.2-1B-Instruct 31.6%
  86. 86 allenai/OLMo-2-1124-7B-Instruct 31.5%
  87. 87 allenai/OLMo-2-1124-7B 31.1%
  88. 88 OPI-PG/Qra-7b 29.1%
  89. 89 meta-llama/Llama-2-7b-hf 28%
  90. 90 microsoft/phi-1 25.7%
  91. 91 utter-project/EuroLLM-1.7B-Instruct 25.6%
  92. 92 Phi-2 25.6%
  93. 93 OPI-PG/Qra-1b 25.5%
  94. 94 sdadas/polish-gpt2-large 24.9%
  95. 95 sdadas/polish-gpt2-medium 24.4%
  96. 96 Phi-1.5 24.3%
  97. 97 sdadas/polish-gpt2-small 24.2%
  98. 98 sdadas/polish-gpt2-xl 24%

Source: Official LLMzSzŁ leaderboard

#EntryScoreSettingDateSource
1 Qwen2.5-72B 69.1% Sep 2024 Official leaderboard
2 mistralai/Mistral-Large-Instruct-2407 67.2% Official leaderboard
3 Llama 3.3 70B Instruct 67.1% Dec 2024 Official leaderboard
4 Qwen3-32B 66.8% Apr 2025 Official leaderboard
5 mistralai/Mistral-Large-Instruct-2411 66.6% Official leaderboard
6 meta-llama/Meta-Llama-3.1-70B-Instruct 66.6% Official leaderboard
7 CYFRAGOVPL/Llama-PLLuM-70B-base 64.6% Official leaderboard
8 CYFRAGOVPL/Llama-PLLuM-70B-chat 64.4% Official leaderboard
9 meta-llama/Meta-Llama-3-70B-Instruct 64% Official leaderboard
10 Qwen3-30B-A3B 63.5% Apr 2025 Official leaderboard
11 Llama 3.1-70B 63.1% Jul 2024 Official leaderboard
12 Llama 3-70B 62.2% Apr 2024 Official leaderboard
13 Qwen2.5-32B 61% Sep 2024 Official leaderboard
14 Qwen3-14B 61% Apr 2025 Official leaderboard
15 CYFRAGOVPL/PLLuM-8x7B-nc-chat 60.5% Official leaderboard
16 speakleash/Bielik-11B-v2.6-Instruct 59.2% Official leaderboard
17 speakleash/Bielik-11B-v2.5-Instruct 58.9% Official leaderboard
18 mistralai/Mixtral-8x22B-Instruct-v0.1 58.2% Official leaderboard
19 CYFRAGOVPL/PLLuM-8x7B-nc-base 57.8% Official leaderboard
20 speakleash/Bielik-11B-v2.1-Instruct 57.5% Official leaderboard
21 speakleash/Bielik-11B-v2.3-Instruct 57.4% Official leaderboard
22 speakleash/Bielik-11B-v2.2-Instruct 57.4% Official leaderboard
23 DeepSeek-R1-Distill-Qwen-32B 55.8% Jan 2025 Official leaderboard
24 speakleash/Bielik-11B-v2.0-Instruct 55.6% Official leaderboard
25 Qwen2.5-14B 55.3% Sep 2024 Official leaderboard
26 Qwen3-8B 55.3% Apr 2025 Official leaderboard
27 speakleash/Bielik-11B-v2 55.1% Official leaderboard
28 speakleash/Bielik-4.5B-v3.0-Instruct 54.8% Official leaderboard
29 CYFRAGOVPL/PLLuM-12b-nc-chat-250715 54.2% Official leaderboard
30 CYFRAGOVPL/PLLuM-12B-nc-chat 53.4% Official leaderboard
31 CYFRAGOVPL/PLLuM-8x7B-chat 52.8% Official leaderboard
32 CYFRAGOVPL/PLLuM-8x7B-base 52.2% Official leaderboard
33 Phi-4 51.8% Dec 2024 Official leaderboard
34 utter-project/EuroLLM-9B-Instruct 51.4% Official leaderboard
35 CYFRAGOVPL/PLLuM-12B-nc-base 50.5% Official leaderboard
36 Qwen3-4B 50.2% Apr 2025 Official leaderboard
37 CYFRAGOVPL/PLLuM-12B-chat 49.7% Official leaderboard
38 mistralai/Mixtral-8x7B-Instruct-v0.1 49.5% Official leaderboard
39 Mixtral 8x7B 47.7% Dec 2023 Official leaderboard
40 CYFRAGOVPL/Llama-PLLuM-8B-chat 47.7% Official leaderboard
41 meta-llama/Meta-Llama-3.1-8B-Instruct 47.4% Official leaderboard
42 Gemma 7B 46.8% Feb 2024 Official leaderboard
43 speakleash/Bielik-1.5B-v3.0-Instruct 46.8% Official leaderboard
44 DeepSeek-R1-Distill-Qwen-14B 46.8% Jan 2025 Official leaderboard
45 CYFRAGOVPL/PLLuM-12B-base 46.5% Official leaderboard
46 meta-llama/Llama-2-70b-hf 46.4% Official leaderboard
47 CYFRAGOVPL/Llama-PLLuM-8B-base 46.3% Official leaderboard
48 Qwen2-7B 45.6% Jun 2024 Official leaderboard
49 mistralai/Mistral-Nemo-Instruct-2407 45.5% Official leaderboard
50 meta-llama/Meta-Llama-3-8B-Instruct 44.8% Official leaderboard
51 Llama 3.1-8B 44.2% Jul 2024 Official leaderboard
52 OLMo 2 32B Instruct 44.2% Mar 2025 Official leaderboard
53 mistralai/Ministral-8B-Instruct-2410 42.3% Official leaderboard
54 Mistral-Nemo-Base-2407 42.2% Jul 2024 Official leaderboard
55 allenai/OLMo-2-0325-32B 41.8% Official leaderboard
56 mistralai/Mistral-7B-Instruct-v0.3 41.7% Official leaderboard
57 01-ai/Yi-1.5-34B-Chat 41.5% Official leaderboard
58 01-ai/Yi-34B-Chat 41.4% Official leaderboard
59 Llama 3-8B 41.4% Apr 2024 Official leaderboard
60 Yi-1.5-34B 41.1% May 2024 Official leaderboard
61 speakleash/Bielik-7B-Instruct-v0.1 40.8% Official leaderboard
62 Mistral-7B-Instruct-v0.2 40.8% Dec 2023 Official leaderboard
63 01-ai/Yi-34B-Chat-4bits 40.3% Official leaderboard
64 Voicelab/trurl-2-13b-8bit 40.2% Official leaderboard
65 Voicelab/trurl-2-13b 40.2% Official leaderboard
66 Qwen3-1.7B 39.9% Apr 2025 Official leaderboard
67 meta-llama/Llama-3.2-3B-Instruct 39.9% Official leaderboard
68 speakleash/Bielik-7B-v0.1 39.1% Official leaderboard
69 lucyknada/microsoft_WizardLM-2-7B 38.2% Official leaderboard
70 Mistral 7B v0.1 37.8% Sep 2023 Official leaderboard
71 Yi-1.5-9B-Chat 37.6% May 2024 Official leaderboard
72 01-ai/Yi-34B-200K 37.6% Official leaderboard
73 Mistral 7B v0.3 37.1% Sep 2023 Official leaderboard
74 01-ai/Yi-1.5-9B 37.1% Official leaderboard
75 Mistral-7B-Instruct-v0.1 36% Sep 2023 Official leaderboard
76 Voicelab/trurl-2-13b-academic 34.9% Official leaderboard
77 OPI-PG/Qra-13b 34.9% Official leaderboard
78 Qwen2-1.5B 34.2% Jun 2024 Official leaderboard
79 meta-llama/Llama-2-13b-hf 33.9% Official leaderboard
80 phi-3-mini 3.8B 33.4% Apr 2024 Official leaderboard
81 allenai/OLMo-2-1124-13B-Instruct 33.2% Official leaderboard
82 allenai/OLMo-2-1124-13B 32.6% Official leaderboard
83 Voicelab/trurl-2-7b 32.3% Official leaderboard
84 Voicelab/trurl-2-7b-8bit 31.9% Official leaderboard
85 meta-llama/Llama-3.2-1B-Instruct 31.6% Official leaderboard
86 allenai/OLMo-2-1124-7B-Instruct 31.5% Official leaderboard
87 allenai/OLMo-2-1124-7B 31.1% Official leaderboard
88 OPI-PG/Qra-7b 29.1% Official leaderboard
89 meta-llama/Llama-2-7b-hf 28% Official leaderboard
90 microsoft/phi-1 25.7% Official leaderboard
91 utter-project/EuroLLM-1.7B-Instruct 25.6% Official leaderboard
92 Phi-2 25.6% Dec 2023 Official leaderboard
93 OPI-PG/Qra-1b 25.5% Official leaderboard
94 sdadas/polish-gpt2-large 24.9% Official leaderboard
95 sdadas/polish-gpt2-medium 24.4% Official leaderboard
96 Phi-1.5 24.3% Sep 2023 Official leaderboard
97 sdadas/polish-gpt2-small 24.2% Official leaderboard
98 sdadas/polish-gpt2-xl 24% Official leaderboard
#ModelScoreSettingDateSource
1 Qwen2.5-72B 69.1% Sep 2024 Official leaderboard
2 Llama 3.3 70B Instruct 67.1% Dec 2024 Official leaderboard
3 Qwen3-32B 66.8% Apr 2025 Official leaderboard
4 Qwen3-30B-A3B 63.5% Apr 2025 Official leaderboard
5 Llama 3.1-70B 63.1% Jul 2024 Official leaderboard
6 Llama 3-70B 62.2% Apr 2024 Official leaderboard
7 Qwen2.5-32B 61% Sep 2024 Official leaderboard
8 Qwen3-14B 61% Apr 2025 Official leaderboard
9 DeepSeek-R1-Distill-Qwen-32B 55.8% Jan 2025 Official leaderboard
10 Qwen2.5-14B 55.3% Sep 2024 Official leaderboard
11 Qwen3-8B 55.3% Apr 2025 Official leaderboard
12 Phi-4 51.8% Dec 2024 Official leaderboard
13 Qwen3-4B 50.2% Apr 2025 Official leaderboard
14 Mixtral 8x7B 47.7% Dec 2023 Official leaderboard
15 Gemma 7B 46.8% Feb 2024 Official leaderboard
16 DeepSeek-R1-Distill-Qwen-14B 46.8% Jan 2025 Official leaderboard
17 Qwen2-7B 45.6% Jun 2024 Official leaderboard
18 Llama 3.1-8B 44.2% Jul 2024 Official leaderboard
19 OLMo 2 32B Instruct 44.2% Mar 2025 Official leaderboard
20 Mistral-Nemo-Base-2407 42.2% Jul 2024 Official leaderboard
21 Llama 3-8B 41.4% Apr 2024 Official leaderboard
22 Yi-1.5-34B 41.1% May 2024 Official leaderboard
23 Mistral-7B-Instruct-v0.2 40.8% Dec 2023 Official leaderboard
24 Qwen3-1.7B 39.9% Apr 2025 Official leaderboard
25 Mistral 7B v0.1 37.8% Sep 2023 Official leaderboard
26 Yi-1.5-9B-Chat 37.6% May 2024 Official leaderboard
27 Mistral 7B v0.3 37.1% Sep 2023 Official leaderboard
28 Mistral-7B-Instruct-v0.1 36% Sep 2023 Official leaderboard
29 Qwen2-1.5B 34.2% Jun 2024 Official leaderboard
30 phi-3-mini 3.8B 33.4% Apr 2024 Official leaderboard
31 Phi-2 25.6% Dec 2023 Official leaderboard
32 Phi-1.5 24.3% Sep 2023 Official leaderboard

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Qwen
Own leaderboard Not tested since Apr 2025
Qwen2.5-72B 69.1%

Source: Official LLMzSzŁ leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) Qwen2.5-72B Llama 3.1-70B Mixtral 8x7B Mistral 7B v0.1 Phi-1.5 0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) Qwen2.5-72B Llama 3-70B

Source: Official LLMzSzŁ leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Multilingual

All 545 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%

Data audit

9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 19 months ago, in April 2025, for Qwen3-1.7B. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (98 entries).
  • Pass Models scored 32 models we track, plus 66 other entries on its leaderboard.
  • Warning Newest score The newest came 19 months ago, in April 2025, for Qwen3-1.7B.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Qwen3-32B (#135).
  • Pass Headroom The best is 69.1%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on huggingface.co: 33 rows for models we track and 66 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check