Evalita-LLM
Models perform tasks written originally in Italian. The suite includes admission exams, question answering and text generation.
- Made by
- Bernardo Magnini and colleagues
- Released
- February 2025
- Kind
- Reading in many languages
- Level
- University
- Scored
- Checked automatically
- Answers
- Open-ended
- Tasks
- All public
- Measures
- Capability
- Subject
- Languages and literature
- Language
- Italian
- Interaction
- Single turn
- Items from
- Exams and contests
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
- gpt-oss-20b #143 on the capability index
- Last model added
- September 2025
- Scores refreshed
- November 20th, 2026
Mistral Large 2 leads with 62.3 among models released by September 2025 Mistral Large 2 leads with 62.3 among models released by September 2025
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
The newest model with a score here came out in September 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 62.3
Mistral Large 2
- 60.2
Tulu 3 (Tülu 3) 70B
- 59.2
Gemma 3 27B
- 57.9
Llama 3.3 70B Instruct
- 57.7
Mistral Small 3
- 57.4
Qwen2.5-72B
- 57.3
Qwen2.5-VL-32B
- 56.8 calme-3.2-instruct-78b
- 55.4
Qwen3-Next-80B-A3B-Instruct
- 54.7
Qwen2.5-14B-Instruct-1M
- 54.7
DeepSeek-R1-Distill-Llama-70B
- 53.9
Gemma 3 12B
- 53.9 gemma-2-27b-it
- 53.6 gemma-2-9b-it
- 53.0 ANITA-NEXT-24B-Magistral-2506-ITA
- 52.7 ANITA-NEXT-24B-Magistral-2506-VISION-ITA
- 52.6 Qwen2.5-7B-Instruct
- 52.4
Gemma 3n E4B
- 51.8
Phi-4
- 51.7 Llama-3.1-SuperNova-Lite
- 51.7 Granite-3.1-8B-Instruct
- 51.6
Llama 4 Scout
- 51.2
phi-3-medium 14B
- 50.4 Meta-Llama-3.1-8B-Instruct
- 50.1
Phi-3.5-mini
- 49.6
Qwen2.5-VL-7B
- 49.5
Gemma 3 4B
- 49.4 Llama-3-8b-Ita
- 49.4 LLaMAntino-3-ANITA-8B-Inst-DPO-ITA
- 49.4 maestrale-chat-v0.4-beta
- 49.3
Aya Expanse 8B
- 48.5 GLM-4.1V-9B-Thinking
- 47.3 Mistral-7B-Instruct-v0.3
- 46.9 Lexora-Medium-7B
- 46.5
InternVL3.5-8B
- 44.9 Llama-3-8B-4bit-UltraChat-Ita
- 44.8
Aya Vision 8B
- 44.1
Pixtral 12B
- 43.8
Zephyr 7B Beta
- 43.7 Volare
- 43.7 occiglot-7b-it-en-instruct
- 42.7 Velvet-14B
- 42.6 FastwebMIIA-7B
- 37.9 LLaMAntino-2-7b-hf-ITA
- 37.2 gpt-oss-20b
- 35.7 Minerva-7B-instruct-v1.0
- 35.1 Minerva-7B-base-v1.0
- 30.1
Gemma 3 270M Base
- 1
Mistral Large 2 Nov 2024, 5-shot, 10 text tasks 62.3
- 2
Tulu 3 (Tülu 3) 70B 5-shot, 10 text tasks 60.2
- 3
Gemma 3 27B 5-shot, 12 text and multimodal tasks 59.2
- 4
Llama 3.3 70B Instruct 5-shot, 10 text tasks 57.9
- 5
Mistral Small 3 5-shot, 10 text tasks 57.7
- 6
Qwen2.5-72B 5-shot, 10 text tasks 57.4
- 7
Qwen2.5-VL-32B 5-shot, 12 text and multimodal tasks 57.3
- 8 calme-3.2-instruct-78b 5-shot, 10 text tasks 56.8
- 9
Qwen3-Next-80B-A3B-Instruct 0-shot, 10 text tasks 55.4
- 10
Qwen2.5-14B-Instruct-1M 5-shot, 10 text tasks 54.7
- 11
DeepSeek-R1-Distill-Llama-70B 5-shot, 10 text tasks 54.7
- 12
Gemma 3 12B 5-shot, 10 text tasks 53.9
- 13 gemma-2-27b-it 5-shot, 10 text tasks 53.9
- 14 gemma-2-9b-it 5-shot, 10 text tasks 53.6
- 15 ANITA-NEXT-24B-Magistral-2506-ITA 5-shot, 10 text tasks 53.0
- 16 ANITA-NEXT-24B-Magistral-2506-VISION-ITA 5-shot, 10 text tasks 52.7
- 17 Qwen2.5-7B-Instruct 5-shot, 10 text tasks 52.6
- 18
Gemma 3n E4B 5-shot, 10 text tasks 52.4
- 19
Phi-4 5-shot, 10 text tasks 51.8
- 20 Llama-3.1-SuperNova-Lite 5-shot, 10 text tasks 51.7
- 21 Granite-3.1-8B-Instruct 5-shot, 10 text tasks 51.7
- 22
Llama 4 Scout 0-shot, 10 text tasks 51.6
- 23
phi-3-medium 14B 5-shot, 10 text tasks 51.2
- 24 Meta-Llama-3.1-8B-Instruct 5-shot, 10 text tasks 50.4
- 25
Phi-3.5-mini 5-shot, 10 text tasks 50.1
- 26
Qwen2.5-VL-7B 5-shot, 12 text and multimodal tasks 49.6
- 27
Gemma 3 4B 5-shot, 12 text and multimodal tasks 49.5
- 28 Llama-3-8b-Ita 5-shot, 10 text tasks 49.4
- 29 LLaMAntino-3-ANITA-8B-Inst-DPO-ITA 5-shot, 10 text tasks 49.4
- 30 maestrale-chat-v0.4-beta 5-shot, 10 text tasks 49.4
- 31
Aya Expanse 8B 5-shot, 10 text tasks 49.3
- 32 GLM-4.1V-9B-Thinking 5-shot, 12 text and multimodal tasks 48.5
- 33 Mistral-7B-Instruct-v0.3 5-shot, 10 text tasks 47.3
- 34 Lexora-Medium-7B 0-shot, 10 text tasks 46.9
- 35
InternVL3.5-8B 0-shot, 10 text tasks 46.5
- 36 Llama-3-8B-4bit-UltraChat-Ita 5-shot, 10 text tasks 44.9
- 37
Aya Vision 8B 0-shot, 12 text and multimodal tasks 44.8
- 38
Pixtral 12B 0-shot, 12 text and multimodal tasks 44.1
- 39
Zephyr 7B Beta 5-shot, 10 text tasks 43.8
- 40 Volare 5-shot, 10 text tasks 43.7
- 41 occiglot-7b-it-en-instruct 5-shot, 10 text tasks 43.7
- 42 Velvet-14B 5-shot, 10 text tasks 42.7
- 43 FastwebMIIA-7B 5-shot, 10 text tasks 42.6
- 44 LLaMAntino-2-7b-hf-ITA 5-shot, 10 text tasks 37.9
- 45 gpt-oss-20b 5-shot, 10 text tasks 37.2
- 46 Minerva-7B-instruct-v1.0 5-shot, 10 text tasks 35.7
- 47 Minerva-7B-base-v1.0 5-shot, 10 text tasks 35.1
- 48
Gemma 3 270M Base 0-shot, 10 text tasks 30.1
Source: Official Evalita-LLM leaderboard
- 62.3
Mistral Large 2
- 60.2
Tulu 3 (Tülu 3) 70B
- 59.2
Gemma 3 27B
- 57.9
Llama 3.3 70B Instruct
- 57.7
Mistral Small 3
- 57.4
Qwen2.5-72B
- 57.3
Qwen2.5-VL-32B
- 55.4
Qwen3-Next-80B-A3B-Instruct
- 54.7
Qwen2.5-14B-Instruct-1M
- 54.7
DeepSeek-R1-Distill-Llama-70B
- 53.9
Gemma 3 12B
- 52.4
Gemma 3n E4B
- 51.8
Phi-4
- 51.7 Granite-3.1-8B-Instruct
- 51.6
Llama 4 Scout
- 51.2
phi-3-medium 14B
- 50.1
Phi-3.5-mini
- 49.6
Qwen2.5-VL-7B
- 49.5
Gemma 3 4B
- 49.3
Aya Expanse 8B
- 48.5 GLM-4.1V-9B-Thinking
- 46.5
InternVL3.5-8B
- 44.8
Aya Vision 8B
- 44.1
Pixtral 12B
- 43.8
Zephyr 7B Beta
- 37.2 gpt-oss-20b
- 30.1
Gemma 3 270M Base
- 1
Mistral Large 2 Nov 2024, 5-shot, 10 text tasks 62.3
- 2
Tulu 3 (Tülu 3) 70B 5-shot, 10 text tasks 60.2
- 3
Gemma 3 27B 5-shot, 12 text and multimodal tasks 59.2
- 4
Llama 3.3 70B Instruct 5-shot, 10 text tasks 57.9
- 5
Mistral Small 3 5-shot, 10 text tasks 57.7
- 6
Qwen2.5-72B 5-shot, 10 text tasks 57.4
- 7
Qwen2.5-VL-32B 5-shot, 12 text and multimodal tasks 57.3
- 8
Qwen3-Next-80B-A3B-Instruct 0-shot, 10 text tasks 55.4
- 9
Qwen2.5-14B-Instruct-1M 5-shot, 10 text tasks 54.7
- 10
DeepSeek-R1-Distill-Llama-70B 5-shot, 10 text tasks 54.7
- 11
Gemma 3 12B 5-shot, 10 text tasks 53.9
- 12
Gemma 3n E4B 5-shot, 10 text tasks 52.4
- 13
Phi-4 5-shot, 10 text tasks 51.8
- 14 Granite-3.1-8B-Instruct 5-shot, 10 text tasks 51.7
- 15
Llama 4 Scout 0-shot, 10 text tasks 51.6
- 16
phi-3-medium 14B 5-shot, 10 text tasks 51.2
- 17
Phi-3.5-mini 5-shot, 10 text tasks 50.1
- 18
Qwen2.5-VL-7B 5-shot, 12 text and multimodal tasks 49.6
- 19
Gemma 3 4B 5-shot, 12 text and multimodal tasks 49.5
- 20
Aya Expanse 8B 5-shot, 10 text tasks 49.3
- 21 GLM-4.1V-9B-Thinking 5-shot, 12 text and multimodal tasks 48.5
- 22
InternVL3.5-8B 0-shot, 10 text tasks 46.5
- 23
Aya Vision 8B 0-shot, 12 text and multimodal tasks 44.8
- 24
Pixtral 12B 0-shot, 12 text and multimodal tasks 44.1
- 25
Zephyr 7B Beta 5-shot, 10 text tasks 43.8
- 26 gpt-oss-20b 5-shot, 10 text tasks 37.2
- 27
Gemma 3 270M Base 0-shot, 10 text tasks 30.1
Source: Official Evalita-LLM leaderboard
| # | Entry | Score | Setting | Date | Link | Source |
|---|---|---|---|---|---|---|
| 1 | | 62.3 | 5-shot, 10 text tasks | Nov 2024 | huggingface.co | Official leaderboard |
| 2 | | 60.2 | 5-shot, 10 text tasks | Nov 2024 | huggingface.co | Official leaderboard |
| 3 | | 59.2 | 5-shot, 12 text and multimodal tasks | Mar 2025 | huggingface.co | Official leaderboard |
| 4 | | 57.9 | 5-shot, 10 text tasks | Dec 2024 | huggingface.co | Official leaderboard |
| 5 | | 57.7 | 5-shot, 10 text tasks | Jan 2025 | huggingface.co | Official leaderboard |
| 6 | | 57.4 | 5-shot, 10 text tasks | Sep 2024 | huggingface.co | Official leaderboard |
| 7 | | 57.3 | 5-shot, 12 text and multimodal tasks | Mar 2025 | huggingface.co | Official leaderboard |
| 8 | calme-3.2-instruct-78b 78B | 56.8 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 9 | | 55.4 | 0-shot, 10 text tasks | Sep 2025 | huggingface.co | Official leaderboard |
| 10 | | 54.7 | 5-shot, 10 text tasks | Jan 2025 | huggingface.co | Official leaderboard |
| 11 | | 54.7 | 5-shot, 10 text tasks | Jan 2025 | huggingface.co | Official leaderboard |
| 12 | | 53.9 | 5-shot, 10 text tasks | Mar 2025 | huggingface.co | Official leaderboard |
| 13 | gemma-2-27b-it 28B | 53.9 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 14 | gemma-2-9b-it 10B | 53.6 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 15 | ANITA-NEXT-24B-Magistral-2506-ITA 24B | 53.0 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 16 | ANITA-NEXT-24B-Magistral-2506-VISION-ITA 25B | 52.7 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 17 | Qwen2.5-7B-Instruct 8B | 52.6 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 18 | | 52.4 | 5-shot, 10 text tasks | Jun 2025 | huggingface.co | Official leaderboard |
| 19 | | 51.8 | 5-shot, 10 text tasks | Dec 2024 | huggingface.co | Official leaderboard |
| 20 | Llama-3.1-SuperNova-Lite 9B | 51.7 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 21 | Granite-3.1-8B-Instruct | 51.7 | 5-shot, 10 text tasks | Dec 2024 | huggingface.co | Official leaderboard |
| 22 | | 51.6 | 0-shot, 10 text tasks | Apr 2025 | huggingface.co | Official leaderboard |
| 23 | | 51.2 | 5-shot, 10 text tasks | Apr 2024 | huggingface.co | Official leaderboard |
| 24 | Meta-Llama-3.1-8B-Instruct 9B | 50.4 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 25 | | 50.1 | 5-shot, 10 text tasks | Aug 2024 | huggingface.co | Official leaderboard |
| 26 | | 49.6 | 5-shot, 12 text and multimodal tasks | Jan 2025 | huggingface.co | Official leaderboard |
| 27 | | 49.5 | 5-shot, 12 text and multimodal tasks | Mar 2025 | huggingface.co | Official leaderboard |
| 28 | Llama-3-8b-Ita 9B | 49.4 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 29 | LLaMAntino-3-ANITA-8B-Inst-DPO-ITA 9B | 49.4 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 30 | maestrale-chat-v0.4-beta 8B | 49.4 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 31 | | 49.3 | 5-shot, 10 text tasks | Oct 2024 | huggingface.co | Official leaderboard |
| 32 | GLM-4.1V-9B-Thinking | 48.5 | 5-shot, 12 text and multimodal tasks | Jul 2025 | huggingface.co | Official leaderboard |
| 33 | Mistral-7B-Instruct-v0.3 8B | 47.3 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 34 | Lexora-Medium-7B 8B | 46.9 | 0-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 35 | | 46.5 | 0-shot, 10 text tasks | Aug 2025 | huggingface.co | Official leaderboard |
| 36 | Llama-3-8B-4bit-UltraChat-Ita 9B | 44.9 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 37 | | 44.8 | 0-shot, 12 text and multimodal tasks | Mar 2025 | huggingface.co | Official leaderboard |
| 38 | | 44.1 | 0-shot, 12 text and multimodal tasks | Sep 2024 | huggingface.co | Official leaderboard |
| 39 | | 43.8 | 5-shot, 10 text tasks | Oct 2023 | huggingface.co | Official leaderboard |
| 40 | Volare 9B | 43.7 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 41 | occiglot-7b-it-en-instruct 8B | 43.7 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 42 | Velvet-14B 15B | 42.7 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 43 | FastwebMIIA-7B 8B | 42.6 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 44 | LLaMAntino-2-7b-hf-ITA 7B | 37.9 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 45 | gpt-oss-20b | 37.2 | 5-shot, 10 text tasks | Aug 2025 | huggingface.co | Official leaderboard |
| 46 | Minerva-7B-instruct-v1.0 8B | 35.7 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 47 | Minerva-7B-base-v1.0 8B | 35.1 | 5-shot, 10 text tasks | huggingface.co | Official leaderboard | |
| 48 | | 30.1 | 0-shot, 10 text tasks | Aug 2025 | huggingface.co | Official leaderboard |
| # | Model | Score | Setting | Date | Link | Source |
|---|---|---|---|---|---|---|
| 1 | | 62.3 | 5-shot, 10 text tasks | Nov 2024 | huggingface.co | Official leaderboard |
| 2 | | 60.2 | 5-shot, 10 text tasks | Nov 2024 | huggingface.co | Official leaderboard |
| 3 | | 59.2 | 5-shot, 12 text and multimodal tasks | Mar 2025 | huggingface.co | Official leaderboard |
| 4 | | 57.9 | 5-shot, 10 text tasks | Dec 2024 | huggingface.co | Official leaderboard |
| 5 | | 57.7 | 5-shot, 10 text tasks | Jan 2025 | huggingface.co | Official leaderboard |
| 6 | | 57.4 | 5-shot, 10 text tasks | Sep 2024 | huggingface.co | Official leaderboard |
| 7 | | 57.3 | 5-shot, 12 text and multimodal tasks | Mar 2025 | huggingface.co | Official leaderboard |
| 8 | | 55.4 | 0-shot, 10 text tasks | Sep 2025 | huggingface.co | Official leaderboard |
| 9 | | 54.7 | 5-shot, 10 text tasks | Jan 2025 | huggingface.co | Official leaderboard |
| 10 | | 54.7 | 5-shot, 10 text tasks | Jan 2025 | huggingface.co | Official leaderboard |
| 11 | | 53.9 | 5-shot, 10 text tasks | Mar 2025 | huggingface.co | Official leaderboard |
| 12 | | 52.4 | 5-shot, 10 text tasks | Jun 2025 | huggingface.co | Official leaderboard |
| 13 | | 51.8 | 5-shot, 10 text tasks | Dec 2024 | huggingface.co | Official leaderboard |
| 14 | Granite-3.1-8B-Instruct | 51.7 | 5-shot, 10 text tasks | Dec 2024 | huggingface.co | Official leaderboard |
| 15 | | 51.6 | 0-shot, 10 text tasks | Apr 2025 | huggingface.co | Official leaderboard |
| 16 | | 51.2 | 5-shot, 10 text tasks | Apr 2024 | huggingface.co | Official leaderboard |
| 17 | | 50.1 | 5-shot, 10 text tasks | Aug 2024 | huggingface.co | Official leaderboard |
| 18 | | 49.6 | 5-shot, 12 text and multimodal tasks | Jan 2025 | huggingface.co | Official leaderboard |
| 19 | | 49.5 | 5-shot, 12 text and multimodal tasks | Mar 2025 | huggingface.co | Official leaderboard |
| 20 | | 49.3 | 5-shot, 10 text tasks | Oct 2024 | huggingface.co | Official leaderboard |
| 21 | GLM-4.1V-9B-Thinking | 48.5 | 5-shot, 12 text and multimodal tasks | Jul 2025 | huggingface.co | Official leaderboard |
| 22 | | 46.5 | 0-shot, 10 text tasks | Aug 2025 | huggingface.co | Official leaderboard |
| 23 | | 44.8 | 0-shot, 12 text and multimodal tasks | Mar 2025 | huggingface.co | Official leaderboard |
| 24 | | 44.1 | 0-shot, 12 text and multimodal tasks | Sep 2024 | huggingface.co | Official leaderboard |
| 25 | | 43.8 | 5-shot, 10 text tasks | Oct 2023 | huggingface.co | Official leaderboard |
| 26 | gpt-oss-20b | 37.2 | 5-shot, 10 text tasks | Aug 2025 | huggingface.co | Official leaderboard |
| 27 | | 30.1 | 0-shot, 10 text tasks | Aug 2025 | huggingface.co | Official leaderboard |
Rankings by part
We read 12 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as TE, SA and HS. Mistral Large 2 leads 3 of them, and 7 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
TE
27 models we track have a score here.
- 1
Qwen2.5-72B ↑5 85.1
- 2
Qwen2.5-14B-Instruct-1M ↑7 85.0
- 3
Qwen2.5-VL-32B ↑4 83.7
- 4
Tulu 3 (Tülu 3) 70B ↓2 82.4
- 5
Mistral Small 3 82.0
- 6
Gemma 3 27B ↓3 81.1
- 7
Mistral Large 2 Nov 2024 ↓6 81.0
- 8
Llama 3.3 70B Instruct ↓4 80.8
- 9 GLM-4.1V-9B-Thinking ↑12 79.8
- 10
Gemma 3 12B ↑1 79.5
SA
27 models we track have a score here.
- 1
Mistral Large 2 Nov 2024 80.9
- 2
Tulu 3 (Tülu 3) 70B 80.4
- 3
Gemma 3 27B 80.3
- 4
Llama 3.3 70B Instruct 79.5
- 5
Gemma 3 12B ↑6 78.3
- 6
Qwen2.5-72B 77.7
- 7
DeepSeek-R1-Distill-Llama-70B ↑3 77.5
- 8
Gemma 3n E4B ↑4 76.3
- 9
Mistral Small 3 ↓4 76.0
- 10
Gemma 3 4B ↑9 75.3
HS
27 models we track have a score here.
- 1
Llama 3.3 70B Instruct ↑3 77.8
- 2
Mistral Large 2 Nov 2024 ↓1 77.0
- 3
Qwen2.5-72B ↑3 76.4
- 4
Gemma 3 27B ↓1 75.9
- 5
Tulu 3 (Tülu 3) 70B ↓3 75.5
- 6
Qwen2.5-VL-32B ↑1 75.5
- 7
DeepSeek-R1-Distill-Llama-70B ↑3 74.6
- 8
Mistral Small 3 ↓3 72.9
- 9
Qwen2.5-14B-Instruct-1M 72.8
- 10
Gemma 3 12B ↑1 71.4
AT
27 models we track have a score here.
- 1
Mistral Large 2 Nov 2024 76.3
- 2
Gemma 3 27B ↑1 73.9
- 3
Tulu 3 (Tülu 3) 70B ↓1 72.4
- 4
Llama 3.3 70B Instruct 70.7
- 5
Mistral Small 3 69.8
- 6
Qwen2.5-72B 69.3
- 7
Llama 4 Scout ↑8 68.2
- 8
Gemma 3 12B ↑3 65.9
- 9
DeepSeek-R1-Distill-Llama-70B ↑1 65.7
- 10
Qwen3-Next-80B-A3B-Instruct ↓2 62.5
WIC
27 models we track have a score here.
- 1
Qwen2.5-VL-32B ↑6 77.6
- 2
Mistral Large 2 Nov 2024 ↓1 75.5
- 3
Qwen2.5-72B ↑3 72.1
- 4
Mistral Small 3 ↑1 71.7
- 5
Qwen3-Next-80B-A3B-Instruct ↑3 71.4
- 6
DeepSeek-R1-Distill-Llama-70B ↑4 69.3
- 7
phi-3-medium 14B ↑9 68.9
- 8
Llama 3.3 70B Instruct ↓4 67.2
- 9
Gemma 3 12B ↑2 66.5
- 10
Gemma 3 27B ↓7 66.5
FAQ
27 models we track have a score here.
- 1 Granite-3.1-8B-Instruct ↑13 54.7
- 2
Mistral Large 2 Nov 2024 ↓1 54.7
- 3
Mistral Small 3 ↑2 54.5
- 4
Tulu 3 (Tülu 3) 70B ↓2 54.5
- 5
Qwen2.5-14B-Instruct-1M ↑4 54.4
- 6
Aya Expanse 8B ↑14 54.4
- 7
Llama 3.3 70B Instruct ↓3 54.3
- 8
Phi-4 ↑5 54.3
- 9
Qwen2.5-72B ↓3 54.0
- 10
Llama 4 Scout ↑5 54.0
LS
27 models we track have a score here.
- 1
Qwen3-Next-80B-A3B-Instruct ↑7 54.5
- 2
Mistral Small 3 ↑3 52.7
- 3
Qwen2.5-72B ↑3 50.2
- 4
Llama 3.3 70B Instruct 46.8
- 5
Tulu 3 (Tülu 3) 70B ↓3 46.5
- 6
Gemma 3 27B ↓3 42.4
- 7
Qwen2.5-VL-32B 40.7
- 8
Llama 4 Scout ↑7 39.3
- 9
DeepSeek-R1-Distill-Llama-70B ↑1 38.6
- 10
Mistral Large 2 Nov 2024 ↓9 38.6
SU
27 models we track have a score here.
- 1
Tulu 3 (Tülu 3) 70B ↑1 35.5
- 2
Mistral Large 2 Nov 2024 ↓1 33.4
- 3 Granite-3.1-8B-Instruct ↑11 32.9
- 4
Mistral Small 3 ↑1 31.9
- 5
Gemma 3 12B ↑6 29.9
- 6
Gemma 3 27B ↓3 29.8
- 7
Gemma 3n E4B ↑5 29.8
- 8
Gemma 3 4B ↑11 29.2
- 9
InternVL3.5-8B ↑13 28.5
- 10
Llama 4 Scout ↑5 28.2
NER
27 models we track have a score here.
- 1
Llama 3.3 70B Instruct ↑3 44.6
- 2
Tulu 3 (Tülu 3) 70B 39.1
- 3
Qwen2.5-72B ↑3 38.9
- 4
Mistral Large 2 Nov 2024 ↓3 38.8
- 5
Gemma 3 27B ↓2 38.8
- 6
Mistral Small 3 ↓1 38.2
- 7
Gemma 3 12B ↑4 37.9
- 8
Qwen2.5-VL-32B ↓1 37.8
- 9
DeepSeek-R1-Distill-Llama-70B ↑1 37.8
- 10
Gemma 3 4B ↑9 36.0
REL
27 models we track have a score here.
- 1
Mistral Large 2 Nov 2024 66.8
- 2
Tulu 3 (Tülu 3) 70B 55.4
- 3
Gemma 3n E4B ↑9 49.6
- 4
Gemma 3 27B ↓1 48.5
- 5
Qwen2.5-14B-Instruct-1M ↑4 44.2
- 6
Qwen2.5-72B 41.6
- 7
Gemma 3 12B ↑4 41.6
- 8
Mistral Small 3 ↓3 40.9
- 9 Granite-3.1-8B-Instruct ↑5 39.9
- 10
Qwen3-Next-80B-A3B-Instruct ↓2 38.9
MAIA-MC
7 models we track have a score here.
- 1
Qwen2.5-VL-32B ↑6 87.3
- 2
Qwen2.5-VL-7B ↑16 81.8
- 3
Gemma 3 27B 81.4
- 4
Pixtral 12B ↑20 80.3
- 5
Aya Vision 8B ↑18 75.9
- 6
Gemma 3 4B ↑13 69.3
- 7 GLM-4.1V-9B-Thinking ↑14 59.6
MAIA-GEN
7 models we track have a score here.
- 1
Gemma 3 4B ↑18 58.4
- 2
Gemma 3 27B ↑1 58.2
- 3
Qwen2.5-VL-32B ↑4 55.9
- 4
Aya Vision 8B ↑19 52.3
- 5
Qwen2.5-VL-7B ↑13 35.3
- 6
Pixtral 12B ↑18 32.4
- 7 GLM-4.1V-9B-Thinking ↑14 9.86
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: Official Evalita-LLM leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| 5-shot, 10 text tasks | 62.3 | ||||
| 5-shot, 10 text tasks | 60.2 | ||||
| 5-shot, 12 text and multimodal tasks | 59.2 | ||||
| 5-shot, 10 text tasks | 57.9 | ||||
| 5-shot, 10 text tasks | 57.7 | ||||
| 5-shot, 10 text tasks | 57.4 | ||||
| 5-shot, 12 text and multimodal tasks | 57.3 | ||||
| 0-shot, 10 text tasks | 55.4 | ||||
| 5-shot, 10 text tasks | 54.7 | ||||
| 5-shot, 10 text tasks | 54.7 | ||||
| 5-shot, 10 text tasks | 53.9 | ||||
| 0-shot, 12 text and multimodal tasks | 53.9 | ||||
| 0-shot, 12 text and multimodal tasks | 53.1 | ||||
| 5-shot, 10 text tasks | 52.4 | ||||
| 0-shot, 10 text tasks | 52.4 | ||||
| 5-shot, 10 text tasks | 51.8 | ||||
| Granite-3.1-8B-Instruct | 5-shot, 10 text tasks | 51.7 | |||
| 0-shot, 10 text tasks | 51.6 | ||||
| 0-shot, 10 text tasks | 51.6 | ||||
| 0-shot, 10 text tasks | 51.3 | ||||
| 5-shot, 10 text tasks | 51.2 | ||||
| 0-shot, 10 text tasks | 50.9 | ||||
| 5-shot, 10 text tasks | 50.1 | ||||
| 5-shot, 12 text and multimodal tasks | 49.6 | ||||
| 5-shot, 12 text and multimodal tasks | 49.5 | ||||
| 5-shot, 10 text tasks | 49.3 | ||||
| GLM-4.1V-9B-Thinking | 5-shot, 12 text and multimodal tasks | 48.5 | |||
| 0-shot, 10 text tasks | 48.4 | ||||
| 0-shot, 10 text tasks | 47.7 | ||||
| 0-shot, 10 text tasks | 47.4 | ||||
| 0-shot, 12 text and multimodal tasks | 47.2 | ||||
| 0-shot, 10 text tasks | 46.5 | ||||
| 0-shot, 12 text and multimodal tasks | 46.4 | ||||
| 0-shot, 12 text and multimodal tasks | 44.8 | ||||
| 0-shot, 10 text tasks | 44.4 | ||||
| 0-shot, 10 text tasks | 44.4 | ||||
| 0-shot, 12 text and multimodal tasks | 44.1 | ||||
| 5-shot, 10 text tasks | 43.8 | ||||
| 0-shot, 10 text tasks | 42.1 | ||||
| 0-shot, 10 text tasks | 42.0 | ||||
| 0-shot, 10 text tasks | 39.9 | ||||
| 0-shot, 10 text tasks | 38.4 | ||||
| GLM-4.1V-9B-Thinking | 0-shot, 12 text and multimodal tasks | 37.5 | |||
| 0-shot, 10 text tasks | 37.3 | ||||
| Granite-3.1-8B-Instruct | 0-shot, 10 text tasks | 37.3 | |||
| gpt-oss-20b | 5-shot, 10 text tasks | 37.2 | |||
| 5-shot, 10 text tasks | 36.5 | ||||
| gpt-oss-20b | 0-shot, 10 text tasks | 35.0 | |||
| 0-shot, 10 text tasks | 30.1 |
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: Official Evalita-LLM leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| 5-shot, 10 text tasks | 62.3 | ||||
| 5-shot, 10 text tasks | 60.2 | ||||
| 5-shot, 12 text and multimodal tasks | 59.2 | ||||
| 5-shot, 10 text tasks | 57.9 | ||||
| 5-shot, 10 text tasks | 57.7 | ||||
| 5-shot, 10 text tasks | 57.4 | ||||
| 5-shot, 12 text and multimodal tasks | 57.3 | ||||
| 0-shot, 10 text tasks | 55.4 | ||||
| 5-shot, 10 text tasks | 54.7 | ||||
| 5-shot, 10 text tasks | 54.7 | ||||
| 5-shot, 10 text tasks | 53.9 | ||||
| 0-shot, 12 text and multimodal tasks | 53.9 | ||||
| 0-shot, 12 text and multimodal tasks | 53.1 | ||||
| 5-shot, 10 text tasks | 52.4 | ||||
| 0-shot, 10 text tasks | 52.4 | ||||
| 5-shot, 10 text tasks | 51.8 | ||||
| Granite-3.1-8B-Instruct | 5-shot, 10 text tasks | 51.7 | |||
| 0-shot, 10 text tasks | 51.6 | ||||
| 0-shot, 10 text tasks | 51.6 | ||||
| 0-shot, 10 text tasks | 51.3 | ||||
| 5-shot, 10 text tasks | 51.2 | ||||
| 0-shot, 10 text tasks | 50.9 | ||||
| 5-shot, 10 text tasks | 50.1 | ||||
| 5-shot, 12 text and multimodal tasks | 49.6 | ||||
| 5-shot, 12 text and multimodal tasks | 49.5 | ||||
| 5-shot, 10 text tasks | 49.3 | ||||
| GLM-4.1V-9B-Thinking | 5-shot, 12 text and multimodal tasks | 48.5 | |||
| 0-shot, 10 text tasks | 48.4 | ||||
| 0-shot, 10 text tasks | 47.7 | ||||
| 0-shot, 10 text tasks | 47.4 | ||||
| 0-shot, 12 text and multimodal tasks | 47.2 | ||||
| 0-shot, 10 text tasks | 46.5 | ||||
| 0-shot, 12 text and multimodal tasks | 46.4 | ||||
| 0-shot, 12 text and multimodal tasks | 44.8 | ||||
| 0-shot, 10 text tasks | 44.4 | ||||
| 0-shot, 10 text tasks | 44.4 | ||||
| 0-shot, 12 text and multimodal tasks | 44.1 | ||||
| 5-shot, 10 text tasks | 43.8 | ||||
| 0-shot, 10 text tasks | 42.1 | ||||
| 0-shot, 10 text tasks | 42.0 | ||||
| 0-shot, 10 text tasks | 39.9 | ||||
| 0-shot, 10 text tasks | 38.4 | ||||
| GLM-4.1V-9B-Thinking | 0-shot, 12 text and multimodal tasks | 37.5 | |||
| 0-shot, 10 text tasks | 37.3 | ||||
| Granite-3.1-8B-Instruct | 0-shot, 10 text tasks | 37.3 | |||
| gpt-oss-20b | 5-shot, 10 text tasks | 37.2 | |||
| 5-shot, 10 text tasks | 36.5 | ||||
| gpt-oss-20b | 0-shot, 10 text tasks | 35.0 | |||
| 0-shot, 10 text tasks | 30.1 |
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official Evalita-LLM leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official Evalita-LLM leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5%
- MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87%
- CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7%
- Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
- MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%
Data audit
8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 14 months ago, in September 2025, for Qwen3-Next-80B-A3B-Instruct. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (48 entries).
- Pass Models scored 27 models we track, plus 21 other entries on its leaderboard.
- Warning Newest score The newest came 14 months ago, in September 2025, for Qwen3-Next-80B-A3B-Instruct.
- Note Frontier models None of the top 10 has a score. The strongest tested is gpt-oss-20b (#143).
- Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on huggingface.co: 49 rows for models we track and 21 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.