Benchmark Almanac

SwitzerlandQA

Llama 3.3 70B Instruct 74.3% leads since September 2025
Llama 3.3 70B Instruct 74.3% leads since September 2025
Only in system cards

Models answer four-option questions on Swiss geography, history, social life, politics and insurance at national, cantonal and commune level. Items come in five languages.

Swiss AI built it for the Apertus report, with 9,167 items per language across 26 cantons. The listed Hugging Face dataset was not reachable on November 1st, 2026.

Released
September 2025
Kind
Knowledge in many languages
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Strongest model tested
Qwen3-32B #134 on the capability index
Last in a system card
September 2025
Last model added
September 2025
Scores refreshed
November 12th, 2026
36
Entries on its own leaderboard, 7 models we track
1
System card reports it, from 1 lab

Llama 3.3 70B Instruct leads with 74.3% among models released by September 2025

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

Llama 3.3 70B Instruct leads with 74.3%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

The newest model with a score here came out in September 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

36 entries. Scroll sideways for the rest.

  1. 74.3 Llama 3.3 70B Instruct
  2. 72.7 Qwen2.5-72B (pretrained checkpoint)
  3. 72.0 Llama4-Scout-16x17B (pretrained checkpoint)
  4. 71.0 Gemma 3 27B
  5. 70.0 Qwen2.5-72B
  6. 68.6 Llama3.1-70B (pretrained checkpoint)
  7. 67.2 Apertus-70B-Instruct
  8. 66.5 Qwen3-32B (pretrained checkpoint)
  9. 65.6 Qwen3-32B
  10. 65.3 GPT-OSS-20B (pretrained checkpoint)
  11. 65.1 gemma-3-12b-it
  12. 63.9 Qwen2.5-7B (pretrained checkpoint)
  13. 63.1 Apertus 8B Instruct
  14. 62.4 OLMo2-32B (pretrained checkpoint)
  15. 62.2 OLMo 2 32B Instruct
  16. 62.1 Apertus-8B (pretrained checkpoint)
  17. 62.0 K2-Chat
  18. 61.6 EuroLLM-22B-Instruct-Preview
  19. 61.2 EuroLLM-9B-Instruct
  20. 60.7 Qwen3-8B
  21. 60.2 Apertus-70B (pretrained checkpoint)
  22. 60.0 Llama-3.1-8B-Instruct
  23. 59.6 salamandra-7b-instruct
  24. 58.5 Llama3.1-8B (pretrained checkpoint)
  25. 58.1 EuroLLM-9B (pretrained checkpoint)
  26. 56.6 SmolLM3-3B
  27. 56.0 ALLaM-7B-Instruct-preview
  28. 55.2 SmolLM3-3B (pretrained checkpoint)
  29. 53.7 marin-8b-instruct
  30. 53.0 Teuken-7B-instruct-v0.6
  31. 52.5 OLMo2-7B (pretrained checkpoint)
  32. 47.2 OLMo-2-1124-7B-Instruct
  33. 42.1 Poro-34B (pretrained checkpoint)
  34. 37.4 Minerva-7B-instruct-v1.0
  35. 25.9 EuroLLM-1.7B (pretrained checkpoint)
  36. 22.4 SmolLM2-1.7B (pretrained checkpoint)
  1. 1 Llama 3.3 70B Instruct Instruction-tuned evaluation 74.3%
  2. 2 Qwen2.5-72B (pretrained checkpoint) Pretrained evaluation 72.7%
  3. 3 Llama4-Scout-16x17B (pretrained checkpoint) Pretrained evaluation 72%
  4. 4 Gemma 3 27B Instruction-tuned evaluation 71%
  5. 5 Qwen2.5-72B Instruction-tuned evaluation 70%
  6. 6 Llama3.1-70B (pretrained checkpoint) Pretrained evaluation 68.6%
  7. 7 Apertus-70B-Instruct Instruction-tuned evaluation 67.2%
  8. 8 Qwen3-32B (pretrained checkpoint) Pretrained evaluation 66.5%
  9. 9 Qwen3-32B Instruction-tuned evaluation 65.6%
  10. 10 GPT-OSS-20B (pretrained checkpoint) Pretrained evaluation 65.3%
  11. 11 gemma-3-12b-it Instruction-tuned evaluation 65.1%
  12. 12 Qwen2.5-7B (pretrained checkpoint) Pretrained evaluation 63.9%
  13. 13 Apertus 8B Instruct Instruction-tuned evaluation 63.1%
  14. 14 OLMo2-32B (pretrained checkpoint) Pretrained evaluation 62.4%
  15. 15 OLMo 2 32B Instruct Instruction-tuned evaluation 62.2%
  16. 16 Apertus-8B (pretrained checkpoint) Pretrained evaluation 62.1%
  17. 17 K2-Chat Instruction-tuned evaluation 62%
  18. 18 EuroLLM-22B-Instruct-Preview Instruction-tuned evaluation 61.6%
  19. 19 EuroLLM-9B-Instruct Instruction-tuned evaluation 61.2%
  20. 20 Qwen3-8B Instruction-tuned evaluation 60.7%
  21. 21 Apertus-70B (pretrained checkpoint) Pretrained evaluation 60.2%
  22. 22 Llama-3.1-8B-Instruct Instruction-tuned evaluation 60%
  23. 23 salamandra-7b-instruct Instruction-tuned evaluation 59.6%
  24. 24 Llama3.1-8B (pretrained checkpoint) Pretrained evaluation 58.5%
  25. 25 EuroLLM-9B (pretrained checkpoint) Pretrained evaluation 58.1%
  26. 26 SmolLM3-3B Instruction-tuned evaluation 56.6%
  27. 27 ALLaM-7B-Instruct-preview Instruction-tuned evaluation 56%
  28. 28 SmolLM3-3B (pretrained checkpoint) Pretrained evaluation 55.2%
  29. 29 marin-8b-instruct Instruction-tuned evaluation 53.7%
  30. 30 Teuken-7B-instruct-v0.6 Instruction-tuned evaluation 53%
  31. 31 OLMo2-7B (pretrained checkpoint) Pretrained evaluation 52.5%
  32. 32 OLMo-2-1124-7B-Instruct Instruction-tuned evaluation 47.2%
  33. 33 Poro-34B (pretrained checkpoint) Pretrained evaluation 42.1%
  34. 34 Minerva-7B-instruct-v1.0 Instruction-tuned evaluation 37.4%
  35. 35 EuroLLM-1.7B (pretrained checkpoint) Pretrained evaluation 25.9%
  36. 36 SmolLM2-1.7B (pretrained checkpoint) Pretrained evaluation 22.4%

Source: Official SwitzerlandQA leaderboard

Sources: Official SwitzerlandQA leaderboard and labs’ model cards

#EntryScoreSettingDateSource
1 Llama 3.3 70B Instruct 74.3% Instruction-tuned evaluation Dec 2024 Official leaderboard
2 Qwen2.5-72B (pretrained checkpoint) 72.7% Pretrained evaluation Official leaderboard
3 Llama4-Scout-16x17B (pretrained checkpoint) 72% Pretrained evaluation Official leaderboard
4 Gemma 3 27B 71% Instruction-tuned evaluation Mar 2025 Official leaderboard
5 Qwen2.5-72B 70% Instruction-tuned evaluation Sep 2024 Official leaderboard
6 Llama3.1-70B (pretrained checkpoint) 68.6% Pretrained evaluation Official leaderboard
7 Apertus-70B-Instruct 67.2% Instruction-tuned evaluation Official leaderboard
8 Qwen3-32B (pretrained checkpoint) 66.5% Pretrained evaluation Official leaderboard
9 Qwen3-32B 65.6% Instruction-tuned evaluation Apr 2025 Official leaderboard
10 GPT-OSS-20B (pretrained checkpoint) 65.3% Pretrained evaluation Official leaderboard
11 gemma-3-12b-it 65.1% Instruction-tuned evaluation Official leaderboard
12 Qwen2.5-7B (pretrained checkpoint) 63.9% Pretrained evaluation Official leaderboard
13 Apertus 8B Instruct 63.1% Instruction-tuned evaluation Sep 2025 Official leaderboard
14 OLMo2-32B (pretrained checkpoint) 62.4% Pretrained evaluation Official leaderboard
15 OLMo 2 32B Instruct 62.2% Instruction-tuned evaluation Mar 2025 Official leaderboard
16 Apertus-8B (pretrained checkpoint) 62.1% Pretrained evaluation Official leaderboard
17 K2-Chat 62% Instruction-tuned evaluation Official leaderboard
18 EuroLLM-22B-Instruct-Preview 61.6% Instruction-tuned evaluation Official leaderboard
19 EuroLLM-9B-Instruct 61.2% Instruction-tuned evaluation Official leaderboard
20 Qwen3-8B 60.7% Instruction-tuned evaluation Apr 2025 Official leaderboard
21 Apertus-70B (pretrained checkpoint) 60.2% Pretrained evaluation Official leaderboard
22 Llama-3.1-8B-Instruct 60% Instruction-tuned evaluation Official leaderboard
23 salamandra-7b-instruct 59.6% Instruction-tuned evaluation Official leaderboard
24 Llama3.1-8B (pretrained checkpoint) 58.5% Pretrained evaluation Official leaderboard
25 EuroLLM-9B (pretrained checkpoint) 58.1% Pretrained evaluation Official leaderboard
26 SmolLM3-3B 56.6% Instruction-tuned evaluation Official leaderboard
27 ALLaM-7B-Instruct-preview 56% Instruction-tuned evaluation Official leaderboard
28 SmolLM3-3B (pretrained checkpoint) 55.2% Pretrained evaluation Official leaderboard
29 marin-8b-instruct 53.7% Instruction-tuned evaluation Official leaderboard
30 Teuken-7B-instruct-v0.6 53% Instruction-tuned evaluation Official leaderboard
31 OLMo2-7B (pretrained checkpoint) 52.5% Pretrained evaluation Official leaderboard
32 OLMo-2-1124-7B-Instruct 47.2% Instruction-tuned evaluation Official leaderboard
33 Poro-34B (pretrained checkpoint) 42.1% Pretrained evaluation Official leaderboard
34 Minerva-7B-instruct-v1.0 37.4% Instruction-tuned evaluation Official leaderboard
35 EuroLLM-1.7B (pretrained checkpoint) 25.9% Pretrained evaluation Official leaderboard
36 SmolLM2-1.7B (pretrained checkpoint) 22.4% Pretrained evaluation Official leaderboard
#ModelScoreSettingDateSource
1 Llama 3.3 70B Instruct 74.3% Instruction-tuned evaluation Dec 2024 Official leaderboard
2 Gemma 3 27B 71% Instruction-tuned evaluation Mar 2025 Official leaderboard
3 Qwen2.5-72B 70% Instruction-tuned evaluation Sep 2024 Official leaderboard
4 Apertus 70B 67.2% Sep 2025 Lab model card
5 Qwen3-32B 65.6% Instruction-tuned evaluation Apr 2025 Official leaderboard
6 Apertus 8B Instruct 63.1% Instruction-tuned evaluation Sep 2025 Official leaderboard
7 OLMo 2 32B Instruct 62.2% Instruction-tuned evaluation Mar 2025 Official leaderboard
8 Qwen3-8B 60.7% Instruction-tuned evaluation Apr 2025 Official leaderboard

Labs report scores for 1 model in their cards. Switch the scores above to rank them together.

Number one over time Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Meta
Own leaderboard Not tested since Sep 2025
Llama 3.3 70B Instruct 74.3%

Source: Official SwitzerlandQA leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 Near the ceiling (85%) Llama 3.3 70B Instruct Qwen2.5-72B 0% 20% 40% 60% 80% 100% 2025 Near the ceiling (85%) Llama 3.3 70B Instruct

Sources: Official SwitzerlandQA leaderboard and labs’ model cards

Labs report it only in system cards

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark. Some system cards do, below.

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Sep 17, 2025 Apertus 70B technical report 67.2 Switzerland QA, 0-shot (Table 20) Capabilities

More in Multilingual

All 120 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Nemotron 3 Ultra 550B-A55B Base 90.1%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%