EXAMS
Models answer high school exam questions in 16 languages across many subjects. Arabic reports use the Arabic subset.
EMNLP 2020. OALL v1 and v2 and TII Falcon posts report the Arabic subset as Exams. Qwen's Multilingual Exams is a different test.
- Made by
- Sofia University and collaborators (Hardalov et al.)
- Released
- November 2020
- Kind
- Knowledge in many languages
- Level
- School
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- All public
In the authors’ words
We propose EXAMS – a new benchmark dataset for cross-lingual and multilingual question answering for high school examinations. We collected more than 24,000 high-quality high school exam questions in 16 languages, covering 8 language families and 24 school subjects from Natural Sciences and Social Sciences, among others. EXAMS offers a fine-grained evaluation framework across multiple languages and subjects, which allows precise analysis and comparison of various models. We perform various experiments with existing top-performing multilingual pre-trained models and we show that EXAMS offers multiple challenges that require multilingual knowledge and reasoning in multiple domains. We hope that EXAMS will enable researchers to explore challenging reasoning and knowledge transfer methods and pre-trained models for school question answering in various languages which was not possible before. The data, code, pre-trained models, and evaluation are available at https://github.com/mhardalov/exams-qa.
XLM-R tops the leaderboard with 42%
Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.
- 1 XLM-R RACE + SciENs + EXAMS (Full) 42%
- 2 mBERT Full training 34.6%
- 3 XLM-R Base Full training 34.1%
Source: Official EXAMS leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | XLM-R | 42% | RACE + SciENs + EXAMS (Full) | Official leaderboard | |
| 2 | mBERT | 34.6% | Full training | Official leaderboard | |
| 3 | XLM-R Base | 34.1% | Full training | Official leaderboard |
No lab model card reports it yet
We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5% - MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87% - CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7% - Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy.
Nemotron 3 Ultra 550B-A55B Base 90.1% - MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%