COPAL-ID
Models choose a plausible cause or consequence in Indonesian scenarios. The questions require familiarity with local customs and expressions.
- Made by
- Haryo Akbarianto Wibowo and colleagues
- Released
- November 2023
- Kind
- Reasoning in many languages
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- All public
- Measures
- Capability
- Language
- Indonesian
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
-
Llama 2-7B #238 on the capability index
- Last model added
- July 2023
- Scores refreshed
- November 20th, 2026
GPT-4 (snapshot unspecified) tops the leaderboard with 92.1% Llama 2-7B leads with 58.9%
Best score per entry from the benchmark's paper, read on November 20th, 2026. A paper’s table doesn’t add newer models. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Llama 2-7B, at 58.9%. This view shows only models we track, each at its best.
- 1 GPT-4 (snapshot unspecified) 5-shot, Monolingual standard 92.1%
- 2 ChatGPT 5-shot, Monolingual standard 76.7%
- 3 Sailor-7B 5-shot, best prompt, Cross-lingual standard 73.9%
- 4 SeaLLM-7B 5-shot, best prompt, Monolingual standard 73.2%
- 5 BLOOMZ-7B 5-shot, best prompt, Monolingual standard 66.9%
- 6 XLMR-Large Fine-tuned, Monolingual standard 64.2%
- 7 Bactrian-X-7B 5-shot, best prompt, Cross-lingual standard 63.7%
- 8 PolyLM-13B 5-shot, best prompt, Cross-lingual standard 63.3%
- 9 IndoBERT Fine-tuned, Monolingual standard 61%
- 10
Llama 2-7B 5-shot, best prompt, Cross-lingual standard 58.9%
- 11 XLMR-Base Fine-tuned, Monolingual colloquial 57.4%
- 12 mBERT Fine-tuned, Translate-test 56.5%
Source: The COPAL-ID paper
Source: The COPAL-ID paper
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-4 (snapshot unspecified) | 92.1% | 5-shot, Monolingual standard | Official leaderboard | |
| 2 | ChatGPT | 76.7% | 5-shot, Monolingual standard | Official leaderboard | |
| 3 | Sailor-7B | 73.9% | 5-shot, best prompt, Cross-lingual standard | Official leaderboard | |
| 4 | SeaLLM-7B | 73.2% | 5-shot, best prompt, Monolingual standard | Official leaderboard | |
| 5 | BLOOMZ-7B | 66.9% | 5-shot, best prompt, Monolingual standard | Official leaderboard | |
| 6 | XLMR-Large | 64.2% | Fine-tuned, Monolingual standard | Official leaderboard | |
| 7 | Bactrian-X-7B | 63.7% | 5-shot, best prompt, Cross-lingual standard | Official leaderboard | |
| 8 | PolyLM-13B | 63.3% | 5-shot, best prompt, Cross-lingual standard | Official leaderboard | |
| 9 | IndoBERT | 61% | Fine-tuned, Monolingual standard | Official leaderboard | |
| 10 | | 58.9% | 5-shot, best prompt, Cross-lingual standard | Jul 2023 | Official leaderboard |
| 11 | XLMR-Base | 57.4% | Fine-tuned, Monolingual colloquial | Official leaderboard | |
| 12 | mBERT | 56.5% | Fine-tuned, Translate-test | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 58.9% | 5-shot, best prompt, Cross-lingual standard | Jul 2023 | Official leaderboard |
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: The COPAL-ID paper
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official COPAL-ID leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5%
- MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87%
- CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7%
- Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
- MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%
Data audit
9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 3 years ago, in July 2023, for Llama 2-7B. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
- Pass Independent runs Scores from the benchmark’s paper (12 entries).
- Pass Models scored 1 model we track, plus 11 other entries on its leaderboard.
- Warning Newest score The newest came 3 years ago, in July 2023, for Llama 2-7B.
- Note Frontier models None of the top 10 has a score. The strongest tested is Llama 2-7B (#238).
- Pass Headroom The best is 58.9%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on arxiv.org: 4 rows for models we track and 11 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.