NormAd-Eti
Models decide whether behavior in an everyday story is socially acceptable in the stated country. Questions vary how much cultural guidance the model receives.
- Made by
- Akhila Yerukola and colleagues
- Released
- April 2024
- Kind
- Reasoning in many languages
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
- GPT-3.5 Turbo Jan 2024 #200 on the capability index
- Last model added
- February 2024
- Scores refreshed
- November 20th, 2026
gpt-4 (snapshot unspecified) tops the leaderboard with 87.6% Mistral-7B-Instruct-v0.2 leads with 81.8% among models released by February 2024
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Mistral-7B-Instruct-v0.2, at 81.8%. This view shows only models we track, each at its best.
The newest model with a score here came out in February 2024, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 87.6 gpt-4 (snapshot unspecified)
- 81.8
Mistral-7B-Instruct-v0.2
- 75.3 olmo-7b-sft
- 71.4 meta-llama/Llama-2-70b-chat-hf
- 71.0 meta-llama/Llama-2-13b-chat-hf
- 63.5
OLMo-7B-Instruct
- 63.1 ContextualAI/archangel_sft-kto_llama30b
- 62.6 GPT-3.5 Turbo
- 57.7 ContextualAI/archangel_sft-dpo_llama30b
- 49.0 meta-llama/Llama-2-7b-chat-hf
- 41.0 ContextualAI/archangel_sft_llama30b
- 39.7 ContextualAI/archangel_sft-kto_llama7b
- 39.0 ContextualAI/archangel_sft-dpo_llama7b
- 36.6 ContextualAI/archangel_sft-kto_llama13b
- 36.2 ContextualAI/archangel_sft_llama13b
- 33.2 ContextualAI/archangel_sft-ppo_llama7b
- 32.4 ContextualAI/archangel_sft_llama7b
- 31.2 ContextualAI/archangel_sft-ppo_llama13b
- 31.1 ContextualAI/archangel_sft-dpo_llama13b
- 30.9 ContextualAI/archangel_sft-ppo_llama30b
- 1 gpt-4 (snapshot unspecified) Rule of thumb 87.6%
- 2
Mistral-7B-Instruct-v0.2 Rule of thumb 81.8%
- 3 olmo-7b-sft Rule of thumb 75.3%
- 4 meta-llama/Llama-2-70b-chat-hf Rule of thumb 71.4%
- 5 meta-llama/Llama-2-13b-chat-hf Rule of thumb 71%
- 6
OLMo-7B-Instruct Rule of thumb 63.5%
- 7 ContextualAI/archangel_sft-kto_llama30b Rule of thumb 63.1%
- 8 GPT-3.5 Turbo Jan 2024, Country and value 62.6%
- 9 ContextualAI/archangel_sft-dpo_llama30b Value 57.7%
- 10 meta-llama/Llama-2-7b-chat-hf Country 49%
- 11 ContextualAI/archangel_sft_llama30b Value 41%
- 12 ContextualAI/archangel_sft-kto_llama7b Value 39.7%
- 13 ContextualAI/archangel_sft-dpo_llama7b Value 39%
- 14 ContextualAI/archangel_sft-kto_llama13b Country and value 36.6%
- 15 ContextualAI/archangel_sft_llama13b Value 36.2%
- 16 ContextualAI/archangel_sft-ppo_llama7b No context 33.2%
- 17 ContextualAI/archangel_sft_llama7b Country and value 32.4%
- 18 ContextualAI/archangel_sft-ppo_llama13b No context 31.2%
- 19 ContextualAI/archangel_sft-dpo_llama13b No context 31.1%
- 20 ContextualAI/archangel_sft-ppo_llama30b No context 30.9%
Source: Official NormAd-Eti leaderboard
Source: Official NormAd-Eti leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | gpt-4 (snapshot unspecified) | 87.6% | Rule of thumb | Official leaderboard | |
| 2 | | 81.8% | Rule of thumb | Dec 2023 | Official leaderboard |
| 3 | olmo-7b-sft | 75.3% | Rule of thumb | Official leaderboard | |
| 4 | meta-llama/Llama-2-70b-chat-hf | 71.4% | Rule of thumb | Official leaderboard | |
| 5 | meta-llama/Llama-2-13b-chat-hf | 71% | Rule of thumb | Official leaderboard | |
| 6 | | 63.5% | Rule of thumb | Feb 2024 | Official leaderboard |
| 7 | ContextualAI/archangel_sft-kto_llama30b | 63.1% | Rule of thumb | Official leaderboard | |
| 8 | GPT-3.5 Turbo Jan 2024 | 62.6% | Country and value | Jan 2024 | Official leaderboard |
| 9 | ContextualAI/archangel_sft-dpo_llama30b | 57.7% | Value | Official leaderboard | |
| 10 | meta-llama/Llama-2-7b-chat-hf | 49% | Country | Official leaderboard | |
| 11 | ContextualAI/archangel_sft_llama30b | 41% | Value | Official leaderboard | |
| 12 | ContextualAI/archangel_sft-kto_llama7b | 39.7% | Value | Official leaderboard | |
| 13 | ContextualAI/archangel_sft-dpo_llama7b | 39% | Value | Official leaderboard | |
| 14 | ContextualAI/archangel_sft-kto_llama13b | 36.6% | Country and value | Official leaderboard | |
| 15 | ContextualAI/archangel_sft_llama13b | 36.2% | Value | Official leaderboard | |
| 16 | ContextualAI/archangel_sft-ppo_llama7b | 33.2% | No context | Official leaderboard | |
| 17 | ContextualAI/archangel_sft_llama7b | 32.4% | Country and value | Official leaderboard | |
| 18 | ContextualAI/archangel_sft-ppo_llama13b | 31.2% | No context | Official leaderboard | |
| 19 | ContextualAI/archangel_sft-dpo_llama13b | 31.1% | No context | Official leaderboard | |
| 20 | ContextualAI/archangel_sft-ppo_llama30b | 30.9% | No context | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 81.8% | Rule of thumb | Dec 2023 | Official leaderboard |
| 2 | | 63.5% | Rule of thumb | Feb 2024 | Official leaderboard |
| 3 | GPT-3.5 Turbo Jan 2024 | 62.6% | Country and value | Jan 2024 | Official leaderboard |
Rankings by part
We read 98 parts from the benchmark’s own leaderboard in 3 groups: country, subaxis and gold label. Each ranks models on one part of the test. Mistral-7B-Instruct-v0.2 leads 93 of them, and 2 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
egypt
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 82.9%
- 2
OLMo-7B-Instruct 65.7%
- 3 GPT-3.5 Turbo Jan 2024 62.9%
lebanon
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 76.9%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 69.2%
- 3
OLMo-7B-Instruct ↓1 50%
zimbabwe
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 85.3%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 67.6%
- 3
OLMo-7B-Instruct ↓1 55.9%
philippines
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 84.4%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 65.6%
- 3
OLMo-7B-Instruct ↓1 65.6%
sudan
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 77.1%
- 2
OLMo-7B-Instruct 62.5%
- 3 GPT-3.5 Turbo Jan 2024 52.1%
spain
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 73.5%
- 2
OLMo-7B-Instruct 73.5%
- 3 GPT-3.5 Turbo Jan 2024 67.6%
tonga
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 85.7%
- 2
OLMo-7B-Instruct 71.4%
- 3 GPT-3.5 Turbo Jan 2024 57.1%
hong kong
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 80%
- 2
OLMo-7B-Instruct 74.3%
- 3 GPT-3.5 Turbo Jan 2024 65.7%
colombia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 68.8%
- 2
OLMo-7B-Instruct 59.4%
- 3 GPT-3.5 Turbo Jan 2024 53.1%
vietnam
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 80.6%
- 2
OLMo-7B-Instruct 63.9%
- 3 GPT-3.5 Turbo Jan 2024 58.3%
north macedonia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 79.4%
- 2
OLMo-7B-Instruct 52.9%
- 3 GPT-3.5 Turbo Jan 2024 50%
italy
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 88.2%
- 2
OLMo-7B-Instruct 73.5%
- 3 GPT-3.5 Turbo Jan 2024 61.8%
canada
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 87.5%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 75%
- 3
OLMo-7B-Instruct ↓1 59.4%
iraq
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 61.8%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 61.8%
- 3
OLMo-7B-Instruct ↓1 52.9%
united kingdom
3 models we track have a score here.
- 1
OLMo-7B-Instruct ↑1 81.5%
- 2
Mistral-7B-Instruct-v0.2 ↓1 74.1%
- 3 GPT-3.5 Turbo Jan 2024 59.3%
singapore
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 83.9%
- 2
OLMo-7B-Instruct 67.7%
- 3 GPT-3.5 Turbo Jan 2024 58.1%
timor-leste
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 75%
- 2
OLMo-7B-Instruct 63.9%
- 3 GPT-3.5 Turbo Jan 2024 55.6%
poland
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 82.9%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 68.6%
- 3
OLMo-7B-Instruct ↓1 65.7%
pakistan
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 77.1%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 68.6%
- 3
OLMo-7B-Instruct ↓1 65.7%
mexico
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 85.3%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 64.7%
- 3
OLMo-7B-Instruct ↓1 58.8%
argentina
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 93.9%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 66.7%
- 3
OLMo-7B-Instruct ↓1 66.7%
taiwan
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 84.4%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 71.9%
- 3
OLMo-7B-Instruct ↓1 62.5%
somalia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 84.4%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 62.2%
- 3
OLMo-7B-Instruct ↓1 62.2%
serbia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 84.6%
- 2
OLMo-7B-Instruct 69.2%
- 3 GPT-3.5 Turbo Jan 2024 59%
sweden
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 79.4%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 61.8%
- 3
OLMo-7B-Instruct ↓1 61.8%
south korea
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 81.5%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 66.7%
- 3
OLMo-7B-Instruct ↓1 66.7%
myanmar
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 82.2%
- 2
OLMo-7B-Instruct 73.3%
- 3 GPT-3.5 Turbo Jan 2024 66.7%
malta
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 88.9%
- 2
OLMo-7B-Instruct 61.1%
- 3 GPT-3.5 Turbo Jan 2024 58.3%
china
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 80.6%
- 2
OLMo-7B-Instruct 75%
- 3 GPT-3.5 Turbo Jan 2024 63.9%
germany
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 83.7%
- 2
OLMo-7B-Instruct 72.1%
- 3 GPT-3.5 Turbo Jan 2024 58.1%
ukraine
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 82.9%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 61%
- 3
OLMo-7B-Instruct ↓1 61%
romania
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 80.6%
- 2
OLMo-7B-Instruct 66.7%
- 3 GPT-3.5 Turbo Jan 2024 61.1%
russia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 85.7%
- 2
OLMo-7B-Instruct 65.7%
- 3 GPT-3.5 Turbo Jan 2024 54.3%
nepal
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 78.4%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 62.2%
- 3
OLMo-7B-Instruct ↓1 62.2%
bangladesh
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 90.9%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 63.6%
- 3
OLMo-7B-Instruct ↓1 51.5%
portugal
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 82.2%
- 2
OLMo-7B-Instruct 62.2%
- 3 GPT-3.5 Turbo Jan 2024 51.1%
thailand
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 82.1%
- 2
OLMo-7B-Instruct 69.2%
- 3 GPT-3.5 Turbo Jan 2024 48.7%
france
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 82.4%
- 2
OLMo-7B-Instruct 64.7%
- 3 GPT-3.5 Turbo Jan 2024 52.9%
ireland
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 85.4%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 68.3%
- 3
OLMo-7B-Instruct ↓1 63.4%
croatia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 80%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 60%
- 3
OLMo-7B-Instruct ↓1 57.1%
mauritius
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 80%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 57.5%
- 3
OLMo-7B-Instruct ↓1 52.5%
fiji
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 100%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 85.7%
- 3
OLMo-7B-Instruct ↓1 66.7%
austria
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 75.8%
- 2
OLMo-7B-Instruct 72.7%
- 3 GPT-3.5 Turbo Jan 2024 66.7%
brazil
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 81.3%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 56.3%
- 3
OLMo-7B-Instruct ↓1 53.1%
south africa
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 80.6%
- 2
OLMo-7B-Instruct 66.7%
- 3 GPT-3.5 Turbo Jan 2024 63.9%
chile
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 77.1%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 60%
- 3
OLMo-7B-Instruct ↓1 57.1%
papua new guinea
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 100%
- 2
OLMo-7B-Instruct 76.5%
- 3 GPT-3.5 Turbo Jan 2024 67.6%
afghanistan
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 75.6%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 68.3%
- 3
OLMo-7B-Instruct ↓1 56.1%
japan
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 82.9%
- 2
OLMo-7B-Instruct 82.9%
- 3 GPT-3.5 Turbo Jan 2024 68.6%
australia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 88%
- 2
OLMo-7B-Instruct 72%
- 3 GPT-3.5 Turbo Jan 2024 68%
netherlands
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 86.1%
- 2
OLMo-7B-Instruct 83.3%
- 3 GPT-3.5 Turbo Jan 2024 61.1%
sri lanka
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 77.1%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 65.7%
- 3
OLMo-7B-Instruct ↓1 57.1%
indonesia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 80.8%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 65.4%
- 3
OLMo-7B-Instruct ↓1 61.5%
malaysia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 79.4%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 70.6%
- 3
OLMo-7B-Instruct ↓1 50%
ethiopia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 68.9%
- 2
OLMo-7B-Instruct 60%
- 3 GPT-3.5 Turbo Jan 2024 57.8%
bosnia and herzegovina
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 79.4%
- 2
OLMo-7B-Instruct 64.7%
- 3 GPT-3.5 Turbo Jan 2024 58.8%
palestinian territories
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 82.4%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 76.5%
- 3
OLMo-7B-Instruct ↓1 61.8%
syria
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 77.8%
- 2
OLMo-7B-Instruct 61.1%
- 3 GPT-3.5 Turbo Jan 2024 44.4%
kenya
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 84.8%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 63.6%
- 3
OLMo-7B-Instruct ↓1 57.6%
new zealand
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 83.3%
- 2
OLMo-7B-Instruct 73.8%
- 3 GPT-3.5 Turbo Jan 2024 71.4%
peru
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 87.5%
- 2
OLMo-7B-Instruct 65.6%
- 3 GPT-3.5 Turbo Jan 2024 56.3%
laos
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 84.8%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 66.7%
- 3
OLMo-7B-Instruct ↓1 54.5%
cyprus
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 80%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 65.7%
- 3
OLMo-7B-Instruct ↓1 60%
india
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 79.3%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 65.5%
- 3
OLMo-7B-Instruct ↓1 55.2%
iran
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 81.8%
- 2
OLMo-7B-Instruct 63.6%
- 3 GPT-3.5 Turbo Jan 2024 61.4%
greece
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 85.3%
- 2
OLMo-7B-Instruct 64.7%
- 3 GPT-3.5 Turbo Jan 2024 58.8%
venezuela
3 models we track have a score here.
- 1 GPT-3.5 Turbo Jan 2024 ↑2 70.6%
- 2
Mistral-7B-Instruct-v0.2 ↓1 67.6%
- 3
OLMo-7B-Instruct ↓1 64.7%
saudi arabia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 79.5%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 63.6%
- 3
OLMo-7B-Instruct ↓1 56.8%
israel
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 81%
- 2
OLMo-7B-Instruct 66.7%
- 3 GPT-3.5 Turbo Jan 2024 61.9%
hungary
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 83.3%
- 2
OLMo-7B-Instruct 61.1%
- 3 GPT-3.5 Turbo Jan 2024 52.8%
samoa
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 86.1%
- 2
OLMo-7B-Instruct 66.7%
- 3 GPT-3.5 Turbo Jan 2024 61.1%
türkiye
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 91.4%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 68.6%
- 3
OLMo-7B-Instruct ↓1 65.7%
united states of america
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 88.1%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 69%
- 3
OLMo-7B-Instruct ↓1 66.7%
cambodia
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 80.6%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 66.7%
- 3
OLMo-7B-Instruct ↓1 50%
south sudan
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 85.2%
- 2
OLMo-7B-Instruct 66.7%
- 3 GPT-3.5 Turbo Jan 2024 63%
basic etiquette
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 82.9%
- 2
OLMo-7B-Instruct 64%
- 3 GPT-3.5 Turbo Jan 2024 60.6%
visiting
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 79.3%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 64.3%
- 3
OLMo-7B-Instruct ↓1 62.1%
eating
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 81.8%
- 2
OLMo-7B-Instruct 68%
- 3 GPT-3.5 Turbo Jan 2024 61.6%
gift giving
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 80.6%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 61.3%
- 3
OLMo-7B-Instruct ↓1 58.1%
gifts
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 89.7%
- 2
OLMo-7B-Instruct 69%
- 3 GPT-3.5 Turbo Jan 2024 64.4%
offering and complimenting items
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 85.4%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 63.4%
- 3
OLMo-7B-Instruct ↓1 61%
manners in vietnam
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 100%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 66.7%
- 3
OLMo-7B-Instruct ↓1 66.7%
visiting and eating
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 75%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 75%
- 3
OLMo-7B-Instruct ↓1 62.5%
tipping
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 87.5%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 75%
- 3
OLMo-7B-Instruct ↓1 68.8%
drinking
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 88.9%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 66.7%
- 3
OLMo-7B-Instruct ↓1 66.7%
cleanliness
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 88.9%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 66.7%
- 3
OLMo-7B-Instruct ↓1 66.7%
direct manners
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 100%
- 2
OLMo-7B-Instruct 87.5%
- 3 GPT-3.5 Turbo Jan 2024 75%
toasting
3 models we track have a score here.
- 1 GPT-3.5 Turbo Jan 2024 ↑2 100%
- 2
Mistral-7B-Instruct-v0.2 ↓1 83.3%
- 3
OLMo-7B-Instruct ↓1 66.7%
pub etiquette
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 100%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 100%
- 3
OLMo-7B-Instruct ↓1 83.3%
visiting a village
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 100%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 100%
- 3
OLMo-7B-Instruct ↓1 66.7%
drinking coffee
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 77.8%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 66.7%
- 3
OLMo-7B-Instruct ↓1 55.6%
eating out
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 100%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 100%
- 3
OLMo-7B-Instruct ↓1 100%
māori etiquette
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 77.8%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 66.7%
- 3
OLMo-7B-Instruct ↓1 55.6%
‘taarof’ (politeness and mutual respect)
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 77.8%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 66.7%
- 3
OLMo-7B-Instruct ↓1 66.7%
religious dietary laws
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 88.9%
- 2
OLMo-7B-Instruct 88.9%
- 3 GPT-3.5 Turbo Jan 2024 66.7%
yes
3 models we track have a score here.
- 1
Mistral-7B-Instruct-v0.2 95.2%
- 2 GPT-3.5 Turbo Jan 2024 ↑1 90.9%
- 3
OLMo-7B-Instruct ↓1 81.3%
no
3 models we track have a score here.
- 1 GPT-3.5 Turbo Jan 2024 ↑2 90.5%
- 2
Mistral-7B-Instruct-v0.2 ↓1 87.8%
- 3
OLMo-7B-Instruct ↓1 34.9%
neutral
3 models we track have a score here.
- 1
OLMo-7B-Instruct ↑1 89.1%
- 2
Mistral-7B-Instruct-v0.2 ↓1 60%
- 3 GPT-3.5 Turbo Jan 2024 0%
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official NormAd-Eti leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official NormAd-Eti leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5%
- MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87%
- CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7%
- Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
- MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%
Data audit
9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 2 years ago, in February 2024, for OLMo-7B-Instruct. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
- Pass Independent runs Scores from the official leaderboard (20 entries).
- Pass Models scored 3 models we track, plus 17 other entries on its leaderboard.
- Warning Newest score The newest came 2 years ago, in February 2024, for OLMo-7B-Instruct.
- Note Frontier models None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#200).
- Pass Headroom The best is 81.8%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on github.com: 11 rows for models we track and 17 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.