Icelandic WinoGrande
Models complete Icelandic sentences by choosing which referent fits the context. The evaluation uses an Icelandic adaptation of WinoGrande.
- Made by
- Vésteinn Snæbjarnarson and colleagues
- Released
- June 2022
- Kind
- Reasoning in many languages
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- All public
- Measures
- Capability
- Language
- Icelandic
- Interaction
- Single turn
- Items from
- Another benchmark, Translated
- Refresh
- One fixed set
- Run by
- Anyone
- Based on
- WinoGrande
- Strongest model tested
- Claude Fable 5 #12 on the capability index
- Last model added
- August 2026
- Scores refreshed
- November 20th, 2026
Gemini 3.7 Flash leads with 96.3% among models released by August 2026 Gemini 3.7 Flash leads with 96.3% among models released by August 2026
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
The newest model with a score here came out in August 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 13 more
- 96.3
Gemini 3.7 Flash
- 96.1
Gemini 3.1 Pro
- 96.0 Claude Fable 5
- 96.0
Gemini 3.5 Flash
- 95.6
Gemini 3 Pro
- 95.6
Gemini 3 Flash
- 95.6 GPT-5.5
- 95.5
Gemini 3.6 Flash
- 94.9 Claude Opus 4.8
- 94.7 Claude Opus 4.5
- 94.6 Claude Opus 4.6
- 94.5 GPT-5.6 Sol
- 94.4 GPT-5
- 93.7 GPT-5.4
- 92.8 o1-preview
- 92.8
Gemini 2.5 Pro
- 92.8 GPT-5.1
- 92.8 Claude Opus 4.7
- 92.7 Claude Sonnet 4.5
- 92.2 GPT-5.2
- 92.1
Gemini 3.1 Flash-Lite
- 92.0
Gemma 4 31B IT
- 91.8
Gemini 2.5 Flash
- 91.8 Claude Sonnet 4.6
- 91.8 GPT-5.6 Terra
- 91.6
DeepSeek-V4-Pro
- 91.3 Claude 3.5 Sonnet
- 91.2 GLM-5.2
- 90.3 Grok 4.3
- 90.3 Claude Sonnet 4
- 90.3
Qwen3.5 397B-A17B
- 90.1
Gemini 2.0 Pro
- 89.7 Claude 3.7 Sonnet
- 89.7 Kimi K2.5
- 88.4
Gemma 4 26B A4B
- 87.5 GPT-5 mini
- 87.4 GPT-4.1
- 87.0 Claude 3 Opus
- 86.0 GPT-5.6 Luna
- 85.8 GPT-4 Turbo
- 85.4 GPT-4o
- 84.4
Gemini 2.0 Flash
- 83.7 o4-mini
- 83.5 GPT-4
- 83.5 GPT-4o
- 82.7 cyboghostginx/gemma-4-31B-it-Adetayo
- 81.4
DeepSeek-R1
- 81.2 GPT-4
- 77.8 o3-mini
- 77.8
MiniMax-M2.5
- 75.9
DeepSeek-V3
- 75.3
DeepSeek-R1-Distill-Llama-70B
- 75.3
Gemini 2.5 Flash-Lite
- 74.6
Mistral Large 3
- 72.6 o1-mini
- 71.8 cyboghostginx/gemma-3-27b-it-Adetayo
- 71.7 GPT-5 nano
- 70.9 GPT-4.1 mini
- 70.6 NousResearch/Hermes-3-Llama-3.1-405B-FP8
- 70.4
Gemma 3 27B
- 70.1 meta-llama/Meta-Llama-3.1-70B-Instruct
- 70.0
Llama-3.1-Nemotron-70B-Instruct
- 68.7
Llama 3.3 70B Instruct
- 68.3 mistral/mistral-medium-2508
- 65.3 GPT-4o mini
- 65.0
Gemma 3 12B
- 64.4 Claude 3.5 Haiku
- 61.4
Qwen2-72B
- 60.0 meta-llama/Meta-Llama-3-70B-Instruct
- 56.9 Sigurdur/icebreaker
- 56.5 AI-Sweden-Models/gpt-sw3-356m-instruct
- 55.1 mistralai/Mistral-Nemo-Instruct-2407
- 55.1 Claude 2.1
- 54.7 mistral/ministral-3b-2410
- 54.3 LumiOpen/Viking-33B
- 53.9 AI-Sweden-Models/gpt-sw3-126m-instruct
- 52.9
Gemma 3 1B
- 52.9 google/gemma-2-9b-it
- 52.5
Llama 3.1-8B
- 52.1 AI-Sweden-Models/gpt-sw3-1.3b-instruct
- 52.1
Gemma 3 4B
- 52.0 cyboghostginx/gemma-3-4b-it-Adetayo
- 52.0 GPT-3.5 Turbo
- 52.0 GPT-4.1 nano
- 51.9 meta-llama/Meta-Llama-3.1-8B-Instruct
- 50.9 meta-llama/Llama-3.2-1B-Instruct
- 49.8 AI-Sweden-Models/gpt-sw3-20b-instruct
- 49.7 meta-llama/Llama-3.2-3B-Instruct
- 49.6
Qwen2-7B-Instruct
- 49.4 AI-Sweden-Models/gpt-sw3-6.7b-v2-instruct
- 49.1 LumiOpen/Viking-13B
- 49.1 meta-llama/Meta-Llama-3-8B-Instruct
- 49.0 LumiOpen/Viking-7B
- 1
Gemini 3.7 Flash 3-shot · thinking high 96.3%
- 2
Gemini 3.1 Pro 3-shot · thinking high 96.1%
- 3 Claude Fable 5 3-shot · thinking high 96%
- 4
Gemini 3.5 Flash 3-shot · thinking high 96%
- 5
Gemini 3 Pro 3-shot · thinking high 95.6%
- 6
Gemini 3 Flash 3-shot · thinking high 95.6%
- 7 GPT-5.5 3-shot · thinking high 95.6%
- 8
Gemini 3.6 Flash 3-shot · thinking high 95.5%
- 9 Claude Opus 4.8 3-shot · thinking high 94.9%
- 10 Claude Opus 4.5 3-shot · thinking high 94.7%
- 11 Claude Opus 4.6 3-shot · thinking high 94.6%
- 12 GPT-5.6 Sol 3-shot · thinking high 94.5%
- 13 GPT-5 3-shot · thinking high 94.4%
- 14 GPT-5.4 3-shot · thinking high 93.7%
- 15 o1-preview 3-shot · thinking on 92.8%
- 16
Gemini 2.5 Pro Jun 2025, 3-shot · thinking on 92.8%
- 17 GPT-5.1 3-shot · thinking high 92.8%
- 18 Claude Opus 4.7 3-shot · thinking high 92.8%
- 19 Claude Sonnet 4.5 3-shot · thinking high 92.7%
- 20 GPT-5.2 3-shot · thinking high 92.2%
- 21
Gemini 3.1 Flash-Lite 3-shot · thinking high 92.1%
- 22
Gemma 4 31B IT 3-shot · thinking on 92%
- 23
Gemini 2.5 Flash Sep 2025, 3-shot · thinking high 91.8%
- 24 Claude Sonnet 4.6 3-shot · thinking high 91.8%
- 25 GPT-5.6 Terra 3-shot · thinking high 91.8%
- 26
DeepSeek-V4-Pro 3-shot · thinking on 91.6%
- 27 Claude 3.5 Sonnet 3-shot 91.3%
- 28 GLM-5.2 3-shot · thinking on 91.2%
- 29 Grok 4.3 3-shot · thinking on 90.3%
- 30 Claude Sonnet 4 3-shot · thinking on 90.3%
- 31
Qwen3.5 397B-A17B 3-shot · thinking on 90.3%
- 32
Gemini 2.0 Pro 3-shot · thinking on 90.1%
- 33 Claude 3.7 Sonnet 3-shot 89.7%
- 34 Kimi K2.5 3-shot · thinking on 89.7%
- 35
Gemma 4 26B A4B 3-shot · thinking on 88.4%
- 36 GPT-5 mini 3-shot · thinking high 87.5%
- 37 GPT-4.1 3-shot 87.4%
- 38 Claude 3 Opus 3-shot 87%
- 39 GPT-5.6 Luna 3-shot · thinking high 86%
- 40 GPT-4 Turbo Apr 2024, 3-shot 85.8%
- 41 GPT-4o Aug 2024, 3-shot 85.4%
- 42
Gemini 2.0 Flash Feb 2025, 3-shot 84.4%
- 43 o4-mini 3-shot · thinking on 83.7%
- 44 GPT-4 Jun 2023, 3-shot 83.5%
- 45 GPT-4o May 2024, 3-shot 83.5%
- 46 cyboghostginx/gemma-4-31B-it-Adetayo 3-shot 82.7%
- 47
DeepSeek-R1 May 2025, 3-shot · thinking on 81.4%
- 48 GPT-4 Mar 2023, 3-shot 81.2%
- 49 o3-mini 3-shot · thinking on 77.8%
- 50
MiniMax-M2.5 3-shot · thinking on 77.8%
- 51
DeepSeek-V3 Mar 2025, 3-shot 75.9%
- 52
DeepSeek-R1-Distill-Llama-70B 3-shot · thinking on 75.3%
- 53
Gemini 2.5 Flash-Lite Sep 2025, 3-shot · thinking high 75.3%
- 54
Mistral Large 3 3-shot 74.6%
- 55 o1-mini 3-shot · thinking on 72.6%
- 56 cyboghostginx/gemma-3-27b-it-Adetayo 3-shot 71.8%
- 57 GPT-5 nano 3-shot · thinking high 71.7%
- 58 GPT-4.1 mini 3-shot 70.9%
- 59 NousResearch/Hermes-3-Llama-3.1-405B-FP8 3-shot 70.6%
- 60
Gemma 3 27B 3-shot 70.4%
- 61 meta-llama/Meta-Llama-3.1-70B-Instruct 3-shot 70.1%
- 62
Llama-3.1-Nemotron-70B-Instruct 3-shot 70%
- 63
Llama 3.3 70B Instruct 3-shot 68.7%
- 64 mistral/mistral-medium-2508 3-shot 68.3%
- 65 GPT-4o mini 3-shot 65.3%
- 66
Gemma 3 12B 3-shot 65%
- 67 Claude 3.5 Haiku 3-shot 64.4%
- 68
Qwen2-72B 3-shot 61.4%
- 69 meta-llama/Meta-Llama-3-70B-Instruct 3-shot 60%
- 70 Sigurdur/icebreaker 3-shot 56.9%
- 71 AI-Sweden-Models/gpt-sw3-356m-instruct 3-shot 56.5%
- 72 mistralai/Mistral-Nemo-Instruct-2407 3-shot 55.1%
- 73 Claude 2.1 3-shot 55.1%
- 74 mistral/ministral-3b-2410 3-shot 54.7%
- 75 LumiOpen/Viking-33B 3-shot 54.3%
- 76 AI-Sweden-Models/gpt-sw3-126m-instruct 3-shot 53.9%
- 77
Gemma 3 1B 3-shot 52.9%
- 78 google/gemma-2-9b-it 3-shot 52.9%
- 79
Llama 3.1-8B 3-shot 52.5%
- 80 AI-Sweden-Models/gpt-sw3-1.3b-instruct 3-shot 52.1%
- 81
Gemma 3 4B 3-shot 52.1%
- 82 cyboghostginx/gemma-3-4b-it-Adetayo 3-shot 52%
- 83 GPT-3.5 Turbo Jan 2024, 3-shot 52%
- 84 GPT-4.1 nano 3-shot 52%
- 85 meta-llama/Meta-Llama-3.1-8B-Instruct 3-shot 51.9%
- 86 meta-llama/Llama-3.2-1B-Instruct 3-shot 50.9%
- 87 AI-Sweden-Models/gpt-sw3-20b-instruct 3-shot 49.8%
- 88 meta-llama/Llama-3.2-3B-Instruct 3-shot 49.7%
- 89
Qwen2-7B-Instruct 3-shot 49.6%
- 90 AI-Sweden-Models/gpt-sw3-6.7b-v2-instruct 3-shot 49.4%
- 91 LumiOpen/Viking-13B 3-shot 49.1%
- 92 meta-llama/Meta-Llama-3-8B-Instruct 3-shot 49.1%
- 93 LumiOpen/Viking-7B 3-shot 49%
Source: Official Icelandic WinoGrande leaderboard
- 96.3
Gemini 3.7 Flash
- 96.1
Gemini 3.1 Pro
- 96.0 Claude Fable 5
- 96.0
Gemini 3.5 Flash
- 95.6
Gemini 3 Pro
- 95.6
Gemini 3 Flash
- 95.6 GPT-5.5
- 95.5
Gemini 3.6 Flash
- 94.9 Claude Opus 4.8
- 94.7 Claude Opus 4.5
- 94.6 Claude Opus 4.6
- 94.5 GPT-5.6 Sol
- 94.4 GPT-5
- 93.7 GPT-5.4
- 92.8 o1-preview
- 92.8
Gemini 2.5 Pro
- 92.8 GPT-5.1
- 92.8 Claude Opus 4.7
- 92.7 Claude Sonnet 4.5
- 92.2 GPT-5.2
- 92.1
Gemini 3.1 Flash-Lite
- 92.0
Gemma 4 31B IT
- 91.8
Gemini 2.5 Flash
- 91.8 Claude Sonnet 4.6
- 91.8 GPT-5.6 Terra
- 91.6
DeepSeek-V4-Pro
- 91.3 Claude 3.5 Sonnet
- 91.2 GLM-5.2
- 90.3 Grok 4.3
- 90.3 Claude Sonnet 4
- 90.3
Qwen3.5 397B-A17B
- 90.1
Gemini 2.0 Pro
- 89.7 Claude 3.7 Sonnet
- 89.7 Kimi K2.5
- 88.4
Gemma 4 26B A4B
- 87.5 GPT-5 mini
- 87.4 GPT-4.1
- 87.0 Claude 3 Opus
- 86.0 GPT-5.6 Luna
- 85.8 GPT-4 Turbo
- 85.4 GPT-4o
- 84.4
Gemini 2.0 Flash
- 83.7 o4-mini
- 83.5 GPT-4
- 83.5 GPT-4o
- 81.4
DeepSeek-R1
- 81.2 GPT-4
- 77.8 o3-mini
- 77.8
MiniMax-M2.5
- 75.9
DeepSeek-V3
- 75.3
DeepSeek-R1-Distill-Llama-70B
- 75.3
Gemini 2.5 Flash-Lite
- 74.6
Mistral Large 3
- 72.6 o1-mini
- 71.7 GPT-5 nano
- 70.9 GPT-4.1 mini
- 70.4
Gemma 3 27B
- 70.0
Llama-3.1-Nemotron-70B-Instruct
- 68.7
Llama 3.3 70B Instruct
- 65.3 GPT-4o mini
- 65.0
Gemma 3 12B
- 64.4 Claude 3.5 Haiku
- 61.4
Qwen2-72B
- 55.1 Claude 2.1
- 52.9
Gemma 3 1B
- 52.5
Llama 3.1-8B
- 52.1
Gemma 3 4B
- 52.0 GPT-3.5 Turbo
- 52.0 GPT-4.1 nano
- 49.6
Qwen2-7B-Instruct
- 1
Gemini 3.7 Flash 3-shot · thinking high 96.3%
- 2
Gemini 3.1 Pro 3-shot · thinking high 96.1%
- 3 Claude Fable 5 3-shot · thinking high 96%
- 4
Gemini 3.5 Flash 3-shot · thinking high 96%
- 5
Gemini 3 Pro 3-shot · thinking high 95.6%
- 6
Gemini 3 Flash 3-shot · thinking high 95.6%
- 7 GPT-5.5 3-shot · thinking high 95.6%
- 8
Gemini 3.6 Flash 3-shot · thinking high 95.5%
- 9 Claude Opus 4.8 3-shot · thinking high 94.9%
- 10 Claude Opus 4.5 3-shot · thinking high 94.7%
- 11 Claude Opus 4.6 3-shot · thinking high 94.6%
- 12 GPT-5.6 Sol 3-shot · thinking high 94.5%
- 13 GPT-5 3-shot · thinking high 94.4%
- 14 GPT-5.4 3-shot · thinking high 93.7%
- 15 o1-preview 3-shot · thinking on 92.8%
- 16
Gemini 2.5 Pro Jun 2025, 3-shot · thinking on 92.8%
- 17 GPT-5.1 3-shot · thinking high 92.8%
- 18 Claude Opus 4.7 3-shot · thinking high 92.8%
- 19 Claude Sonnet 4.5 3-shot · thinking high 92.7%
- 20 GPT-5.2 3-shot · thinking high 92.2%
- 21
Gemini 3.1 Flash-Lite 3-shot · thinking high 92.1%
- 22
Gemma 4 31B IT 3-shot · thinking on 92%
- 23
Gemini 2.5 Flash Sep 2025, 3-shot · thinking high 91.8%
- 24 Claude Sonnet 4.6 3-shot · thinking high 91.8%
- 25 GPT-5.6 Terra 3-shot · thinking high 91.8%
- 26
DeepSeek-V4-Pro 3-shot · thinking on 91.6%
- 27 Claude 3.5 Sonnet 3-shot 91.3%
- 28 GLM-5.2 3-shot · thinking on 91.2%
- 29 Grok 4.3 3-shot · thinking on 90.3%
- 30 Claude Sonnet 4 3-shot · thinking on 90.3%
- 31
Qwen3.5 397B-A17B 3-shot · thinking on 90.3%
- 32
Gemini 2.0 Pro 3-shot · thinking on 90.1%
- 33 Claude 3.7 Sonnet 3-shot 89.7%
- 34 Kimi K2.5 3-shot · thinking on 89.7%
- 35
Gemma 4 26B A4B 3-shot · thinking on 88.4%
- 36 GPT-5 mini 3-shot · thinking high 87.5%
- 37 GPT-4.1 3-shot 87.4%
- 38 Claude 3 Opus 3-shot 87%
- 39 GPT-5.6 Luna 3-shot · thinking high 86%
- 40 GPT-4 Turbo Apr 2024, 3-shot 85.8%
- 41 GPT-4o Aug 2024, 3-shot 85.4%
- 42
Gemini 2.0 Flash Feb 2025, 3-shot 84.4%
- 43 o4-mini 3-shot · thinking on 83.7%
- 44 GPT-4 Jun 2023, 3-shot 83.5%
- 45 GPT-4o May 2024, 3-shot 83.5%
- 46
DeepSeek-R1 May 2025, 3-shot · thinking on 81.4%
- 47 GPT-4 Mar 2023, 3-shot 81.2%
- 48 o3-mini 3-shot · thinking on 77.8%
- 49
MiniMax-M2.5 3-shot · thinking on 77.8%
- 50
DeepSeek-V3 Mar 2025, 3-shot 75.9%
- 51
DeepSeek-R1-Distill-Llama-70B 3-shot · thinking on 75.3%
- 52
Gemini 2.5 Flash-Lite Sep 2025, 3-shot · thinking high 75.3%
- 53
Mistral Large 3 3-shot 74.6%
- 54 o1-mini 3-shot · thinking on 72.6%
- 55 GPT-5 nano 3-shot · thinking high 71.7%
- 56 GPT-4.1 mini 3-shot 70.9%
- 57
Gemma 3 27B 3-shot 70.4%
- 58
Llama-3.1-Nemotron-70B-Instruct 3-shot 70%
- 59
Llama 3.3 70B Instruct 3-shot 68.7%
- 60 GPT-4o mini 3-shot 65.3%
- 61
Gemma 3 12B 3-shot 65%
- 62 Claude 3.5 Haiku 3-shot 64.4%
- 63
Qwen2-72B 3-shot 61.4%
- 64 Claude 2.1 3-shot 55.1%
- 65
Gemma 3 1B 3-shot 52.9%
- 66
Llama 3.1-8B 3-shot 52.5%
- 67
Gemma 3 4B 3-shot 52.1%
- 68 GPT-3.5 Turbo Jan 2024, 3-shot 52%
- 69 GPT-4.1 nano 3-shot 52%
- 70
Qwen2-7B-Instruct 3-shot 49.6%
Source: Official Icelandic WinoGrande leaderboard
| # | Entry | Score | Setting | Date | Link | Source |
|---|---|---|---|---|---|---|
| 1 | | 96.3% | 3-shot · thinking high | Aug 2026 | ai.google.dev | Official leaderboard |
| 2 | | 96.1% | 3-shot · thinking high | Feb 2026 | ai.google.dev | Official leaderboard |
| 3 | Claude Fable 5 | 96% | 3-shot · thinking high | Jun 2026 | anthropic.com | Official leaderboard |
| 4 | | 96% | 3-shot · thinking high | May 2026 | ai.google.dev | Official leaderboard |
| 5 | | 95.6% | 3-shot · thinking high | Nov 2025 | ai.google.dev | Official leaderboard |
| 6 | | 95.6% | 3-shot · thinking high | Dec 2025 | ai.google.dev | Official leaderboard |
| 7 | GPT-5.5 | 95.6% | 3-shot · thinking high | Apr 2026 | developers.openai.com | Official leaderboard |
| 8 | | 95.5% | 3-shot · thinking high | Jul 2026 | ai.google.dev | Official leaderboard |
| 9 | Claude Opus 4.8 | 94.9% | 3-shot · thinking high | May 2026 | anthropic.com | Official leaderboard |
| 10 | Claude Opus 4.5 | 94.7% | 3-shot · thinking high | Nov 2025 | huggingface.co | Official leaderboard |
| 11 | Claude Opus 4.6 | 94.6% | 3-shot · thinking high | Feb 2026 | anthropic.com | Official leaderboard |
| 12 | GPT-5.6 Sol | 94.5% | 3-shot · thinking high | Jul 2026 | developers.openai.com | Official leaderboard |
| 13 | GPT-5 | 94.4% | 3-shot · thinking high | Aug 2025 | huggingface.co | Official leaderboard |
| 14 | GPT-5.4 | 93.7% | 3-shot · thinking high | Mar 2026 | huggingface.co | Official leaderboard |
| 15 | o1-preview | 92.8% | 3-shot · thinking on | Sep 2024 | huggingface.co | Official leaderboard |
| 16 | | 92.8% | 3-shot · thinking on | Jun 2025 | huggingface.co | Official leaderboard |
| 17 | GPT-5.1 | 92.8% | 3-shot · thinking high | Nov 2025 | huggingface.co | Official leaderboard |
| 18 | Claude Opus 4.7 | 92.8% | 3-shot · thinking high | Apr 2026 | anthropic.com | Official leaderboard |
| 19 | Claude Sonnet 4.5 | 92.7% | 3-shot · thinking high | Sep 2025 | huggingface.co | Official leaderboard |
| 20 | GPT-5.2 | 92.2% | 3-shot · thinking high | Dec 2025 | huggingface.co | Official leaderboard |
| 21 | | 92.1% | 3-shot · thinking high | Mar 2026 | ai.google.dev | Official leaderboard |
| 22 | | 92% | 3-shot · thinking on | Apr 2026 | huggingface.co | Official leaderboard |
| 23 | | 91.8% | 3-shot · thinking high | Sep 2025 | huggingface.co | Official leaderboard |
| 24 | Claude Sonnet 4.6 | 91.8% | 3-shot · thinking high | Feb 2026 | anthropic.com | Official leaderboard |
| 25 | GPT-5.6 Terra | 91.8% | 3-shot · thinking high | Jul 2026 | developers.openai.com | Official leaderboard |
| 26 | | 91.6% | 3-shot · thinking on | Apr 2026 | huggingface.co | Official leaderboard |
| 27 | Claude 3.5 Sonnet | 91.3% | 3-shot | Jun 2024 | huggingface.co | Official leaderboard |
| 28 | GLM-5.2 | 91.2% | 3-shot · thinking on | Jun 2026 | huggingface.co | Official leaderboard |
| 29 | Grok 4.3 | 90.3% | 3-shot · thinking on | Apr 2026 | docs.x.ai | Official leaderboard |
| 30 | Claude Sonnet 4 | 90.3% | 3-shot · thinking on | May 2025 | huggingface.co | Official leaderboard |
| 31 | | 90.3% | 3-shot · thinking on | Feb 2026 | huggingface.co | Official leaderboard |
| 32 | | 90.1% | 3-shot · thinking on | Feb 2025 | huggingface.co | Official leaderboard |
| 33 | Claude 3.7 Sonnet | 89.7% | 3-shot | Feb 2025 | huggingface.co | Official leaderboard |
| 34 | Kimi K2.5 | 89.7% | 3-shot · thinking on | Jan 2026 | kimi.com | Official leaderboard |
| 35 | | 88.4% | 3-shot · thinking on | Apr 2026 | huggingface.co | Official leaderboard |
| 36 | GPT-5 mini | 87.5% | 3-shot · thinking high | Aug 2025 | huggingface.co | Official leaderboard |
| 37 | GPT-4.1 | 87.4% | 3-shot | Apr 2025 | huggingface.co | Official leaderboard |
| 38 | Claude 3 Opus | 87% | 3-shot | Feb 2024 | huggingface.co | Official leaderboard |
| 39 | GPT-5.6 Luna | 86% | 3-shot · thinking high | Jul 2026 | developers.openai.com | Official leaderboard |
| 40 | GPT-4 Turbo Apr 2024 | 85.8% | 3-shot | Apr 2024 | huggingface.co | Official leaderboard |
| 41 | GPT-4o Aug 2024 | 85.4% | 3-shot | Aug 2024 | huggingface.co | Official leaderboard |
| 42 | | 84.4% | 3-shot | Feb 2025 | huggingface.co | Official leaderboard |
| 43 | o4-mini | 83.7% | 3-shot · thinking on | Apr 2025 | huggingface.co | Official leaderboard |
| 44 | GPT-4 Jun 2023 | 83.5% | 3-shot | Jun 2023 | huggingface.co | Official leaderboard |
| 45 | GPT-4o May 2024 | 83.5% | 3-shot | May 2024 | huggingface.co | Official leaderboard |
| 46 | cyboghostginx/gemma-4-31B-it-Adetayo 31B | 82.7% | 3-shot | huggingface.co | Official leaderboard | |
| 47 | | 81.4% | 3-shot · thinking on | May 2025 | huggingface.co | Official leaderboard |
| 48 | GPT-4 Mar 2023 | 81.2% | 3-shot | Mar 2023 | huggingface.co | Official leaderboard |
| 49 | o3-mini | 77.8% | 3-shot · thinking on | Jan 2025 | huggingface.co | Official leaderboard |
| 50 | | 77.8% | 3-shot · thinking on | Feb 2026 | minimax.io | Official leaderboard |
| 51 | | 75.9% | 3-shot | Mar 2025 | huggingface.co | Official leaderboard |
| 52 | | 75.3% | 3-shot · thinking on | Jan 2025 | huggingface.co | Official leaderboard |
| 53 | | 75.3% | 3-shot · thinking high | Sep 2025 | huggingface.co | Official leaderboard |
| 54 | | 74.6% | 3-shot | Dec 2025 | huggingface.co | Official leaderboard |
| 55 | o1-mini | 72.6% | 3-shot · thinking on | Sep 2024 | huggingface.co | Official leaderboard |
| 56 | cyboghostginx/gemma-3-27b-it-Adetayo 27B | 71.8% | 3-shot | huggingface.co | Official leaderboard | |
| 57 | GPT-5 nano | 71.7% | 3-shot · thinking high | Aug 2025 | huggingface.co | Official leaderboard |
| 58 | GPT-4.1 mini | 70.9% | 3-shot | Apr 2025 | huggingface.co | Official leaderboard |
| 59 | NousResearch/Hermes-3-Llama-3.1-405B-FP8 405.9B | 70.6% | 3-shot | huggingface.co | Official leaderboard | |
| 60 | | 70.4% | 3-shot | Mar 2025 | huggingface.co | Official leaderboard |
| 61 | meta-llama/Meta-Llama-3.1-70B-Instruct 70.6B | 70.1% | 3-shot | huggingface.co | Official leaderboard | |
| 62 | | 70% | 3-shot | Oct 2024 | huggingface.co | Official leaderboard |
| 63 | | 68.7% | 3-shot | Dec 2024 | huggingface.co | Official leaderboard |
| 64 | mistral/mistral-medium-2508 | 68.3% | 3-shot | huggingface.co | Official leaderboard | |
| 65 | GPT-4o mini | 65.3% | 3-shot | Jul 2024 | huggingface.co | Official leaderboard |
| 66 | | 65% | 3-shot | Mar 2025 | huggingface.co | Official leaderboard |
| 67 | Claude 3.5 Haiku | 64.4% | 3-shot | Oct 2024 | huggingface.co | Official leaderboard |
| 68 | | 61.4% | 3-shot | Jun 2024 | huggingface.co | Official leaderboard |
| 69 | meta-llama/Meta-Llama-3-70B-Instruct 70.6B | 60% | 3-shot | huggingface.co | Official leaderboard | |
| 70 | Sigurdur/icebreaker 0.1B | 56.9% | 3-shot | huggingface.co | Official leaderboard | |
| 71 | AI-Sweden-Models/gpt-sw3-356m-instruct 0.5B | 56.5% | 3-shot | huggingface.co | Official leaderboard | |
| 72 | mistralai/Mistral-Nemo-Instruct-2407 12.3B | 55.1% | 3-shot | huggingface.co | Official leaderboard | |
| 73 | Claude 2.1 | 55.1% | 3-shot | Nov 2023 | huggingface.co | Official leaderboard |
| 74 | mistral/ministral-3b-2410 | 54.7% | 3-shot | huggingface.co | Official leaderboard | |
| 75 | LumiOpen/Viking-33B 33.1B | 54.3% | 3-shot | huggingface.co | Official leaderboard | |
| 76 | AI-Sweden-Models/gpt-sw3-126m-instruct 0.2B | 53.9% | 3-shot | huggingface.co | Official leaderboard | |
| 77 | | 52.9% | 3-shot | Mar 2025 | huggingface.co | Official leaderboard |
| 78 | google/gemma-2-9b-it 9.2B | 52.9% | 3-shot | huggingface.co | Official leaderboard | |
| 79 | | 52.5% | 3-shot | Jul 2024 | huggingface.co | Official leaderboard |
| 80 | AI-Sweden-Models/gpt-sw3-1.3b-instruct 1.4B | 52.1% | 3-shot | huggingface.co | Official leaderboard | |
| 81 | | 52.1% | 3-shot | Mar 2025 | huggingface.co | Official leaderboard |
| 82 | cyboghostginx/gemma-3-4b-it-Adetayo 4B | 52% | 3-shot | huggingface.co | Official leaderboard | |
| 83 | GPT-3.5 Turbo Jan 2024 | 52% | 3-shot | Jan 2024 | huggingface.co | Official leaderboard |
| 84 | GPT-4.1 nano | 52% | 3-shot | Apr 2025 | huggingface.co | Official leaderboard |
| 85 | meta-llama/Meta-Llama-3.1-8B-Instruct 8B | 51.9% | 3-shot | huggingface.co | Official leaderboard | |
| 86 | meta-llama/Llama-3.2-1B-Instruct 1.2B | 50.9% | 3-shot | huggingface.co | Official leaderboard | |
| 87 | AI-Sweden-Models/gpt-sw3-20b-instruct 20.9B | 49.8% | 3-shot | huggingface.co | Official leaderboard | |
| 88 | meta-llama/Llama-3.2-3B-Instruct 3.2B | 49.7% | 3-shot | huggingface.co | Official leaderboard | |
| 89 | | 49.6% | 3-shot | Jun 2024 | huggingface.co | Official leaderboard |
| 90 | AI-Sweden-Models/gpt-sw3-6.7b-v2-instruct 7.1B | 49.4% | 3-shot | huggingface.co | Official leaderboard | |
| 91 | LumiOpen/Viking-13B 14B | 49.1% | 3-shot | huggingface.co | Official leaderboard | |
| 92 | meta-llama/Meta-Llama-3-8B-Instruct 8B | 49.1% | 3-shot | huggingface.co | Official leaderboard | |
| 93 | LumiOpen/Viking-7B 7.6B | 49% | 3-shot | huggingface.co | Official leaderboard |
| # | Model | Score | Setting | Date | Link | Source |
|---|---|---|---|---|---|---|
| 1 | | 96.3% | 3-shot · thinking high | Aug 2026 | ai.google.dev | Official leaderboard |
| 2 | | 96.1% | 3-shot · thinking high | Feb 2026 | ai.google.dev | Official leaderboard |
| 3 | Claude Fable 5 | 96% | 3-shot · thinking high | Jun 2026 | anthropic.com | Official leaderboard |
| 4 | | 96% | 3-shot · thinking high | May 2026 | ai.google.dev | Official leaderboard |
| 5 | | 95.6% | 3-shot · thinking high | Nov 2025 | ai.google.dev | Official leaderboard |
| 6 | | 95.6% | 3-shot · thinking high | Dec 2025 | ai.google.dev | Official leaderboard |
| 7 | GPT-5.5 | 95.6% | 3-shot · thinking high | Apr 2026 | developers.openai.com | Official leaderboard |
| 8 | | 95.5% | 3-shot · thinking high | Jul 2026 | ai.google.dev | Official leaderboard |
| 9 | Claude Opus 4.8 | 94.9% | 3-shot · thinking high | May 2026 | anthropic.com | Official leaderboard |
| 10 | Claude Opus 4.5 | 94.7% | 3-shot · thinking high | Nov 2025 | huggingface.co | Official leaderboard |
| 11 | Claude Opus 4.6 | 94.6% | 3-shot · thinking high | Feb 2026 | anthropic.com | Official leaderboard |
| 12 | GPT-5.6 Sol | 94.5% | 3-shot · thinking high | Jul 2026 | developers.openai.com | Official leaderboard |
| 13 | GPT-5 | 94.4% | 3-shot · thinking high | Aug 2025 | huggingface.co | Official leaderboard |
| 14 | GPT-5.4 | 93.7% | 3-shot · thinking high | Mar 2026 | huggingface.co | Official leaderboard |
| 15 | o1-preview | 92.8% | 3-shot · thinking on | Sep 2024 | huggingface.co | Official leaderboard |
| 16 | | 92.8% | 3-shot · thinking on | Jun 2025 | huggingface.co | Official leaderboard |
| 17 | GPT-5.1 | 92.8% | 3-shot · thinking high | Nov 2025 | huggingface.co | Official leaderboard |
| 18 | Claude Opus 4.7 | 92.8% | 3-shot · thinking high | Apr 2026 | anthropic.com | Official leaderboard |
| 19 | Claude Sonnet 4.5 | 92.7% | 3-shot · thinking high | Sep 2025 | huggingface.co | Official leaderboard |
| 20 | GPT-5.2 | 92.2% | 3-shot · thinking high | Dec 2025 | huggingface.co | Official leaderboard |
| 21 | | 92.1% | 3-shot · thinking high | Mar 2026 | ai.google.dev | Official leaderboard |
| 22 | | 92% | 3-shot · thinking on | Apr 2026 | huggingface.co | Official leaderboard |
| 23 | | 91.8% | 3-shot · thinking high | Sep 2025 | huggingface.co | Official leaderboard |
| 24 | Claude Sonnet 4.6 | 91.8% | 3-shot · thinking high | Feb 2026 | anthropic.com | Official leaderboard |
| 25 | GPT-5.6 Terra | 91.8% | 3-shot · thinking high | Jul 2026 | developers.openai.com | Official leaderboard |
| 26 | | 91.6% | 3-shot · thinking on | Apr 2026 | huggingface.co | Official leaderboard |
| 27 | Claude 3.5 Sonnet | 91.3% | 3-shot | Jun 2024 | huggingface.co | Official leaderboard |
| 28 | GLM-5.2 | 91.2% | 3-shot · thinking on | Jun 2026 | huggingface.co | Official leaderboard |
| 29 | Grok 4.3 | 90.3% | 3-shot · thinking on | Apr 2026 | docs.x.ai | Official leaderboard |
| 30 | Claude Sonnet 4 | 90.3% | 3-shot · thinking on | May 2025 | huggingface.co | Official leaderboard |
| 31 | | 90.3% | 3-shot · thinking on | Feb 2026 | huggingface.co | Official leaderboard |
| 32 | | 90.1% | 3-shot · thinking on | Feb 2025 | huggingface.co | Official leaderboard |
| 33 | Claude 3.7 Sonnet | 89.7% | 3-shot | Feb 2025 | huggingface.co | Official leaderboard |
| 34 | Kimi K2.5 | 89.7% | 3-shot · thinking on | Jan 2026 | kimi.com | Official leaderboard |
| 35 | | 88.4% | 3-shot · thinking on | Apr 2026 | huggingface.co | Official leaderboard |
| 36 | GPT-5 mini | 87.5% | 3-shot · thinking high | Aug 2025 | huggingface.co | Official leaderboard |
| 37 | GPT-4.1 | 87.4% | 3-shot | Apr 2025 | huggingface.co | Official leaderboard |
| 38 | Claude 3 Opus | 87% | 3-shot | Feb 2024 | huggingface.co | Official leaderboard |
| 39 | GPT-5.6 Luna | 86% | 3-shot · thinking high | Jul 2026 | developers.openai.com | Official leaderboard |
| 40 | GPT-4 Turbo Apr 2024 | 85.8% | 3-shot | Apr 2024 | huggingface.co | Official leaderboard |
| 41 | GPT-4o Aug 2024 | 85.4% | 3-shot | Aug 2024 | huggingface.co | Official leaderboard |
| 42 | | 84.4% | 3-shot | Feb 2025 | huggingface.co | Official leaderboard |
| 43 | o4-mini | 83.7% | 3-shot · thinking on | Apr 2025 | huggingface.co | Official leaderboard |
| 44 | GPT-4 Jun 2023 | 83.5% | 3-shot | Jun 2023 | huggingface.co | Official leaderboard |
| 45 | GPT-4o May 2024 | 83.5% | 3-shot | May 2024 | huggingface.co | Official leaderboard |
| 46 | | 81.4% | 3-shot · thinking on | May 2025 | huggingface.co | Official leaderboard |
| 47 | GPT-4 Mar 2023 | 81.2% | 3-shot | Mar 2023 | huggingface.co | Official leaderboard |
| 48 | o3-mini | 77.8% | 3-shot · thinking on | Jan 2025 | huggingface.co | Official leaderboard |
| 49 | | 77.8% | 3-shot · thinking on | Feb 2026 | minimax.io | Official leaderboard |
| 50 | | 75.9% | 3-shot | Mar 2025 | huggingface.co | Official leaderboard |
| 51 | | 75.3% | 3-shot · thinking on | Jan 2025 | huggingface.co | Official leaderboard |
| 52 | | 75.3% | 3-shot · thinking high | Sep 2025 | huggingface.co | Official leaderboard |
| 53 | | 74.6% | 3-shot | Dec 2025 | huggingface.co | Official leaderboard |
| 54 | o1-mini | 72.6% | 3-shot · thinking on | Sep 2024 | huggingface.co | Official leaderboard |
| 55 | GPT-5 nano | 71.7% | 3-shot · thinking high | Aug 2025 | huggingface.co | Official leaderboard |
| 56 | GPT-4.1 mini | 70.9% | 3-shot | Apr 2025 | huggingface.co | Official leaderboard |
| 57 | | 70.4% | 3-shot | Mar 2025 | huggingface.co | Official leaderboard |
| 58 | | 70% | 3-shot | Oct 2024 | huggingface.co | Official leaderboard |
| 59 | | 68.7% | 3-shot | Dec 2024 | huggingface.co | Official leaderboard |
| 60 | GPT-4o mini | 65.3% | 3-shot | Jul 2024 | huggingface.co | Official leaderboard |
| 61 | | 65% | 3-shot | Mar 2025 | huggingface.co | Official leaderboard |
| 62 | Claude 3.5 Haiku | 64.4% | 3-shot | Oct 2024 | huggingface.co | Official leaderboard |
| 63 | | 61.4% | 3-shot | Jun 2024 | huggingface.co | Official leaderboard |
| 64 | Claude 2.1 | 55.1% | 3-shot | Nov 2023 | huggingface.co | Official leaderboard |
| 65 | | 52.9% | 3-shot | Mar 2025 | huggingface.co | Official leaderboard |
| 66 | | 52.5% | 3-shot | Jul 2024 | huggingface.co | Official leaderboard |
| 67 | | 52.1% | 3-shot | Mar 2025 | huggingface.co | Official leaderboard |
| 68 | GPT-3.5 Turbo Jan 2024 | 52% | 3-shot | Jan 2024 | huggingface.co | Official leaderboard |
| 69 | GPT-4.1 nano | 52% | 3-shot | Apr 2025 | huggingface.co | Official leaderboard |
| 70 | | 49.6% | 3-shot | Jun 2024 | huggingface.co | Official leaderboard |
9 models have held number one on the official leaderboard since 2023
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official Icelandic WinoGrande leaderboard
Claude 3 Opus on the official leaderboard passed 85% in February 2024
That was 20 months after its release. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official Icelandic WinoGrande leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5%
- MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87%
- CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7%
- Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
- MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%
Data audit
All 9 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (93 entries).
- Pass Models scored 70 models we track, plus 23 other entries on its leaderboard.
- Pass Newest score The newest is from August 2026, for Gemini 3.7 Flash.
- Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12).
- Note Headroom Saturated. The best is 96.3%, past 85% since February 2024.
Leaderboard import 3 pass
- Pass Importer We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 71 rows for models we track and 23 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.