SWAG
Models read a partial description of an everyday activity. They choose the most plausible continuation from four options.
- Made by
- Rowan Zellers and colleagues
- Released
- August 2018
- Kind
- Commonsense
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
5 turkers tops the leaderboard with 88%
Best score per entry from the benchmark's paper, read on November 20th, 2026. A paper’s table doesn’t add newer models. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.
- 88.0 5 turkers
- 85.1 3 turkers
- 85.0 Expert
- 82.8 1 turker
- 58.5 ESIM+ELMo
- 52.5 ESIM+GloVe
- 50.4 LSTM+ELMo
- 48.6 DecompAttn+GloVe
- 48.3 DecompAttn+Numberbatch
- 45.7 LSTM+GloVe
- 45.4 DecompAttn+ELMo
- 44.6 ESIM+Numberbatch
- 40.5 LSTM+Numberbatch
- 39.4 InferSent-MLP
- 38.4 InferSent-Bilinear
- 36.4 SkipThoughts-MLP
- 36.0 SNLI-ESIM
- 35.7 SNLI-DecompAttn
- 34.9 SkipThoughts-Bilinear
- 34.1 DualBoW+Numberbatch
- 33.1 DualBoW+GloVe
- 29.8 fastText
- 0.0 InferSent
- 0.0 SkipThoughts
- 1 5 turkers 88%
- 2 3 turkers 85.1%
- 3 Expert 85%
- 4 1 turker 82.8%
- 5 ESIM+ELMo 58.5%
- 6 ESIM+GloVe 52.5%
- 7 LSTM+ELMo 50.4%
- 8 DecompAttn+GloVe 48.6%
- 9 DecompAttn+Numberbatch 48.3%
- 10 LSTM+GloVe 45.7%
- 11 DecompAttn+ELMo 45.4%
- 12 ESIM+Numberbatch 44.6%
- 13 LSTM+Numberbatch 40.5%
- 14 InferSent-MLP 39.4%
- 15 InferSent-Bilinear 38.4%
- 16 SkipThoughts-MLP 36.4%
- 17 SNLI-ESIM 36%
- 18 SNLI-DecompAttn 35.7%
- 19 SkipThoughts-Bilinear 34.9%
- 20 DualBoW+Numberbatch 34.1%
- 21 DualBoW+GloVe 33.1%
- 22 fastText 29.8%
- 23 InferSent 0%
- 24 SkipThoughts 0%
Source: The SWAG paper
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | 5 turkers | 88% | Official leaderboard | ||
| 2 | 3 turkers | 85.1% | Official leaderboard | ||
| 3 | Expert | 85% | Official leaderboard | ||
| 4 | 1 turker | 82.8% | Official leaderboard | ||
| 5 | ESIM+ELMo | 58.5% | Official leaderboard | ||
| 6 | ESIM+GloVe | 52.5% | Official leaderboard | ||
| 7 | LSTM+ELMo | 50.4% | Official leaderboard | ||
| 8 | DecompAttn+GloVe | 48.6% | Official leaderboard | ||
| 9 | DecompAttn+Numberbatch | 48.3% | Official leaderboard | ||
| 10 | LSTM+GloVe | 45.7% | Official leaderboard | ||
| 11 | DecompAttn+ELMo | 45.4% | Official leaderboard | ||
| 12 | ESIM+Numberbatch | 44.6% | Official leaderboard | ||
| 13 | LSTM+Numberbatch | 40.5% | Official leaderboard | ||
| 14 | InferSent-MLP | 39.4% | Official leaderboard | ||
| 15 | InferSent-Bilinear | 38.4% | Official leaderboard | ||
| 16 | SkipThoughts-MLP | 36.4% | Official leaderboard | ||
| 17 | SNLI-ESIM | 36% | Official leaderboard | ||
| 18 | SNLI-DecompAttn | 35.7% | Official leaderboard | ||
| 19 | SkipThoughts-Bilinear | 34.9% | Official leaderboard | ||
| 20 | DualBoW+Numberbatch | 34.1% | Official leaderboard | ||
| 21 | DualBoW+GloVe | 33.1% | Official leaderboard | ||
| 22 | fastText | 29.8% | Official leaderboard | ||
| 23 | InferSent | 0% | Official leaderboard | ||
| 24 | SkipThoughts | 0% | Official leaderboard |
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Knowledge
- MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects.
Intern-S2-397B 89.8%
- MMLU In use 57-subject multiple-choice exam questions from elementary to professional level. Scored by accuracy.
Seed1.8 92.3%
- SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy.
Qwen 3.6 Max (Preview) 73.9%
- WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1.
Tencent Hy4 preview 83.9%
- BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly.
Gemma 4 31B IT 74.4%
- HLE-Diamond In use Models answer 1,000 cleaned expert questions from Humanity's Last Exam, 500 on reasoning and 500 on knowledge, closed book. Graded on exact answers. Claude Fable 5.2 77.9%
Data audit
All 8 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
- Pass Independent runs Scores from the benchmark’s paper (24 entries).
- Pass Models scored 24 entries on its leaderboard, none a model we track.
- Note Newest score No scores yet.
- Note Frontier models No models scored yet.
- Note Headroom No scores yet.
Leaderboard import 3 pass
- Pass Importer We read the board on arxiv.org: 24 entries at the last read, none a model we track.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.