SuperGLUE
PaLM (540B) 90.4% leads the models we track PaLM (540B) 90.4% leads since April 2022 Models solve eight English language understanding tasks, including BoolQ, COPA, MultiRC, ReCoRD, RTE, WiC, and WSC. Scored by the average task score.
Saturated, with models above the human baseline since 2021.
- Made by
- NYU, Facebook AI, University of Washington, and DeepMind
- Released
- May 2019
- Kind
- Language reasoning
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- Public, plus a private test set
- Last model added
- April 2022
- Scores refreshed
- November 12th, 2026
In the authors’ words
In the last year, new models and methods for pretraining and transfer learning have driven striking performance improvements across a range of language understanding tasks. The GLUE benchmark, introduced a little over one year ago, offers a single-number metric that summarizes progress on a diverse set of such tasks, but performance on the benchmark has recently surpassed the level of non-expert humans, suggesting limited headroom for further research. In this paper we present SuperGLUE, a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, a software toolkit, and a public leaderboard. SuperGLUE is available at super.gluebenchmark.com.
Hairuo tops the leaderboard with 91.4%
Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is PaLM (540B), at 90.4%.
PaLM (540B) leads with 90.4%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.
The newest model with a score here came out in April 2022, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- Claude Fable 5.2
- Gemini 4 Carbon
- GPT-6 Astra
- Claude Opus 5.5
- and 16 more
- 91.4 Hairuo
- 91.3 Vega v2
- 91.2 ST-MoE-32B
- 90.9 Turing NLR v5
- 90.6 ERNIE 3.0
- 90.4 PaLM (540B)
- 90.4 T5 + UDG, Single Model (Google Brain)
- 90.3 DeBERTa / TuringNLRv4
- 89.8 SuperGLUE Human Baselines
- 89.3 T5-11B
- 89.2 Frozen T5 1.1 + SPoT
- 86.7 NEZHA-Plus
- 86.1 PAI Albert
- 86.0 RoBERTa-iCETS
- 85.9 RoBERTa (ensemble)
- 85.7 RoBERTa-mtl-adv
- 84.5 RoBERTa
- 82.6 AILabs Team, Transformers
- 77.7 FSL++(ALBERT)-Few-Shot(32 Examples)
- 77.0 Text to Text PETL
- 76.0 ADAPET (ALBERT) - few-shot
- 75.4 iPET (ALBERT) - Few-Shot (32 Examples)
- 74.1 Bort (Alexa AI)
- 73.5 BERT-mtl
- 71.8 GPT-3 175B (davinci)
- 71.5 BERT++
- 1 Hairuo 91.4%
- 2 Vega v2 91.3%
- 3 ST-MoE-32B 91.2%
- 4 Turing NLR v5 90.9%
- 5 ERNIE 3.0 90.6%
- 6 PaLM (540B) 90.4%
- 7 T5 + UDG, Single Model (Google Brain) 90.4%
- 8 DeBERTa / TuringNLRv4 90.3%
- 9 SuperGLUE Human Baselines 89.8%
- 10 T5-11B 89.3%
- 11 Frozen T5 1.1 + SPoT 89.2%
- 12 NEZHA-Plus 86.7%
- 13 PAI Albert 86.1%
- 14 RoBERTa-iCETS 86%
- 15 RoBERTa (ensemble) 85.9%
- 16 RoBERTa-mtl-adv 85.7%
- 17 RoBERTa 84.5%
- 18 AILabs Team, Transformers 82.6%
- 19 FSL++(ALBERT)-Few-Shot(32 Examples) 77.7%
- 20 Text to Text PETL 77%
- 21 ADAPET (ALBERT) - few-shot 76%
- 22 iPET (ALBERT) - Few-Shot (32 Examples) 75.4%
- 23 Bort (Alexa AI) 74.1%
- 24 BERT-mtl 73.5%
- 25 GPT-3 175B (davinci) 71.8%
- 26 BERT++ 71.5%
Source: Official SuperGLUE leaderboard
- 1 PaLM (540B) 90.4%
- 2 T5-11B 89.3%
- 3 T5-3B 86.4%
- 4 Switch-Large 84.7%
- 5 T5-Large 82.7%
- 6 T5-Base 76.2%
- 7 Switch-Base 73.3%
- 8 GPT-3 175B (davinci) 71.8%
- 9 InstructGPT 175B 71.8%
- 10 T5-Small 63.3%
Sources: Official SuperGLUE leaderboard and Another board, via Epoch AI (CC BY 4.0)
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Hairuo | 91.4% | Aug 2024 | Official leaderboard | |
| 2 | Vega v2 | 91.3% | Oct 2022 | Official leaderboard | |
| 3 | ST-MoE-32B | 91.2% | Feb 2022 | Official leaderboard | |
| 4 | Turing NLR v5 | 90.9% | Dec 2021 | Official leaderboard | |
| 5 | ERNIE 3.0 | 90.6% | Jul 2021 | Official leaderboard | |
| 6 | PaLM (540B) | 90.4% | Apr 2022 | Official leaderboard | |
| 7 | T5 + UDG, Single Model (Google Brain) | 90.4% | Mar 2021 | Official leaderboard | |
| 8 | DeBERTa / TuringNLRv4 | 90.3% | Jan 2021 | Official leaderboard | |
| 9 | SuperGLUE Human Baselines | 89.8% | May 2019 | Official leaderboard | |
| 10 | T5-11B | 89.3% | Oct 2019 | Official leaderboard | |
| 11 | Frozen T5 1.1 + SPoT | 89.2% | Feb 2022 | Official leaderboard | |
| 12 | NEZHA-Plus | 86.7% | Sep 2020 | Official leaderboard | |
| 13 | PAI Albert | 86.1% | Aug 2020 | Official leaderboard | |
| 14 | RoBERTa-iCETS | 86% | Mar 2021 | Official leaderboard | |
| 15 | RoBERTa (ensemble) | 85.9% | Jul 2020 | Official leaderboard | |
| 16 | RoBERTa-mtl-adv | 85.7% | Jan 2020 | Official leaderboard | |
| 17 | RoBERTa | 84.5% | Aug 2019 | Official leaderboard | |
| 18 | AILabs Team, Transformers | 82.6% | Mar 2021 | Official leaderboard | |
| 19 | FSL++(ALBERT)-Few-Shot(32 Examples) | 77.7% | Oct 2022 | Official leaderboard | |
| 20 | Text to Text PETL | 77% | Apr 2022 | Official leaderboard | |
| 21 | ADAPET (ALBERT) - few-shot | 76% | Mar 2021 | Official leaderboard | |
| 22 | iPET (ALBERT) - Few-Shot (32 Examples) | 75.4% | Sep 2020 | Official leaderboard | |
| 23 | Bort (Alexa AI) | 74.1% | Oct 2020 | Official leaderboard | |
| 24 | BERT-mtl | 73.5% | Oct 2019 | Official leaderboard | |
| 25 | GPT-3 175B (davinci) | 71.8% | May 2020 | Official leaderboard | |
| 26 | BERT++ | 71.5% | Jul 2019 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | PaLM (540B) | 90.4% | Apr 2022 | Official leaderboard | |
| 2 | T5-11B | 89.3% | Oct 2019 | Official leaderboard | |
| 3 | T5-3B | 86.4% | Oct 2019 | Claim in Exploring the Limits of Transfer Learning with a Unified…, via Epoch AI | |
| 4 | Switch-Large | 84.7% | Claim in Switch Transformers: Scaling to Trillion Parameter Models…, via Epoch AI | ||
| 5 | T5-Large | 82.7% | Claim in Switch Transformers: Scaling to Trillion Parameter Models…, via Epoch AI | ||
| 6 | T5-Base | 76.2% | Claim in Exploring the Limits of Transfer Learning with a Unified…, via Epoch AI | ||
| 7 | Switch-Base | 73.3% | Claim in Switch Transformers: Scaling to Trillion Parameter Models…, via Epoch AI | ||
| 8 | GPT-3 175B (davinci) | 71.8% | May 2020 | Official leaderboard | |
| 9 | InstructGPT 175B | 71.8% | Jan 2022 | Claim in Language Models are Few-Shot Learners, via Epoch AI | |
| 10 | T5-Small | 63.3% | Claim in Exploring the Limits of Transfer Learning with a Unified…, via Epoch AI |
Epoch AI collected claims for 8 models from labs’ reports and papers. Switch the scores above to rank them together.
Number one over time Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official SuperGLUE leaderboard
T5-11B on the official leaderboard passed 85% in October 2019
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official SuperGLUE leaderboard and claims collected by Epoch AI (CC BY 4.0)
No lab model card reports it yet
We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.
More in Knowledge
- MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects.
Intern-S2-397B 89.8% - SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy.
Qwen 3.6 Max (Preview) 73.9% - WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1.
Tencent Hy4 preview 83.9% - BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
- KMMLU-Pro In use Models answer multiple-choice questions taken from Korean National Professional Licensure exams. It tests professional knowledge used in Korean industry.
Solar Pro 4 79.2% - HLE-Verified In use Verified and revised version of Humanity's Last Exam (1,811 items) with uncertain items removed.
Seed2.0 Pro 73.6%