PaLM (540B)
PaLM (540B).
15
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Google DeepMind Apr 2022 PaLM (540B)
Closed weights
No Epoch Capabilities Index score yet
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| WinoGrande | 85.1% | #7 of 76 | Claim in GPT-4 Technical report, via Epoch AI | |
| ARC-Challenge | 85.2% | #12 of 75 | Claim in GPT-4 Technical Report, via Epoch AI | |
| LAMBADA | 77.9% | #6 of 26 | Claim in The Falcon Series of Open Language Models, via Epoch AI | |
| HellaSwag | 83.8% | #15 of 63 | Claim in PaLM: Scaling Language Modeling with Pathways, via Epoch AI | |
| OpenBookQA | 68% | #10 of 41 | Claim in PaLM: Scaling Language Modeling with Pathways, via Epoch AI | |
| MMLU | 69.3% | #27 of 85 | Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI | |
| BoolQ | 91.9% | #1 of 3 | Official leaderboard | |
| MultiRC | 88.7% | #1 of 3 | Official leaderboard | |
| SuperGLUE | 90.4% | #1 of 3 | Official leaderboard | |
| WSC | 95.9% | #1 of 3 | Official leaderboard | |
| PIQA | 82.3% | #21 of 58 | Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI | |
| TriviaQA | 81.4% | #14 of 38 | Claim in PaLM: Scaling Language Modeling with Pathways, via Epoch AI | |
| CB | 96% | #2 of 3 | Official leaderboard | |
| BIG-Bench Hard | 65.2% | #1 of 1 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 56.5% | #31 of 70 | Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI |