InstructGPT 175B
InstructGPT 175B places in the top three on LAMBADA.
OpenAI Released January 27th, 2022
12
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
OpenAI Jan 2022
InstructGPT 175B
Closed weights
No Epoch Capabilities Index score yet
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LAMBADA | 86.4% | #2 of 26 | Claim in GLaM: Efficient Scaling of Language Models with Mixture-o…, via Epoch AI | |
| OpenBookQA | 65.4% | #11 of 41 | Claim in GLaM: Efficient Scaling of Language Models with Mixture-o…, via Epoch AI | |
| WinoGrande | 77.7% | #25 of 76 | Claim in Language Models are Few-Shot Learners, via Epoch AI | |
| PIQA | 82.3% | #20 of 58 | Claim in GLaM: Efficient Scaling of Language Models with Mixture-o…, via Epoch AI | |
| HellaSwag | 79.3% | #27 of 63 | Claim in Using DeepSpeed and Megatron to Train Megatron-Turing NLG…, via Epoch AI | |
| ARC-Challenge | 53.2% | #41 of 75 | Claim in GLaM: Efficient Scaling of Language Models with Mixture-o…, via Epoch AI | |
| BoolQ | 77.5% | #39 of 61 | Claim in GLaM: Efficient Scaling of Language Models with Mixture-o…, via Epoch AI | |
| TriviaQA | 71.2% | #28 of 38 | Claim in Language Models are Few-Shot Learners, via Epoch AI | |
| MMLU | 43.9% | #64 of 85 | Claim in Scaling Language Models: Methods, Analysis & Insights fro…, via Epoch AI | |
| CommonsenseQA 2.0 | 52.9% | #5 of 6 | Claim in CommonsenseQA 2.0: Exposing the Limits of AI through Gami…, via Epoch AI | |
| SuperGLUE | 71.8% | #7 of 8 | Claim in Language Models are Few-Shot Learners, via Epoch AI | |
| ANLI | 35.4% | #11 of 11 | Claim in Using DeepSpeed and Megatron to Train Megatron-Turing NLG…, via Epoch AI |