Announcing Grok-1.5 Mar 28, 2024
- MMLU (5-shot) 81.3
- MATH (4-shot, maj@1) 50.6
- GSM8K (8-shot, maj@1) 90
- HumanEval (0-shot, pass@1) 74.1
Grok-1.5
Grok-1.5.
4
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
xAI Mar 2024
Grok-1.5
Flagship
No Epoch Capabilities Index score yet
- MMLU 81.3% #69 /131
- HumanEval 74.1% #65 /101
- MATH 50.6% #67 /99
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMLU | 81.3% | #69 of 131 | Lab model card |
Math and science
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval | 74.1% | #65 of 101 | Lab model card |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.