Announcing Grok Nov 3, 2023
Grok-1
Grok-1.
5
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
xAI Nov 2023
Grok-1
Flagship
No Epoch Capabilities Index score yet
- MMLU 73% #94 /131
- HumanEval 63.2% #78 /101
- MATH 23.9% #92 /99
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMLU | 73% | #94 of 131 | Lab model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Hungarian National High-School Exam | 59% | #1 of 2 | Lab model card | |
| GSM8K | 62.9% | #77 of 92 | Lab model card | |
| MATH | 23.9% | #92 of 99 | Lab model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval | 63.2% | #78 of 101 | Lab model card |
Its lab's number next to a leaderboard
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
| Benchmark | Lab says | Board says | Gap |
|---|---|---|---|
| Hungarian National High-School Exam | 59% Hungarian national high school finals in mathematics (1-shot, human-graded) | 59% Leaderboard | Same |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.