Grok-2 Dec 2024
Grok-2.
21
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
xAI Dec 2024
Grok-2
Dec 2024 snapshot Closed weights
129.7
#158 of 256
Epoch Capabilities Index
- GPQA Diamond 53.8% #150 /229
- SimpleQA Verified 20.5% #40 /56
- CorpFin v2 51.1% #93 /119
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (HHEM) | 1.90% | #22 of 148 | Official leaderboard | |
| MMLU-Pro | 75.5% | #73 of 212 | Official leaderboard | |
| BuzzBench | 51.8 | #8 of 20 | Official leaderboard | |
| ECLeKTic | 18.4% | #18 of 35 | Official leaderboard | |
| FACTS Grounding | 76.2% | #19 of 34 | Official leaderboard | |
| LLM Confabulation Leaderboard | 20.1% | #38 of 58 | Official leaderboard | |
| SimpleQA Verified | 20.5% | #40 of 56 | Official leaderboard | |
| Phare | 54.6% | #61 of 62 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH Level 5 | 63.5% | #40 of 94 | Epoch AI run | |
| GPQA Diamond | 53.8% | #150 of 229 | Epoch AI run | |
| DTBench | 65.2% | #124 of 180 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 11.5% | #161 of 212 | Epoch AI run | |
| MATH-500 | 78.4% | #33 of 38 | Official leaderboard | |
| FrontierMath | 0.69% | #62 of 70 | Epoch AI run |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| DesignArena Website | 850 | #144 of 151 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Elimination Game | 3 | #42 of 56 | Official leaderboard | |
| LLM Chess | -335 | #105 of 123 | Official leaderboard | |
| Step Race | 1 | #56 of 59 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| TaxEval v2 | 67% | #100 of 129 | Official leaderboard | |
| CorpFin v2 | 51.1% | #93 of 119 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BaxBench | 18.4% | #23 of 34 | Official leaderboard |
xAI's models over time
xAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)