Mistral Large 2 Nov 2024
Mistral Large 2.
39
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Mistral AI Nov 2024 Mistral Large 2
Nov 2024 snapshot Open weights
No Epoch Capabilities Index score yet
- GPQA Diamond 51.3% #153 /229
- SimpleQA Verified 22.3% #38 /56
- TaxEval v2 63.8% #107 /129
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (2025 dataset) | 4.50% | #8 of 104 | Official leaderboard | |
| IFEval | 87.7% | #13 of 60 | Official leaderboard | |
| AraGen | 52% | #18 of 42 | Official leaderboard | |
| MMLU-Pro | 67.9% | #92 of 212 | Official leaderboard | |
| BuzzBench | 47.5 | #9 of 20 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.10% | #69 of 148 | Official leaderboard | |
| SuperGPQA | 40.6% | #36 of 59 | Official leaderboard | |
| ECLeKTic | 13.4% | #22 of 35 | Official leaderboard | |
| LMArena (Elo) | 1,306 | #225 of 335 | Official leaderboard | |
| SimpleQA Verified | 22.3% | #38 of 56 | Official leaderboard | |
| LongBench v2 | 39.6% | #23 of 33 | Official leaderboard | |
| FACTS Grounding | 72.6% | #24 of 34 | Official leaderboard | |
| LLM Confabulation Leaderboard | 21.4% | #41 of 58 | Official leaderboard | |
| WritingBench | 54.4 | #35 of 45 | Official leaderboard | |
| EQ-Bench 3 | 998 | #63 of 78 | Official leaderboard | |
| HELM Arabic | 71.7 | #23 of 28 | Official leaderboard | |
| HELM Arabic Enterprise | 62.0 | #19 of 23 | Official leaderboard | |
| Creative Writing v3 | 51.9 | #101 of 120 | Official leaderboard | |
| Phare | 58.8% | #54 of 62 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH Level 5 | 50.3% | #51 of 94 | Epoch AI run | |
| U-MATH | 47.6% | #17 of 30 | Official leaderboard | |
| ChemBench | 57% | #14 of 24 | Official leaderboard | |
| GPQA Diamond | 51.3% | #153 of 229 | Epoch AI run | |
| GPQA Main | 43.5% | #41 of 60 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 7.78% | #167 of 212 | Epoch AI run | |
| DTBench | 60.8% | #142 of 180 | Official leaderboard | |
| FrontierMath | 0.34% | #67 of 70 | Epoch AI run |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMTU | 44.6% | #7 of 9 | Official leaderboard | |
| SWE-PolyBench | 5.88% | #5 of 6 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BFCL V4 | 38.4% | #29 of 65 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 58% | #30 of 67 | Official leaderboard | |
| Elimination Game | 4 | #28 of 56 | Official leaderboard | |
| ComplexFuncBench | 20.1% | #6 of 11 | Official leaderboard | |
| Galileo Agent Leaderboard | 0.81 | #21 of 29 | Official leaderboard | |
| LisanBench | 138.0 | #102 of 133 | Official leaderboard | |
| Step Race | 1 | #48 of 59 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| TaxEval v2 | 63.8% | #107 of 129 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LiveSecBench | 29.7 | #27 of 34 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Capabilities | 59.8 | #43 of 60 | Official leaderboard |