Mistral Medium
Mistral Medium places in the top three on AlpacaEval.
9
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Mistral AI Dec 2023 Mistral Medium
Closed weights
No Epoch Capabilities Index score yet
- PRBench 39.5% #38 /48
- Phare 58.1% #57 /62
- MixEval 81.9% #14 /34
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AlpacaEval | 96.8% | #2 of 19 | Official leaderboard | |
| EQ-Bench v2 | 82.6 | #13 of 85 | Official leaderboard | |
| AlpacaEval 2.0 | 28.6% | #17 of 47 | Official leaderboard | |
| MixEval | 81.9% | #14 of 34 | Official leaderboard | |
| Arena-Hard-Auto | 31.9% | #24 of 38 | Official leaderboard | |
| LMArena (Elo) | 1,223 | #276 of 335 | Official leaderboard | |
| Phare | 58.1% | #57 of 62 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| PRBench | 39.5% | #38 of 48 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OR-Bench | 14% | #8 of 28 | Official leaderboard |