Mistral 7B v0.3
Mistral 7B v0.3.
32
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Mistral AI Sep 2023 Mistral 7B v0.3
Open weights
No Epoch Capabilities Index score yet
- GPQA Diamond 15.2% #226 /229
- KoBALT 12% #12 /12
- KOR-Bench 24.2% #31 /33
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MuSR | 40.3% | #57 of 98 | Official leaderboard | |
| AlpacaEval 2.0 | 20.6% | #28 of 47 | Official leaderboard | |
| RepoQA | 62% | #14 of 22 | Official leaderboard | |
| EQ-Bench v2 | 63.1 | #62 of 85 | Official leaderboard | |
| Open Arabic LLM Leaderboard v1 | 38.7% | #43 of 55 | Official leaderboard | |
| AbstentionBench | 70.3% | #4 of 5 | Official leaderboard | |
| MMLU-CF | 51.1% | #31 of 37 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 9.50% | #128 of 148 | Official leaderboard | |
| ThaiExam | 32% | #21 of 24 | Official leaderboard | |
| HELMET | 10.8% | #14 of 16 | Official leaderboard | |
| ToRR | 32.4 | #13 of 14 | Official leaderboard | |
| SuperGPQA | 17.8% | #58 of 59 | Official leaderboard | |
| IFEval | 56.8% | #60 of 60 | Official leaderboard | |
| AgentIF | 46.8% | #12 of 12 | Official leaderboard | |
| KoBALT | 12% | #12 of 12 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| KOR-Bench | 24.2% | #31 of 33 | Official leaderboard | |
| GPQA Main | 30.3% | #57 of 60 | Official leaderboard | |
| GPQA Diamond | 15.2% | #226 of 229 | Epoch AI run | |
| MATH Level 5 | 3.68% | #93 of 94 | Epoch AI run | |
| OTIS Mock AIME 2024-2025 | 0.28% | #210 of 212 | Epoch AI run |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench-Hard | 5.40% | #90 of 96 | Official leaderboard | |
| BigCodeBench | 23.4% | #65 of 69 | Official leaderboard | |
| MHPP | 9.00% | #23 of 23 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedQA | 51.7% | #58 of 71 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SGP-Bench | 41.7% | #18 of 22 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AIR-Bench 2024 | 32.5% | #79 of 82 | Official leaderboard | |
| Anthropic Red Team | 90.1% | #73 of 75 | Official leaderboard | |
| HELM Safety | 73.0 | #73 of 75 | Official leaderboard | |
| SimpleSafetyTests | 81% | #73 of 75 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open LLM Leaderboard v2 | 14.2 | #74 of 99 | Official leaderboard | |
| HELM Lite | 19.6 | #68 of 71 | Official leaderboard | |
| HELM Capabilities | 37.6 | #60 of 60 | Official leaderboard |