Pixtral 12B
Pixtral 12B.
26
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Mistral AI Sep 2024 Pixtral 12B
Open weights
No Epoch Capabilities Index score yet
- MicroVQA 45.6% #14 /19
- HumanEval-V 13.8% #12 /24
- MM-Vet 58.5% #61 /164
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench-CN v1.1 | 70.1% | #98 of 178 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval-V | 13.8% | #12 of 24 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SnakeBench | 17 | #158 of 188 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| CharXiv | 42.4% | #19 of 77 | Official leaderboard | |
| SEED-Bench-2-Plus | 67.4% | #35 of 136 | Official leaderboard | |
| MM-Vet | 58.5% | #61 of 164 | Official leaderboard | |
| CharXiv (DQ) | 68.1% | #27 of 67 | Official leaderboard | |
| A-Bench | 73.1% | #50 of 116 | Official leaderboard | |
| MMBench | 77.9% | #70 of 155 | Official leaderboard | |
| MMBench-EN v1.1 | 75.3% | #89 of 178 | Official leaderboard | |
| MEGA-Bench | 33.7% | #11 of 22 | Official leaderboard | |
| MMStar | 54.5% | #88 of 173 | Official leaderboard | |
| MathVista | 58% | #22 of 43 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 37.6% | #95 of 152 | Official leaderboard | |
| Image2Struct | 41.2 | #12 of 18 | Official leaderboard | |
| CSVQA | 10.5% | #8 of 12 | Official leaderboard | |
| MicroVQA | 45.6% | #14 of 19 | Official leaderboard | |
| VL-RewardBench | 40.4% | #6 of 8 | Official leaderboard | |
| LEGO-Puzzles | 31.3% | #13 of 16 | Official leaderboard | |
| VHELM | 29.6 | #34 of 41 | Official leaderboard | |
| VISTA | 26% | #50 of 54 | Official leaderboard | |
| GeoBench | 37% | #46 of 48 | Official leaderboard | |
| MMVU | 32.3% | #28 of 28 | Official leaderboard | |
| Vibe-Eval | 52.5% | #5 of 5 | Official leaderboard | |
| AyaVisionBench | 41.8% | #1 of 1 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 61.0 | #78 of 173 | Official leaderboard |