IDEFICS-9B-Instruct
IDEFICS-9B-Instruct ranks first on MuirBench.
15
Benchmarks with a score
1
Ranked first, out of five or more models
1
In the top three
Hugging Face Jul 2023 IDEFICS-9B-Instruct
Open weights Small
No Epoch Capabilities Index score yet
- MM-Vet 30% #154 /164
- SEED-Bench-2-Plus 35.4% #130 /136
- CoreCognition 39.4% #113 /119
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench-CN v1.1 | 19.5% | #174 of 178 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MuirBench | 35.4% | #1 of 8 | Official leaderboard | |
| Image2Struct | 17.5 | #16 of 18 | Official leaderboard | |
| MM-Vet | 30% | #154 of 164 | Official leaderboard | |
| CoreCognition | 39.4% | #113 of 119 | Official leaderboard | |
| SEED-Bench-2-Plus | 35.4% | #130 of 136 | Official leaderboard | |
| MMBench | 45.3% | #149 of 155 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 7.80% | #147 of 152 | Official leaderboard | |
| MMBench-EN v1.1 | 35.2% | #174 of 178 | Official leaderboard | |
| MMStar | 21.6% | #173 of 173 | Official leaderboard | |
| MathVista | 19.8% | #43 of 43 | Official leaderboard | |
| VHELM | 4.51 | #41 of 41 | Official leaderboard | |
| MMIU | 12.8% | #10 of 10 | Official leaderboard | |
| SEED-Bench-2 | 38.8% | #5 of 5 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 27.2 | #170 of 173 | Official leaderboard |