Idefics2-8B
Idefics2-8B ranks first on MMIU.
24
Benchmarks with a score
1
Ranked first, out of five or more models
2
In the top three
Hugging Face Apr 2024 Idefics2-8B
Open weights Small
No Epoch Capabilities Index score yet
- MM-Vet 34% #139 /164
- SEED-Bench-2-Plus 56.6% #91 /136
- CoreCognition 59.8% #34 /119
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MM-NIAH | 11.6% | #1 of 2 | Official leaderboard | |
| MMBench-CN v1.1 | 65.1% | #123 of 178 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ChartMimic | 16.0 | #14 of 14 | Official leaderboard | |
| Design2Code | 81.7 | #5 of 5 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMIU | 27.8% | #1 of 10 | Official leaderboard | |
| CoreCognition | 59.8% | #34 of 119 | Official leaderboard | |
| MIRB | 33% | #3 of 7 | Official leaderboard | |
| MMBench | 75.7% | #78 of 155 | Official leaderboard | |
| MMBench-EN v1.1 | 72.7% | #98 of 178 | Official leaderboard | |
| OmniBench | 45.1% | #9 of 16 | Official leaderboard | |
| A-Bench | 70.5% | #68 of 116 | Official leaderboard | |
| WildVision Arena | 965 | #6 of 10 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 37.6% | #94 of 152 | Official leaderboard | |
| MMStar | 49.5% | #108 of 173 | Official leaderboard | |
| MuirBench | 26.1% | #5 of 8 | Official leaderboard | |
| SEED-Bench-2-Plus | 56.6% | #91 of 136 | Official leaderboard | |
| LongVideoBench | 49.4% | #16 of 23 | Official leaderboard | |
| MM-Vet | 34% | #139 of 164 | Official leaderboard | |
| VHELM | 21.5 | #37 of 41 | Official leaderboard | |
| CharXiv | 18.2% | #70 of 77 | Official leaderboard | |
| Image2Struct | 12.3 | #17 of 18 | Official leaderboard | |
| CharXiv (DQ) | 32.8% | #64 of 67 | Official leaderboard | |
| MMDU | 25.4 | #5 of 5 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 53.0 | #116 of 173 | Official leaderboard |