MMMU team
We track 2 benchmarks that MMMU team made or helped make.
2 in use 2 with public code 2 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| MMMU 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. | General visual questions | Nov 2023 | | In use |
| MMMU-Pro Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. | General visual questions | Sep 2024 | 84.6% | In use |