Qwen2-VL-72B-Instruct
Qwen2-VL-72B-Instruct ranks first on TOMATO.
Released August 29th, 2024 Compare with others
39
Benchmarks with a score
1
Ranked first, out of five or more models
6
In the top three
Unknown lab Aug 2024
Qwen2-VL-72B-Instruct
No Epoch Capabilities Index score yet
- ZeroBench 2.00% #66 /76
- MicroVQA 47.5% #11 /19
- VisualPuzzles 34.9% #15 /29
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench-CN v1.1 | 85.8% | #11 of 178 | Official leaderboard | |
| MuSR | 44.9% | #17 of 98 | Official leaderboard | |
| MM-IFEval | 50.8% | #2 of 9 | Official leaderboard | |
| SimpleVQA | 44.7% | #4 of 13 | Official leaderboard | |
| Creation-MMBench | -22.45 | #11 of 19 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| U-MATH | 41.8% | #22 of 30 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval-V | 16.6% | #10 of 24 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ScreenSpot-Pro | 1.01% | #11 of 11 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedXpertQA-MM | 28.4% | #8 of 9 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| A-Bench | 79.4% | #3 of 116 | Official leaderboard | |
| TOMATO | 37.9% | #1 of 22 | Official leaderboard | |
| MMBench | 86.9% | #8 of 155 | Official leaderboard | |
| MMBench-EN v1.1 | 86.1% | #12 of 178 | Official leaderboard | |
| MMStar | 68.6% | #19 of 173 | Official leaderboard | |
| MTVQA-Test | 30.9% | #2 of 18 | Official leaderboard | |
| Video-MME | 77.8% | #3 of 26 | Official leaderboard | |
| CoreCognition | 61.1% | #21 of 119 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 69.8% | #27 of 152 | Official leaderboard | |
| MathVista | 70.5% | #8 of 43 | Official leaderboard | |
| VHELM | 71.0 | #8 of 41 | Official leaderboard | |
| CharXiv | 43% | #17 of 77 | Official leaderboard | |
| MEGA-Bench | 47.7% | #5 of 22 | Official leaderboard | |
| MME-CoT | 56.2% | #3 of 10 | Official leaderboard | |
| MMMU | 64.5% | #47 of 136 | Official leaderboard | |
| MMVU | 53% | #10 of 28 | Official leaderboard | |
| EmbodiedBench EB-ALFRED | 33.7% | #7 of 17 | Official leaderboard | |
| MMMU-Pro | 46.2% | #27 of 57 | Official leaderboard | |
| VisualPuzzles | 34.9% | #15 of 29 | Official leaderboard | |
| EmbodiedBench EB-Habitat | 35.7% | #9 of 17 | Official leaderboard | |
| LVBench | 41.3% | #5 of 9 | Official leaderboard | |
| MicroVQA | 47.5% | #11 of 19 | Official leaderboard | |
| EmbodiedBench EB-Manipulation | 13.6% | #11 of 17 | Official leaderboard | |
| EMMA | 33.5% | #5 of 7 | Official leaderboard | |
| MATH-Vision (MathVision) | 25.9% | #79 of 105 | Official leaderboard | |
| VISTA | 28.6% | #46 of 54 | Official leaderboard | |
| ZeroBench | 2.00% | #66 of 76 | Official leaderboard | |
| EmbodiedBench EB-Navigation | 21.2% | #15 of 17 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open LLM Leaderboard v2 | 39.5 | #6 of 99 | Official leaderboard | |
| OpenVLM Leaderboard | 74.8 | #13 of 173 | Official leaderboard |