Qwen2-VL-7B-Instruct
Qwen2-VL-7B-Instruct ranks first on VCR (Visual Caption Restoration).
Released August 29th, 2024 Compare with others
65
Benchmarks with a score
1
Ranked first, out of five or more models
8
In the top three
Unknown lab Aug 2024
Qwen2-VL-7B-Instruct
No Epoch Capabilities Index score yet
- WorldSense (omnimodal) 32.4% #10 /12
- Video SimpleQA 23.4% #16 /27
- MicroVQA 48.8% #8 /19
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench-CN v1.1 | 80.3% | #43 of 178 | Official leaderboard | |
| MuSR | 43.8% | #29 of 98 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.20% | #70 of 148 | Official leaderboard | |
| Video SimpleQA | 23.4% | #16 of 27 | Official leaderboard | |
| MM-IFEval | 42% | #6 of 9 | Official leaderboard | |
| Creation-MMBench | -43.76 | #18 of 19 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| U-MATH | 26.3% | #28 of 30 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ChartMimic | 34.0 | #7 of 14 | Official leaderboard | |
| HumanEval-V | 6.30% | #14 of 24 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| UI-Vision Element Grounding | 2.70% | #2 of 5 | Official leaderboard | |
| UI-Vision Layout Grounding | 24.3% | #1 of 2 | Official leaderboard | |
| ScreenSpot-Pro | 1.58% | #10 of 11 | Official leaderboard | |
| BALROG | 3.70% | #33 of 33 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| VCR (Visual Caption Restoration) | 75.6% | #1 of 42 | Official leaderboard | |
| A-Bench | 76.4% | #19 of 116 | Official leaderboard | |
| SEED-Bench-2-Plus | 68.6% | #24 of 136 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 65.7% | #30 of 152 | Official leaderboard | |
| MMBench | 82.8% | #33 of 155 | Official leaderboard | |
| MMBench-EN v1.1 | 81.8% | #40 of 178 | Official leaderboard | |
| TOMATO | 31.5% | #5 of 22 | Official leaderboard | |
| TempCompass | 67.8% | #3 of 13 | Official leaderboard | |
| MTVQA-Test | 25.6% | #5 of 18 | Official leaderboard | |
| OVBench | 49.7% | #2 of 7 | Official leaderboard | |
| MM-Vet | 61.8% | #49 of 164 | Official leaderboard | |
| TemporalBench | 21.7% | #3 of 10 | Official leaderboard | |
| MMStar | 60.7% | #57 of 173 | Official leaderboard | |
| TVBench | 43.8% | #2 of 6 | Official leaderboard | |
| DriveBench Perception | 35.2 | #1 of 3 | Official leaderboard | |
| DriveBench Planning | 57.0 | #1 of 3 | Official leaderboard | |
| DynaMath | 42.1% | #4 of 11 | Official leaderboard | |
| CoreCognition | 57.9% | #44 of 119 | Official leaderboard | |
| VideoEval-Pro | 26.5% | #4 of 10 | Official leaderboard | |
| StreamingBench | 54.1% | #2 of 5 | Official leaderboard | |
| MEGA-Bench | 36% | #9 of 22 | Official leaderboard | |
| MicroVQA | 48.8% | #8 of 19 | Official leaderboard | |
| CharXiv | 34.6% | #33 of 77 | Official leaderboard | |
| VHELM | 53.4 | #19 of 41 | Official leaderboard | |
| VRBench | 54.1% | #9 of 19 | Official leaderboard | |
| LEGO-Puzzles | 36% | #8 of 16 | Official leaderboard | |
| HR-Bench | 66.6% | #4 of 8 | Official leaderboard | |
| LongVideoBench | 56.8% | #13 of 23 | Official leaderboard | |
| We-Math | 25.6% | #12 of 21 | Official leaderboard | |
| CharXiv (DQ) | 58% | #39 of 67 | Official leaderboard | |
| MME-CoT | 42.1% | #6 of 10 | Official leaderboard | |
| LongDocURL | 30.6% | #3 of 5 | Official leaderboard | |
| MMBench-Video | 1.45 | #17 of 28 | Official leaderboard | |
| DREAM-1K | 29.6% | #5 of 8 | Official leaderboard | |
| MMVU | 42.1% | #18 of 28 | Official leaderboard | |
| DriveBench Behavior | 54.5 | #2 of 3 | Official leaderboard | |
| DriveBench Prediction | 39.5 | #2 of 3 | Official leaderboard | |
| ERQA | 34.4% | #12 of 17 | Official leaderboard | |
| VLADBench | 49.4% | #5 of 7 | Official leaderboard | |
| VisualPuzzles | 30.2% | #21 of 29 | Official leaderboard | |
| WorldSense (omnimodal) | 32.4% | #10 of 12 | Official leaderboard | |
| EmbodiedBench EB-Habitat | 18.3% | #15 of 17 | Official leaderboard | |
| Vision Arena | 1,031 | #123 of 136 | Official leaderboard | |
| MATH-Vision (MathVision) | 16.3% | #95 of 105 | Official leaderboard | |
| EmbodiedBench EB-ALFRED | 1.70% | #17 of 17 | Official leaderboard | |
| EmbodiedBench EB-Manipulation | 0.10% | #17 of 17 | Official leaderboard | |
| EmbodiedBench EB-Navigation | 14% | #17 of 17 | Official leaderboard | |
| VL-RewardBench | 33.9% | #8 of 8 | Official leaderboard | |
| OVO-Bench | 50.4% | #2 of 2 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Video-SafetyBench | 2.70 | #13 of 15 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 67.1 | #49 of 173 | Official leaderboard | |
| Open LLM Leaderboard v2 | 26.5 | #35 of 99 | Official leaderboard |