Qwen-VL-Chat
Qwen-VL-Chat places in the top three on SEED-Bench-2.
Released August 20th, 2023 Compare with others
28
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Unknown lab Aug 2023
Qwen-VL-Chat
No Epoch Capabilities Index score yet
- CG-Bench 20.7% #6 /6
- SEED-Bench 58.2% #5 /9
- MM-Vet 47.3% #96 /164
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench-CN v1.1 | 57.5% | #152 of 178 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OlympicArena | 7.34% | #24 of 25 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GroundUI | 0% | #10 of 11 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GMAI-MMBench | 37% | #8 of 13 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| CMMMU | 31.3% | #5 of 9 | Official leaderboard | |
| SEED-Bench | 58.2% | #5 of 9 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 41.2% | #88 of 152 | Official leaderboard | |
| MM-Vet | 47.3% | #96 of 164 | Official leaderboard | |
| VCR (Visual Caption Restoration) | 21.1% | #25 of 42 | Official leaderboard | |
| SEED-Bench-2 | 50.3% | #3 of 5 | Official leaderboard | |
| MMT-Bench | 52.5% | #10 of 15 | Official leaderboard | |
| MME | 1487.6 | #13 of 17 | Official leaderboard | |
| ScienceQA | 68.2% | #13 of 17 | Claim in VisionLLM v2: An End-to-End Generalist Multimodal Large L…, via Epoch AI | |
| Image2Struct | 24.6 | #14 of 18 | Official leaderboard | |
| WildVision | 18.1 | #11 of 14 | Official leaderboard | |
| SEED-Bench-2-Plus | 46% | #115 of 136 | Official leaderboard | |
| TempCompass | 52.9% | #11 of 13 | Official leaderboard | |
| MMBench-EN v1.1 | 60.8% | #158 of 178 | Official leaderboard | |
| MMBench | 61.8% | #140 of 155 | Official leaderboard | |
| A-Bench | 63.1% | #105 of 116 | Official leaderboard | |
| MMMU | 35.9% | #126 of 136 | Official leaderboard | |
| MMStar | 34.5% | #163 of 173 | Official leaderboard | |
| VHELM | 4.73 | #40 of 41 | Official leaderboard | |
| Video-MME | 41.9% | #26 of 26 | Official leaderboard | |
| MME-RealWorld | 21.1% | #11 of 11 | Official leaderboard | |
| MIRB | 14.4% | #7 of 7 | Official leaderboard | |
| CG-Bench | 20.7% | #6 of 6 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 45.2 | #146 of 173 | Official leaderboard |