Benchmark Almanac

Qwen2.5-VL-72B

Qwen2.5-VL-72B ranks first on 7 of the 124 benchmarks with five or more models scored.

Qwen Released January 28th, 2025 Family Qwen2.5-VL The lab's model card Compare with others
135
Benchmarks with a score
7
Ranked first, out of five or more models
24
In the top three
Qwen Jan 2025

Qwen2.5-VL-72B

Open weights Flagship

128.2 #288 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
  • MMMU-Pro 51.1% #78 /90
  • MMVU 62.9% #22 /26
  • HallusionBench 55.2% #27 /31

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
MMBench-CN v1.1 87.4% #5 of 178 Official leaderboard
Stick to Your Role! 0.90 #1 of 15 Official leaderboard
Kaleidoscope 52.9% #1 of 7 Official leaderboard
SimpleVQA 49.4% #2 of 13 Official leaderboard
Video SimpleQA 39.5% #5 of 27 Official leaderboard
Creation-MMBench -5.82 #4 of 19 Official leaderboard
IFEval 86.3% #68 of 135 Lab model card
MMLU-Pro 71.2% #119 of 167 Lab model card
MMLU-Redux 85.9% #35 of 45 Lab model card
HallusionBench 55.2% #27 of 31 Lab model card

Math and science

BenchmarkScoreRankSource
GSM8K 95.3% #11 of 92 Lab model card
MATH 83% #20 of 99 Lab model card
GPQA Diamond 49% #251 of 302 Lab model card
HiPhO 29% #30 of 36 Official leaderboard

Coding

BenchmarkScoreRankSource
HumanEval 87.8% #28 of 101 Lab model card
MultiPL-E 79.5% #5 of 18 Lab model card
SArena 0.84 #6 of 17 Official leaderboard
ArtifactsBench (Gemini 2.5 Pro 0506 judge) 31.7% #18 of 18 Official leaderboard

Agents and tools

BenchmarkScoreRankSource
BrowseComp-VL 11.5% #1 of 4 Official leaderboard
WARC-Bench 37.3% #3 of 8 Official leaderboard
ScienceBoard 12.9% #8 of 15 Official leaderboard
ScreenSpot 87.1% #7 of 11 Lab model card
SnakeBench 20 #136 of 188 Official leaderboard
AndroidWorld 62.9% #7 of 9 Official leaderboard
MM-BrowseComp 1.75% #9 of 11 Official leaderboard
ScreenSpot-Pro 43.6% #30 of 32 Lab model card
OSWorld-Verified 5.00% #32 of 33 Official leaderboard
OSWorld 8.83% #13 of 13 Lab model card
OSWorld-MCP 15.6% #9 of 9 Official leaderboard
AndroidControl 67.4% #2 of 2 Lab model card
Android in the Zoo (AitZ) 83.2% #1 of 1 Lab model card
MobileMiniWoB++ 68% #1 of 1 Lab model card

Professional work

BenchmarkScoreRankSource
BizFinBench 67.5% #4 of 13 Official leaderboard
MedXpertQA-MM 29.9% #7 of 9 Official leaderboard
MedXpertQA 23.4% #4 of 4 Official leaderboard

Vision, audio and robotics

BenchmarkScoreRankSource
SEED-Bench-2-Plus 73.8% #1 of 136 Official leaderboard
A-Bench 81% #1 of 116 Official leaderboard
MMBench-EN v1.1 88.3% #2 of 178 Official leaderboard
CoreCognition 68.3% #2 of 119 Official leaderboard
MMBench 88.3% #3 of 155 Official leaderboard
All-Angles Bench 55.7% #1 of 18 Official leaderboard
MM-Vet 76.9% #10 of 164 Official leaderboard
VHELM 78.0 #3 of 41 Official leaderboard
DocVQA 96.4% #3 of 36 Lab model card
EgoSchema 76.2% #1 of 12 Lab model card
CharXiv (DQ) 87.4% #7 of 67 Official leaderboard
EmbodiedBench EB-ALFRED 39.7% #2 of 17 Official leaderboard
MMBench-Video 1.84 #4 of 28 Official leaderboard
VLADBench 58% #1 of 7 Official leaderboard
CCBench (MMBench Chinese Culture) 73.9% #22 of 152 Official leaderboard
We-Math 49.1% #4 of 21 Official leaderboard
OCR-Reasoning 37.5% #3 of 15 Official leaderboard
VRBench 61.7% #4 of 19 Official leaderboard
SpatialViz-Bench 35% #3 of 14 Official leaderboard
ChartQA 89.5% #8 of 37 Lab model card
VCR-Bench 37.9% #2 of 9 Official leaderboard
CrossVid 34.4% #3 of 13 Official leaderboard
MMSI-Bench 30.7% #7 of 28 Official leaderboard
LEGO-Puzzles 46.6% #4 of 16 Official leaderboard
CSVQA 38.5% #3 of 12 Official leaderboard
CRPE 79.2% #1 of 4 Lab model card
InfographicVQA 87.3% #7 of 26 Lab model card
EMMA 38.6% #2 of 7 Official leaderboard
TextVQA 83.5% #5 of 17 Lab model card
OCRBench 88.5% #11 of 34 Lab model card
MME 2448.0 #2 of 6 Lab model card
MMMU 64.5% #48 of 136 Official leaderboard
Point-Bench 59% #6 of 17 Official leaderboard
VideoReasonBench 7.20% #5 of 14 Official leaderboard
MORSE-500 17.8% #4 of 11 Official leaderboard
EmoBench-M 57.8% #6 of 15 Official leaderboard
EmbodiedBench EB-Habitat 37.7% #7 of 17 Official leaderboard
EmbodiedBench EB-Navigation 40% #7 of 17 Official leaderboard
VisualPuzzles 42.2% #12 of 29 Official leaderboard
MathCanvas-Bench 21.1% #7 of 16 Official leaderboard
Progress-Bench 32.9 #4 of 9 Official leaderboard
EmbodiedBench EB-Manipulation 16.2% #8 of 17 Official leaderboard
MM-HELIX 20.1% #8 of 17 Official leaderboard
MeasureBench Real-world 14.5% #9 of 19 Official leaderboard
AI2D 88.7% #26 of 52 Lab model card
MME-CC 23.6% #6 of 12 Official leaderboard
MM-MT-Bench 7.60 #3 of 6 Lab model card
Perception Test MCVQA 73.2% #3 of 6 Lab model card
MME-RealWorld 63.2% #2 of 4 Lab model card
OmniAI OCR Benchmark 75.2% #2 of 4 Official leaderboard
VideoMME v2 (with subtitles) 22.1% #16 of 31 Official leaderboard
BLINK 64.4% #15 of 29 Lab model card
MathVista 74.8% #37 of 70 Lab model card
CountBench 93.6% #15 of 28 Lab model card
OCRBench v2 61.5% #12 of 22 Lab model card
MME-Reasoning 34.1% #6 of 11 Official leaderboard
RealWorldQA 75.7% #21 of 38 Lab model card
MeasureBench Synthetic 11.7% #11 of 19 Official leaderboard
RefAdv-S 54% #11 of 19 Official leaderboard
Video-MME 79.1% #30 of 51 Lab model card
ODinW-13 43.1% #8 of 13 Lab model card
VisuLogic 26.2% #11 of 17 Official leaderboard
MuirBench 70.7% #8 of 12 Lab model card
MTVQA-Test 31.7% #4 of 6 Lab model card
VCR (Visual Caption Restoration) 79.8% #2 of 3 Lab model card
RoboRewardBench 0.99 #15 of 22 Official leaderboard
SpaCE-10 38.7% #25 of 36 Official leaderboard
GeoBench 62% #34 of 48 Official leaderboard
GSM8K-V 21.9% #17 of 24 Official leaderboard
CC-OCR 79.8% #10 of 14 Lab model card
OmniDocBench v1.5 87.0 #8 of 11 Official leaderboard
Charades-STA 50.9% #3 of 4 Lab model card
Math-VR 13.7% #20 of 26 Official leaderboard
MMStar 70.8% #28 of 36 Lab model card
MVBench 70.4% #18 of 23 Lab model card
Video Reality Test 58.9% #4 of 5 Official leaderboard
Vision Arena 1,121 #110 of 136 Official leaderboard
MMLongBench-Doc 35.2% #14 of 17 Official leaderboard
MotionBench 56.1% #10 of 12 Official leaderboard
MMVU 62.9% #22 of 26 Lab model card
MLVU 74.6% #19 of 22 Lab model card
MMMU-Pro 51.1% #78 of 90 Lab model card
OmniVideoBench 29.5% #7 of 8 Official leaderboard
ViVerBench 66.1% #7 of 8 Official leaderboard
SpatialWorld Physical 3.40% #8 of 9 Official leaderboard
LongVideoBench 60.7% #10 of 11 Lab model card
MathVerse 57.6% #11 of 12 Lab model card
CharXiv 49.7% #70 of 75 Lab model card
LVBench 47.3% #29 of 31 Lab model card
MATH-Vision (MathVision) 38.1% #48 of 51 Lab model card
ClockBench 6.10% #35 of 37 Official leaderboard
Video-MMMU 60.2% #35 of 36 Lab model card
VisFactor 10.9% #26 of 26 Official leaderboard
SpatialWorld Digital 7.60% #9 of 9 Official leaderboard
TempCompass 74.8% #7 of 7 Lab model card
MEGA-Bench 51.3% #1 of 1 Lab model card

Safety and security

BenchmarkScoreRankSource
nocot-bench (NCRI) 65.2 #120 of 185 Official leaderboard
Video-SafetyBench 2.40 #11 of 15 Official leaderboard

Other

BenchmarkScoreRankSource
OpenVLM Leaderboard 76.1 #8 of 173 Official leaderboard
LiveBench 57% #17 of 27 Lab model card

Its lab's numbers put it at 128.2, number 288 of 404

The Self-Reported Index fits one score to the 42 capability results Qwen reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 126.4 to 129.9. How we compute it.

Its lab's numbers next to 14 leaderboards

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark Lab says Board says Gap What differs
AndroidWorld 35% AndroidWorld_SR 62.9% Leaderboard, MobileUse −28 pts The lab evaluates Qwen's visual agent, while the board pairs Qwen2.5-VL-72B with the MobileUse agent.
ScreenSpot-Pro 43.6% ScreenSpot Pro 53.3% Leaderboard −9.7 pts
MMBench-Video 2.02 MMBench-Video 1.84 Leaderboard, 64 frames +0.18 The lab allowed up to 768 video frames, while the board evaluated the model with 64 frames.
MMMU 70.2% MMMUval 64.5% Leaderboard +5.7 pts
MMMU-Pro 51.1% MMMU_Pro 46.2% Leaderboard +4.9 pts
SEED-Bench-2-Plus 73% SEED-Bench-2-Plus (Table 5) 73.8% Leaderboard −0.8 pts
MM-Vet 76.2% MMVet turbo (Table 3) 76.9% Leaderboard −0.7 pts
MMBench 88.6% MMBench-EN test 88.6, MMBench-CN test 87.9 (Table 3) 88.3% Leaderboard +0.3 pts
MMStar 70.8% MMStar 70.5% Leaderboard +0.3 pts
MMBench-EN v1.1 88.4% MMBench-V1.1-EN test (Table 3) 88.3% Leaderboard Same
Video-MMMU 60.2% Video-MMMU (Table 8) 60.2% Leaderboard Same
CharXiv 49.7% CharXiv RQ (Table 5) 49.7% Leaderboard Same
CharXiv (DQ) 87.4% CharXiv DQ (Table 5) 87.4% Leaderboard, validation; descriptive questions Same
MATH-Vision (MathVision) 38.1% MathVision_FULL 38.1% Leaderboard Same

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Qwen2.5-VL-72B-Instruct model card (Hugging Face) Jan 28, 2025

Its system card

Every result in the longer documents the lab published with it. 34 of them aren't in the launch post.

Qwen2.5-VL Technical Report Feb 19, 2025

Capabilities