OpenVLM Video Leaderboard
Models complete video understanding tasks. OpenCompass compares models across video evaluation datasets.
The leaderboard aggregates results from component evaluations.
- Made by
- OpenCompass
- Released
- September 2024
- Kind
- Composite indices
- Level
- University
- Scored
- Settled by events
- Answers
- Short answer
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Single turn
- Items from
- Another benchmark
- Refresh
- One fixed set
- Run by
- Anyone
- In views
- Vision
- Strongest model tested
-
Gemini 2.0 Flash Feb 2025 #147 on the capability index
- Last model added
- May 2025
- Scores refreshed
- November 20th, 2026
Video-ChatGPT tops the leaderboard with 37.8 SmolVLM2-500M-Video-Instruct leads with 35.4 among models released by May 2025
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is SmolVLM2-500M-Video-Instruct, at 35.4. This view shows only models we track, each at its best.
The newest model with a score here came out in May 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 37.8 Video-ChatGPT
- 35.4
SmolVLM2-500M-Video-Instruct
- 34.8
Phi-4-multimodal
- 34.2 LLaMA-VID-7B
- 33.2 Chat-UniVi-7B-v1.5
- 33.0 Video-LLaVA
- 31.8
Idefics3-8B-Llama3
- 28.8 Claude 3.5 Sonnet
- 28.0
mPLUG-Owl3-7B
- 27.2
SmolVLM2-2.2B-Instruct
- 27.0
VILA1.5-40B
- 27.0 Long-VITA-16K
- 26.4 Phi-3.5-vision-instruct
- 25.8 VideoChat2-HD
- 24.0 PLLaVA-34B
- 21.4
InternVL2-8B
- 21.4 MiniCPM-V-2_6
- 21.0
Gemini 1.5 Flash
- 20.4 MiniCPM-o-2_6
- 19.6 MiMo-VL-7B-RL
- 17.4 LLaVA-OneVision-72B-sft
- 17.2 Qwen2-VL-7B-Instruct
- 16.4 InternVL2-76B
- 15.6 LLaVA-Video-7B-Qwen2
- 14.8
Qwen2.5-VL-7B
- 12.4
InternVL2.5-8B
- 12.2
Qwen2.5-Omni-7B
- 12.0 GPT-4o
- 11.0
Gemini 2.0 Flash
- 9.60 Aria
- 9.00 LLaVA-Video-72B-Qwen2
- 8.80
InternVL3-8B
- 7.80 Qwen2-VL-72B-Instruct
- 7.20
InternVL2_5-78B
- 6.40
Qwen2.5-VL-72B
- 3.60
InternVL3-38B
- 3.00
InternVL3-78B
- 1 Video-ChatGPT 100 frames ✓ 37.8
- 2
SmolVLM2-500M-Video-Instruct 64 frames ✓ 35.4
- 3
Phi-4-multimodal 64 frames ✓ 34.8
- 4 LLaMA-VID-7B 1 fps frames ✓ 34.2
- 5 Chat-UniVi-7B-v1.5 1.0 fps frames ✓ 33.2
- 6 Video-LLaVA 8 frames ✓ 33.0
- 7
Idefics3-8B-Llama3 16 frames ✓ 31.8
- 8 Claude 3.5 Sonnet 8 frames ✓ 28.8
- 9
mPLUG-Owl3-7B 16 frames ✓ 28.0
- 10
SmolVLM2-2.2B-Instruct 64 frames ✓ 27.2
- 11
VILA1.5-40B 14 frames ✓ 27.0
- 12 Long-VITA-16K 64 frames ✓ 27.0
- 13 Phi-3.5-vision-instruct 16 frames ✓ 26.4
- 14 VideoChat2-HD 16 frames ✓ 25.8
- 15 PLLaVA-34B 16 frames ✓ 24.0
- 16
InternVL2-8B 16 frames ✓ 21.4
- 17 MiniCPM-V-2_6 64 frames ✓ 21.4
- 18
Gemini 1.5 Flash May 2024, 16 frames ✓ 21.0
- 19 MiniCPM-o-2_6 64 frames ✓ 20.4
- 20 MiMo-VL-7B-RL 64 frames ✓ 19.6
- 21 LLaVA-OneVision-72B-sft 32 frames ✓ 17.4
- 22 Qwen2-VL-7B-Instruct 64 frames ✓ 17.2
- 23 InternVL2-76B 16 frames ✓ 16.4
- 24 LLaVA-Video-7B-Qwen2 64 frames ✓ 15.6
- 25
Qwen2.5-VL-7B 64 frames ✓ 14.8
- 26
InternVL2.5-8B 64 frames ✓ 12.4
- 27
Qwen2.5-Omni-7B 64 frames ✓ 12.2
- 28 GPT-4o Aug 2024, 16 frames ✓ 12.0
- 29
Gemini 2.0 Flash Feb 2025, 64 frames ✓ 11.0
- 30 Aria 64 frames 9.60
- 31 LLaVA-Video-72B-Qwen2 64 frames ✓ 9.00
- 32
InternVL3-8B 64 frames ✓ 8.80
- 33 Qwen2-VL-72B-Instruct 64 frames ✓ 7.80
- 34
InternVL2_5-78B 64 frames ✓ 7.20
- 35
Qwen2.5-VL-72B 64 frames ✓ 6.40
- 36
InternVL3-38B 64 frames ✓ 3.60
- 37
InternVL3-78B 64 frames ✓ 3.00
Source: Official OpenVLM Video Leaderboard leaderboard
- 35.4
SmolVLM2-500M-Video-Instruct
- 34.8
Phi-4-multimodal
- 33.2 Chat-UniVi-7B-v1.5
- 31.8
Idefics3-8B-Llama3
- 28.8 Claude 3.5 Sonnet
- 28.0
mPLUG-Owl3-7B
- 27.2
SmolVLM2-2.2B-Instruct
- 27.0
VILA1.5-40B
- 26.4 Phi-3.5-vision-instruct
- 21.4
InternVL2-8B
- 21.4 MiniCPM-V-2_6
- 21.0
Gemini 1.5 Flash
- 20.4 MiniCPM-o-2_6
- 19.6 MiMo-VL-7B-RL
- 17.2 Qwen2-VL-7B-Instruct
- 15.6 LLaVA-Video-7B-Qwen2
- 14.8
Qwen2.5-VL-7B
- 12.4
InternVL2.5-8B
- 12.2
Qwen2.5-Omni-7B
- 12.0 GPT-4o
- 11.0
Gemini 2.0 Flash
- 9.60 Aria
- 9.00 LLaVA-Video-72B-Qwen2
- 8.80
InternVL3-8B
- 7.80 Qwen2-VL-72B-Instruct
- 7.20
InternVL2_5-78B
- 6.40
Qwen2.5-VL-72B
- 3.60
InternVL3-38B
- 3.00
InternVL3-78B
- 1
SmolVLM2-500M-Video-Instruct 64 frames ✓ 35.4
- 2
Phi-4-multimodal 64 frames ✓ 34.8
- 3 Chat-UniVi-7B-v1.5 1.0 fps frames ✓ 33.2
- 4
Idefics3-8B-Llama3 16 frames ✓ 31.8
- 5 Claude 3.5 Sonnet 8 frames ✓ 28.8
- 6
mPLUG-Owl3-7B 16 frames ✓ 28.0
- 7
SmolVLM2-2.2B-Instruct 64 frames ✓ 27.2
- 8
VILA1.5-40B 14 frames ✓ 27.0
- 9 Phi-3.5-vision-instruct 16 frames ✓ 26.4
- 10
InternVL2-8B 16 frames ✓ 21.4
- 11 MiniCPM-V-2_6 64 frames ✓ 21.4
- 12
Gemini 1.5 Flash May 2024, 16 frames ✓ 21.0
- 13 MiniCPM-o-2_6 64 frames ✓ 20.4
- 14 MiMo-VL-7B-RL 64 frames ✓ 19.6
- 15 Qwen2-VL-7B-Instruct 64 frames ✓ 17.2
- 16 LLaVA-Video-7B-Qwen2 64 frames ✓ 15.6
- 17
Qwen2.5-VL-7B 64 frames ✓ 14.8
- 18
InternVL2.5-8B 64 frames ✓ 12.4
- 19
Qwen2.5-Omni-7B 64 frames ✓ 12.2
- 20 GPT-4o Aug 2024, 16 frames ✓ 12.0
- 21
Gemini 2.0 Flash Feb 2025, 64 frames ✓ 11.0
- 22 Aria 64 frames 9.60
- 23 LLaVA-Video-72B-Qwen2 64 frames ✓ 9.00
- 24
InternVL3-8B 64 frames ✓ 8.80
- 25 Qwen2-VL-72B-Instruct 64 frames ✓ 7.80
- 26
InternVL2_5-78B 64 frames ✓ 7.20
- 27
Qwen2.5-VL-72B 64 frames ✓ 6.40
- 28
InternVL3-38B 64 frames ✓ 3.60
- 29
InternVL3-78B 64 frames ✓ 3.00
Source: Official OpenVLM Video Leaderboard leaderboard
| # | Entry | Score | Setting | Released | Run on | Link | Source |
|---|---|---|---|---|---|---|---|
| 1 | Video-ChatGPT ✓ open weights | 37.8 | 100 frames | May 2023 | github.com | Official leaderboard | |
| 2 | | 35.4 | 64 frames | Feb 2025 | Feb 2025 | huggingface.co | Official leaderboard |
| 3 | | 34.8 | 64 frames | Feb 2025 | Feb 2025 | huggingface.co | Official leaderboard |
| 4 | LLaMA-VID-7B ✓ open weights | 34.2 | 1 fps frames | Nov 2023 | huggingface.co | Official leaderboard | |
| 5 | Chat-UniVi-7B-v1.5 ✓ | 33.2 | 1.0 fps frames | Apr 2024 | Apr 2024 | huggingface.co | Official leaderboard |
| 6 | Video-LLaVA ✓ open weights, 7.5B | 33.0 | 8 frames | Nov 2023 | github.com | Official leaderboard | |
| 7 | | 31.8 | 16 frames | Aug 2024 | Aug 2024 | huggingface.co | Official leaderboard |
| 8 | Claude 3.5 Sonnet ✓ | 28.8 | 8 frames | Jun 2024 | Jun 2024 | anthropic.com | Official leaderboard |
| 9 | | 28.0 | 16 frames | Aug 2024 | Aug 2024 | huggingface.co | Official leaderboard |
| 10 | | 27.2 | 64 frames | Feb 2025 | Feb 2025 | huggingface.co | Official leaderboard |
| 11 | | 27.0 | 14 frames | Apr 2024 | May 2024 | huggingface.co | Official leaderboard |
| 12 | Long-VITA-16K ✓ open weights, 14B | 27.0 | 64 frames | Jun 2025 | huggingface.co | Official leaderboard | |
| 13 | Phi-3.5-vision-instruct ✓ | 26.4 | 16 frames | Aug 2024 | Aug 2024 | huggingface.co | Official leaderboard |
| 14 | VideoChat2-HD ✓ open weights | 25.8 | 16 frames | Jun 2024 | huggingface.co | Official leaderboard | |
| 15 | PLLaVA-34B ✓ open weights, 34.9B | 24.0 | 16 frames | Apr 2024 | huggingface.co | Official leaderboard | |
| 16 | | 21.4 | 16 frames | Jul 2024 | Jun 2024 | huggingface.co | Official leaderboard |
| 17 | MiniCPM-V-2_6 ✓ | 21.4 | 64 frames | Aug 2024 | Aug 2024 | huggingface.co | Official leaderboard |
| 18 | | 21.0 | 16 frames | May 2024 | May 2024 | deepmind.google | Official leaderboard |
| 19 | MiniCPM-o-2_6 ✓ | 20.4 | 64 frames | Jan 2025 | Jan 2025 | huggingface.co | Official leaderboard |
| 20 | MiMo-VL-7B-RL ✓ | 19.6 | 64 frames | May 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 21 | LLaVA-OneVision-72B-sft ✓ open weights, 73.2B | 17.4 | 32 frames | Aug 2024 | huggingface.co | Official leaderboard | |
| 22 | Qwen2-VL-7B-Instruct ✓ | 17.2 | 64 frames | Aug 2024 | Jun 2025 | huggingface.co | Official leaderboard |
| 23 | InternVL2-76B ✓ open weights, 76.3B | 16.4 | 16 frames | Jul 2024 | huggingface.co | Official leaderboard | |
| 24 | LLaVA-Video-7B-Qwen2 ✓ | 15.6 | 64 frames | Sep 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 25 | | 14.8 | 64 frames | Jan 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 26 | | 12.4 | 64 frames | Dec 2024 | Nov 2024 | huggingface.co | Official leaderboard |
| 27 | | 12.2 | 64 frames | Mar 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 28 | GPT-4o Aug 2024 ✓ | 12.0 | 16 frames | Aug 2024 | Aug 2024 | openai.com | Official leaderboard |
| 29 | | 11.0 | 64 frames | Feb 2025 | Jun 2025 | deepmind.google | Official leaderboard |
| 30 | Aria | 9.60 | 64 frames | Sep 2024 | Oct 2024 | huggingface.co | Official leaderboard |
| 31 | LLaVA-Video-72B-Qwen2 ✓ | 9.00 | 64 frames | Sep 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 32 | | 8.80 | 64 frames | Apr 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 33 | Qwen2-VL-72B-Instruct ✓ | 7.80 | 64 frames | Aug 2024 | Jun 2025 | huggingface.co | Official leaderboard |
| 34 | | 7.20 | 64 frames | Dec 2024 | Dec 2024 | huggingface.co | Official leaderboard |
| 35 | | 6.40 | 64 frames | Jan 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 36 | | 3.60 | 64 frames | Apr 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 37 | | 3.00 | 64 frames | Apr 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| # | Model | Score | Setting | Released | Run on | Link | Source |
|---|---|---|---|---|---|---|---|
| 1 | | 35.4 | 64 frames | Feb 2025 | Feb 2025 | huggingface.co | Official leaderboard |
| 2 | | 34.8 | 64 frames | Feb 2025 | Feb 2025 | huggingface.co | Official leaderboard |
| 3 | Chat-UniVi-7B-v1.5 ✓ | 33.2 | 1.0 fps frames | Apr 2024 | Apr 2024 | huggingface.co | Official leaderboard |
| 4 | | 31.8 | 16 frames | Aug 2024 | Aug 2024 | huggingface.co | Official leaderboard |
| 5 | Claude 3.5 Sonnet ✓ | 28.8 | 8 frames | Jun 2024 | Jun 2024 | anthropic.com | Official leaderboard |
| 6 | | 28.0 | 16 frames | Aug 2024 | Aug 2024 | huggingface.co | Official leaderboard |
| 7 | | 27.2 | 64 frames | Feb 2025 | Feb 2025 | huggingface.co | Official leaderboard |
| 8 | | 27.0 | 14 frames | Apr 2024 | May 2024 | huggingface.co | Official leaderboard |
| 9 | Phi-3.5-vision-instruct ✓ | 26.4 | 16 frames | Aug 2024 | Aug 2024 | huggingface.co | Official leaderboard |
| 10 | | 21.4 | 16 frames | Jul 2024 | Jun 2024 | huggingface.co | Official leaderboard |
| 11 | MiniCPM-V-2_6 ✓ | 21.4 | 64 frames | Aug 2024 | Aug 2024 | huggingface.co | Official leaderboard |
| 12 | | 21.0 | 16 frames | May 2024 | May 2024 | deepmind.google | Official leaderboard |
| 13 | MiniCPM-o-2_6 ✓ | 20.4 | 64 frames | Jan 2025 | Jan 2025 | huggingface.co | Official leaderboard |
| 14 | MiMo-VL-7B-RL ✓ | 19.6 | 64 frames | May 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 15 | Qwen2-VL-7B-Instruct ✓ | 17.2 | 64 frames | Aug 2024 | Jun 2025 | huggingface.co | Official leaderboard |
| 16 | LLaVA-Video-7B-Qwen2 ✓ | 15.6 | 64 frames | Sep 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 17 | | 14.8 | 64 frames | Jan 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 18 | | 12.4 | 64 frames | Dec 2024 | Nov 2024 | huggingface.co | Official leaderboard |
| 19 | | 12.2 | 64 frames | Mar 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 20 | GPT-4o Aug 2024 ✓ | 12.0 | 16 frames | Aug 2024 | Aug 2024 | openai.com | Official leaderboard |
| 21 | | 11.0 | 64 frames | Feb 2025 | Jun 2025 | deepmind.google | Official leaderboard |
| 22 | Aria | 9.60 | 64 frames | Sep 2024 | Oct 2024 | huggingface.co | Official leaderboard |
| 23 | LLaVA-Video-72B-Qwen2 ✓ | 9.00 | 64 frames | Sep 2024 | Sep 2024 | huggingface.co | Official leaderboard |
| 24 | | 8.80 | 64 frames | Apr 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 25 | Qwen2-VL-72B-Instruct ✓ | 7.80 | 64 frames | Aug 2024 | Jun 2025 | huggingface.co | Official leaderboard |
| 26 | | 7.20 | 64 frames | Dec 2024 | Dec 2024 | huggingface.co | Official leaderboard |
| 27 | | 6.40 | 64 frames | Jan 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 28 | | 3.60 | 64 frames | Apr 2025 | Jun 2025 | huggingface.co | Official leaderboard |
| 29 | | 3.00 | 64 frames | Apr 2025 | Jun 2025 | huggingface.co | Official leaderboard |
Rankings by part
We read 75 parts from the benchmark’s own leaderboard in 5 groups: video-mme (w/o subs), mvbench, mmbench-video, tempcompass and mlvu. Each ranks models on one part of the test. InternVL3-78B leads 25 of them, and 9 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
Video-MME (w/o subs)
29 models we track have a score here.
- 1
InternVL3-78B ↑28 73.1
- 2
InternVL2_5-78B ↑24 72.1
- 3
InternVL3-38B ↑25 72.1
- 4
Gemini 2.0 Flash Feb 2025 ↑17 71.0
- 5 LLaVA-Video-72B-Qwen2 ↑18 70.5
- 6
Qwen2.5-VL-72B ↑21 68.6
- 7 GPT-4o Aug 2024 ↑13 67.9
- 8 Qwen2-VL-72B-Instruct ↑17 67.3
- 9 Aria ↑13 66.0
- 10
InternVL3-8B ↑14 66.0
MVBench
29 models we track have a score here.
- 1
InternVL3-78B ↑28 79.2
- 2
InternVL3-38B ↑26 76.0
- 3
InternVL2_5-78B ↑23 75.6
- 4
InternVL3-8B ↑20 73.2
- 5
Qwen2.5-VL-72B ↑22 71.3
- 6
InternVL2.5-8B ↑12 70.5
- 7 Qwen2-VL-72B-Instruct ↑18 70.2
- 8
Qwen2.5-Omni-7B ↑11 69.0
- 9 Aria ↑13 67.9
- 10
Qwen2.5-VL-7B ↑7 67.5
MMBench-Video
29 models we track have a score here.
- 1
Gemini 2.0 Flash Feb 2025 ↑20 2.01
- 2
InternVL2_5-78B ↑24 1.98
- 3 GPT-4o Aug 2024 ↑17 1.87
- 4
Qwen2.5-VL-72B ↑23 1.84
- 5 Aria ↑17 1.81
- 6
InternVL3-78B ↑23 1.81
- 7
InternVL3-38B ↑21 1.80
- 8 MiniCPM-o-2_6 ↑5 1.76
- 9 LLaVA-Video-72B-Qwen2 ↑14 1.71
- 10 MiniCPM-V-2_6 ↑1 1.70
TempCompass
29 models we track have a score here.
- 1 Qwen2-VL-72B-Instruct ↑24 79.4
- 2
InternVL3-38B ↑26 78.5
- 3
Qwen2.5-VL-72B ↑24 77.7
- 4
InternVL3-78B ↑25 77.0
- 5 GPT-4o Aug 2024 ↑15 72.7
- 6
Qwen2.5-VL-7B ↑11 72.0
- 7
Qwen2.5-Omni-7B ↑12 70.7
- 8
InternVL3-8B ↑16 70.4
- 9 LLaVA-Video-72B-Qwen2 ↑14 70.4
- 10 Aria ↑12 69.6
MLVU
29 models we track have a score here.
- 1
InternVL3-78B ↑28 74.0
- 2
InternVL3-38B ↑26 72.7
- 3
InternVL2_5-78B ↑23 70.9
- 4 LLaVA-Video-72B-Qwen2 ↑19 70.0
- 5
Gemini 2.0 Flash Feb 2025 ↑16 68.0
- 6
InternVL3-8B ↑18 68.0
- 7 LLaVA-Video-7B-Qwen2 ↑9 67.7
- 8 Aria ↑14 67.1
- 9 Qwen2-VL-72B-Instruct ↑16 65.2
- 10
Qwen2.5-VL-72B ↑17 64.1
short
29 models we track have a score here.
- 1
InternVL3-78B ↑28 83.8
- 2
InternVL2_5-78B ↑24 83.0
- 3
InternVL3-38B ↑25 82.1
- 4 LLaVA-Video-72B-Qwen2 ↑19 80.8
- 5
Gemini 2.0 Flash Feb 2025 ↑16 79.7
- 6
SmolVLM2-2.2B-Instruct ↑1 79.0
- 7 Qwen2-VL-72B-Instruct ↑18 78.9
- 8
Qwen2.5-VL-72B ↑19 78.2
- 9 GPT-4o Aug 2024 ↑11 77.7
- 10 Aria ↑12 77.1
medium
29 models we track have a score here.
- 1
InternVL3-38B ↑27 72.1
- 2
Gemini 2.0 Flash Feb 2025 ↑19 71.8
- 3
InternVL2_5-78B ↑23 71.1
- 4
InternVL3-78B ↑25 71.1
- 5 LLaVA-Video-72B-Qwen2 ↑18 68.6
- 6
Qwen2.5-VL-72B ↑21 68.6
- 7
InternVL3-8B ↑17 66.4
- 8 GPT-4o Aug 2024 ↑12 66.2
- 9 MiMo-VL-7B-RL ↑5 65.8
- 10 Aria ↑12 64.9
long
29 models we track have a score here.
- 1
InternVL3-78B ↑28 64.4
- 2
InternVL2_5-78B ↑24 62.2
- 3 LLaVA-Video-72B-Qwen2 ↑20 62.1
- 4
InternVL3-38B ↑24 62.1
- 5
Gemini 2.0 Flash Feb 2025 ↑16 61.4
- 6 GPT-4o Aug 2024 ↑14 59.8
- 7 Qwen2-VL-72B-Instruct ↑18 59.1
- 8
Qwen2.5-VL-72B ↑19 58.9
- 9 Aria ↑13 56.0
- 10
InternVL3-8B ↑14 55.4
Temporal Perception
29 models we track have a score here.
- 1
InternVL3-38B ↑27 87.3
- 2
InternVL2_5-78B ↑24 81.8
- 3
Gemini 2.0 Flash Feb 2025 ↑18 80.0
- 4 LLaVA-Video-72B-Qwen2 ↑19 78.2
- 5 LLaVA-Video-7B-Qwen2 ↑11 78.2
- 6 MiMo-VL-7B-RL ↑8 78.2
- 7
InternVL3-78B ↑22 76.4
- 8 Qwen2-VL-72B-Instruct ↑17 74.5
- 9 Aria ↑13 74.5
- 10
Qwen2.5-Omni-7B ↑9 74.5
Spatial Perception
29 models we track have a score here.
- 1 GPT-4o Aug 2024 ↑19 79.6
- 2
Gemini 2.0 Flash Feb 2025 ↑19 77.8
- 3
InternVL3-78B ↑26 77.8
- 4
VILA1.5-40B ↑4 75.9
- 5 LLaVA-Video-72B-Qwen2 ↑18 75.9
- 6
InternVL3-38B ↑22 75.9
- 7
InternVL2_5-78B ↑19 74.1
- 8
Qwen2.5-VL-72B ↑19 72.2
- 9 MiMo-VL-7B-RL ↑5 72.2
- 10 Qwen2-VL-72B-Instruct ↑15 70.4
Attribute Perception
29 models we track have a score here.
- 1
InternVL2_5-78B ↑25 85.6
- 2
Gemini 2.0 Flash Feb 2025 ↑19 85.1
- 3
InternVL3-78B ↑26 85.1
- 4
InternVL3-38B ↑24 83.3
- 5 LLaVA-Video-72B-Qwen2 ↑18 82.0
- 6 Qwen2-VL-72B-Instruct ↑19 80.6
- 7 Aria ↑15 80.6
- 8
Qwen2.5-VL-72B ↑19 78.8
- 9
InternVL3-8B ↑15 77.9
- 10 LLaVA-Video-7B-Qwen2 ↑6 77.0
Action Recognition
29 models we track have a score here.
- 1
InternVL3-38B ↑27 74.8
- 2
InternVL3-78B ↑27 72.2
- 3 LLaVA-Video-72B-Qwen2 ↑20 70.6
- 4
InternVL2_5-78B ↑22 70.3
- 5
Gemini 2.0 Flash Feb 2025 ↑16 70.0
- 6 GPT-4o Aug 2024 ↑14 66.8
- 7 MiMo-VL-7B-RL ↑7 66.5
- 8
Qwen2.5-VL-72B ↑19 66.1
- 9 Qwen2-VL-72B-Instruct ↑16 65.5
- 10 LLaVA-Video-7B-Qwen2 ↑6 64.5
Object Recognition
29 models we track have a score here.
- 1
InternVL2_5-78B ↑25 78.5
- 2
InternVL3-78B ↑27 78.5
- 3
InternVL3-38B ↑25 77.7
- 4
Qwen2.5-VL-72B ↑23 75.4
- 5
Gemini 2.0 Flash Feb 2025 ↑16 74.6
- 6 LLaVA-Video-72B-Qwen2 ↑17 73.2
- 7 Aria ↑15 73.2
- 8
InternVL3-8B ↑16 72.3
- 9 GPT-4o Aug 2024 ↑11 71.2
- 10 Qwen2-VL-72B-Instruct ↑15 70.9
OCR Problems
29 models we track have a score here.
- 1
Gemini 2.0 Flash Feb 2025 ↑20 79.9
- 2
InternVL3-38B ↑26 79.9
- 3
InternVL2_5-78B ↑23 79.1
- 4
InternVL3-78B ↑25 78.4
- 5
Qwen2.5-VL-72B ↑22 77.0
- 6 Aria ↑16 75.5
- 7
InternVL2.5-8B ↑11 75.5
- 8
InternVL3-8B ↑16 74.1
- 9 MiMo-VL-7B-RL ↑5 74.1
- 10 Qwen2-VL-72B-Instruct ↑15 72.7
Counting Problem
29 models we track have a score here.
- 1 LLaVA-Video-72B-Qwen2 ↑22 50.4
- 2 LLaVA-Video-7B-Qwen2 ↑14 49.6
- 3
Gemini 2.0 Flash Feb 2025 ↑18 48.9
- 4
InternVL2_5-78B ↑22 48.5
- 5
InternVL3-78B ↑24 48.5
- 6 GPT-4o Aug 2024 ↑14 47.4
- 7
VILA1.5-40B ↑1 47.0
- 8
InternVL3-38B ↑20 45.9
- 9
InternVL2.5-8B ↑9 44.0
- 10 MiniCPM-o-2_6 ↑3 44.0
Temporal Reasoning
29 models we track have a score here.
- 1
InternVL2_5-78B ↑25 66.1
- 2
InternVL3-78B ↑27 63.3
- 3
Qwen2.5-VL-72B ↑24 61.6
- 4
InternVL3-38B ↑24 60.5
- 5
Gemini 2.0 Flash Feb 2025 ↑16 57.6
- 6 LLaVA-Video-72B-Qwen2 ↑17 57.1
- 7 Aria ↑15 55.4
- 8
Qwen2.5-Omni-7B ↑11 54.2
- 9 Qwen2-VL-72B-Instruct ↑16 52.0
- 10
InternVL3-8B ↑14 52.0
Spatial Reasoning
29 models we track have a score here.
- 1 GPT-4o Aug 2024 ↑19 89.3
- 2
InternVL3-38B ↑26 87.5
- 3 Qwen2-VL-72B-Instruct ↑22 85.7
- 4 LLaVA-Video-72B-Qwen2 ↑19 85.7
- 5
InternVL2_5-78B ↑21 83.9
- 6
InternVL3-78B ↑23 83.9
- 7
Qwen2.5-Omni-7B ↑12 82.1
- 8
InternVL3-8B ↑16 82.1
- 9
Gemini 1.5 Flash May 2024 ↑3 80.4
- 10 Qwen2-VL-7B-Instruct ↑5 80.4
Action Reasoning
29 models we track have a score here.
- 1
InternVL3-78B ↑28 69.1
- 2 GPT-4o Aug 2024 ↑18 66.0
- 3 LLaVA-Video-72B-Qwen2 ↑20 65.6
- 4
InternVL3-38B ↑24 65.6
- 5
Gemini 2.0 Flash Feb 2025 ↑16 63.5
- 6
InternVL2_5-78B ↑20 62.8
- 7
InternVL3-8B ↑17 62.1
- 8 Qwen2-VL-72B-Instruct ↑17 61.4
- 9
Qwen2.5-VL-72B ↑18 60.7
- 10
VILA1.5-40B ↓2 60.0
Object Reasoning
29 models we track have a score here.
- 1
InternVL3-78B ↑28 71.1
- 2
InternVL2_5-78B ↑24 70.9
- 3 LLaVA-Video-72B-Qwen2 ↑20 70.7
- 4
InternVL3-38B ↑24 69.6
- 5 GPT-4o Aug 2024 ↑15 68.9
- 6
Qwen2.5-VL-72B ↑21 68.5
- 7
Gemini 2.0 Flash Feb 2025 ↑14 68.1
- 8 Qwen2-VL-72B-Instruct ↑17 67.8
- 9 Aria ↑13 63.9
- 10
InternVL3-8B ↑14 63.2
Information Synopsis
29 models we track have a score here.
- 1
Gemini 2.0 Flash Feb 2025 ↑20 86.4
- 2
InternVL3-78B ↑27 86.4
- 3
InternVL3-38B ↑25 83.9
- 4 Qwen2-VL-72B-Instruct ↑21 83.3
- 5
InternVL2_5-78B ↑21 83.3
- 6 LLaVA-Video-72B-Qwen2 ↑17 82.7
- 7
Qwen2.5-VL-72B ↑20 82.0
- 8
InternVL2.5-8B ↑10 81.4
- 9 GPT-4o Aug 2024 ↑11 81.1
- 10 Aria ↑12 81.1
Overall
29 models we track have a score here.
- 1
InternVL3-78B ↑28 73.1
- 2
InternVL2_5-78B ↑24 72.1
- 3
InternVL3-38B ↑25 72.1
- 4
Gemini 2.0 Flash Feb 2025 ↑17 71.0
- 5 LLaVA-Video-72B-Qwen2 ↑18 70.5
- 6
Qwen2.5-VL-72B ↑21 68.6
- 7 GPT-4o Aug 2024 ↑13 67.9
- 8 Qwen2-VL-72B-Instruct ↑17 67.3
- 9 Aria ↑13 66.0
- 10
InternVL3-8B ↑14 66.0
action_sequence
29 models we track have a score here.
- 1
InternVL3-38B ↑27 90.5
- 2
InternVL2_5-78B ↑24 88.0
- 3
InternVL3-78B ↑26 87.5
- 4
InternVL2.5-8B ↑14 82.5
- 5 Qwen2-VL-72B-Instruct ↑20 81.0
- 6 Qwen2-VL-7B-Instruct ↑9 79.5
- 7
Qwen2.5-Omni-7B ↑12 79.5
- 8
InternVL3-8B ↑16 79.5
- 9 LLaVA-Video-7B-Qwen2 ↑7 75.5
- 10
Qwen2.5-VL-72B ↑17 75.5
action_prediction
29 models we track have a score here.
- 1
InternVL3-78B ↑28 72.0
- 2
Qwen2.5-VL-72B ↑25 71.5
- 3
InternVL2_5-78B ↑23 69.0
- 4 Qwen2-VL-7B-Instruct ↑11 68.5
- 5
InternVL3-8B ↑19 68.5
- 6 LLaVA-Video-72B-Qwen2 ↑17 68.0
- 7 Qwen2-VL-72B-Instruct ↑18 67.5
- 8 LLaVA-Video-7B-Qwen2 ↑8 67.5
- 9
Qwen2.5-VL-7B ↑8 67.5
- 10
InternVL3-38B ↑18 67.5
action_antonym
29 models we track have a score here.
- 1 Aria ↑21 95.5
- 2
InternVL2_5-78B ↑24 94.5
- 3
InternVL3-78B ↑26 93.0
- 4
InternVL2.5-8B ↑14 92.0
- 5
InternVL3-38B ↑23 92.0
- 6 Qwen2-VL-72B-Instruct ↑19 90.5
- 7
InternVL3-8B ↑17 90.0
- 8
Qwen2.5-VL-7B ↑9 85.0
- 9
Qwen2.5-Omni-7B ↑10 84.0
- 10
InternVL2-8B 83.5
fine_grained_action
29 models we track have a score here.
- 1
InternVL3-78B ↑28 58.5
- 2
InternVL2_5-78B ↑24 54.5
- 3 GPT-4o Aug 2024 ↑17 54.0
- 4
InternVL3-38B ↑24 54.0
- 5
Qwen2.5-VL-72B ↑22 52.5
- 6 LLaVA-Video-7B-Qwen2 ↑10 51.0
- 7
Gemini 2.0 Flash Feb 2025 ↑14 51.0
- 8 Qwen2-VL-7B-Instruct ↑7 50.0
- 9
Qwen2.5-VL-7B ↑8 50.0
- 10
Gemini 1.5 Flash May 2024 ↑2 49.0
unexpected_action
29 models we track have a score here.
- 1 LLaVA-Video-72B-Qwen2 ↑22 90.0
- 2 Qwen2-VL-72B-Instruct ↑23 87.5
- 3
InternVL3-78B ↑26 87.0
- 4
Qwen2.5-VL-72B ↑23 84.0
- 5
InternVL3-38B ↑23 84.0
- 6 LLaVA-Video-7B-Qwen2 ↑10 83.0
- 7 GPT-4o Aug 2024 ↑13 82.0
- 8
InternVL2_5-78B ↑18 81.5
- 9
Gemini 2.0 Flash Feb 2025 ↑12 81.5
- 10
Qwen2.5-Omni-7B ↑9 79.0
object_existence
29 models we track have a score here.
- 1
InternVL3-38B ↑27 99.5
- 2
InternVL2_5-78B ↑24 97.0
- 3
InternVL3-78B ↑26 96.5
- 4
InternVL3-8B ↑20 96.5
- 5
InternVL2-8B ↑5 96.0
- 6
Qwen2.5-Omni-7B ↑13 95.5
- 7
InternVL2.5-8B ↑11 94.5
- 8 Aria ↑14 92.5
- 9
Qwen2.5-VL-72B ↑18 91.0
- 10
Qwen2.5-VL-7B ↑7 91.0
object_interaction
29 models we track have a score here.
- 1
InternVL2_5-78B ↑25 86.0
- 2
InternVL3-78B ↑27 84.0
- 3
InternVL2.5-8B ↑15 82.5
- 4
InternVL3-38B ↑24 82.5
- 5 LLaVA-Video-72B-Qwen2 ↑18 82.0
- 6 LLaVA-Video-7B-Qwen2 ↑10 82.0
- 7
Qwen2.5-VL-72B ↑20 80.5
- 8 Qwen2-VL-7B-Instruct ↑7 77.5
- 9
Qwen2.5-Omni-7B ↑10 77.0
- 10
InternVL3-8B ↑14 77.0
object_shuffle
29 models we track have a score here.
- 1 Aria ↑21 47.0
- 2 GPT-4o Aug 2024 ↑18 44.0
- 3 LLaVA-Video-7B-Qwen2 ↑13 44.0
- 4
InternVL3-78B ↑25 44.0
- 5
InternVL3-38B ↑23 42.5
- 6
InternVL2.5-8B ↑12 41.5
- 7 MiniCPM-o-2_6 ↑6 41.5
- 8
Phi-4-multimodal ↓6 41.5
- 9
InternVL3-8B ↑15 41.0
- 10
Qwen2.5-VL-72B ↑17 40.0
moving_direction
29 models we track have a score here.
- 1
InternVL3-78B ↑28 90.0
- 2
InternVL3-38B ↑26 88.0
- 3
InternVL2_5-78B ↑23 85.5
- 4
InternVL3-8B ↑20 79.0
- 5
InternVL2.5-8B ↑13 78.5
- 6
Qwen2.5-VL-72B ↑21 62.5
- 7 MiMo-VL-7B-RL ↑7 60.0
- 8
InternVL2-8B ↑2 58.0
- 9 Aria ↑13 51.5
- 10 Qwen2-VL-72B-Instruct ↑15 48.5
action_localization
29 models we track have a score here.
- 1 Qwen2-VL-72B-Instruct ↑24 73.0
- 2
InternVL3-38B ↑26 69.0
- 3
InternVL3-78B ↑26 68.5
- 4 LLaVA-Video-72B-Qwen2 ↑19 67.0
- 5
Qwen2.5-Omni-7B ↑14 66.5
- 6
Qwen2.5-VL-7B ↑11 66.0
- 7 LLaVA-Video-7B-Qwen2 ↑9 65.0
- 8
InternVL2_5-78B ↑18 64.0
- 9
Qwen2.5-VL-72B ↑18 64.0
- 10 Qwen2-VL-7B-Instruct ↑5 60.5
scene_transition
29 models we track have a score here.
- 1
InternVL3-78B ↑28 97.0
- 2 Qwen2-VL-72B-Instruct ↑23 95.0
- 3 Qwen2-VL-7B-Instruct ↑12 94.5
- 4 LLaVA-Video-72B-Qwen2 ↑19 94.0
- 5
Qwen2.5-VL-72B ↑22 93.5
- 6 GPT-4o Aug 2024 ↑14 93.0
- 7
Gemini 2.0 Flash Feb 2025 ↑14 93.0
- 8
InternVL3-38B ↑20 93.0
- 9
InternVL3-8B ↑15 93.0
- 10 LLaVA-Video-7B-Qwen2 ↑6 92.5
action_count
29 models we track have a score here.
- 1 Aria ↑21 71.5
- 2
InternVL3-78B ↑27 69.0
- 3
InternVL3-38B ↑25 67.0
- 4 MiniCPM-o-2_6 ↑9 58.0
- 5
InternVL3-8B ↑19 57.5
- 6
Qwen2.5-VL-72B ↑21 56.5
- 7 MiMo-VL-7B-RL ↑7 56.5
- 8
InternVL2_5-78B ↑18 56.0
- 9 LLaVA-Video-7B-Qwen2 ↑7 55.5
- 10 GPT-4o Aug 2024 ↑10 54.5
moving_count
29 models we track have a score here.
- 1
InternVL3-78B ↑28 92.5
- 2
InternVL2_5-78B ↑24 92.0
- 3
InternVL3-8B ↑21 91.0
- 4
InternVL2.5-8B ↑14 85.0
- 5 Qwen2-VL-72B-Instruct ↑20 83.5
- 6
Qwen2.5-Omni-7B ↑13 83.5
- 7
InternVL2-8B ↑3 83.0
- 8
Qwen2.5-VL-72B ↑19 82.0
- 9 Aria ↑13 80.5
- 10
Qwen2.5-VL-7B ↑7 77.5
moving_attribute
29 models we track have a score here.
- 1
InternVL3-78B ↑28 98.5
- 2
Qwen2.5-Omni-7B ↑17 98.0
- 3
InternVL3-8B ↑21 98.0
- 4
InternVL2_5-78B ↑22 97.5
- 5
InternVL3-38B ↑23 97.5
- 6
InternVL2-8B ↑4 97.0
- 7
InternVL2.5-8B ↑11 97.0
- 8 Qwen2-VL-72B-Instruct ↑17 96.5
- 9 Aria ↑13 95.5
- 10
Qwen2.5-VL-72B ↑17 94.5
state_change
29 models we track have a score here.
- 1
InternVL3-78B ↑28 81.5
- 2
InternVL3-38B ↑26 75.5
- 3 Aria ↑19 74.0
- 4
InternVL3-8B ↑20 71.5
- 5
Qwen2.5-VL-72B ↑22 68.5
- 6
InternVL2_5-78B ↑20 66.5
- 7
Qwen2.5-VL-7B ↑10 63.5
- 8
Qwen2.5-Omni-7B ↑11 61.0
- 9
InternVL2.5-8B ↑9 60.5
- 10 MiMo-VL-7B-RL ↑4 60.5
fine_grained_pose
29 models we track have a score here.
- 1
InternVL3-8B ↑23 79.5
- 2
InternVL2-8B ↑8 78.5
- 3
Qwen2.5-VL-72B ↑24 78.5
- 4
InternVL3-38B ↑24 78.5
- 5
InternVL2_5-78B ↑21 77.0
- 6
InternVL3-78B ↑23 76.5
- 7
Gemini 2.0 Flash Feb 2025 ↑14 76.0
- 8 GPT-4o Aug 2024 ↑12 74.5
- 9
InternVL2.5-8B ↑9 74.0
- 10 Qwen2-VL-7B-Instruct ↑5 68.5
character_order
29 models we track have a score here.
- 1
Qwen2.5-VL-72B ↑26 87.0
- 2
InternVL3-78B ↑27 86.5
- 3
InternVL3-38B ↑25 85.0
- 4
Qwen2.5-VL-7B ↑13 82.0
- 5 Aria ↑17 81.5
- 6 Qwen2-VL-72B-Instruct ↑19 80.0
- 7
InternVL2_5-78B ↑19 80.0
- 8
Qwen2.5-Omni-7B ↑11 80.0
- 9 LLaVA-Video-7B-Qwen2 ↑7 78.5
- 10 MiMo-VL-7B-RL ↑4 77.5
egocentric_navigation
29 models we track have a score here.
- 1
InternVL3-78B ↑28 46.5
- 2
InternVL2_5-78B ↑24 43.5
- 3 Qwen2-VL-7B-Instruct ↑12 42.5
- 4
Qwen2.5-Omni-7B ↑15 41.0
- 5 LLaVA-Video-72B-Qwen2 ↑18 39.0
- 6
Phi-4-multimodal ↓4 39.0
- 7
InternVL3-8B ↑17 39.0
- 8
Qwen2.5-VL-72B ↑19 38.5
- 9 MiniCPM-V-2_6 ↑2 38.0
- 10
Idefics3-8B-Llama3 ↓6 37.5
episodic_reasoning
29 models we track have a score here.
- 1 GPT-4o Aug 2024 ↑19 71.0
- 2
InternVL3-78B ↑27 70.5
- 3
InternVL3-8B ↑21 67.5
- 4
Gemini 2.0 Flash Feb 2025 ↑17 65.5
- 5
InternVL2_5-78B ↑21 65.0
- 6 LLaVA-Video-72B-Qwen2 ↑17 64.5
- 7
InternVL3-38B ↑21 64.5
- 8 Qwen2-VL-72B-Instruct ↑17 61.5
- 9 Aria ↑13 61.0
- 10
Qwen2.5-VL-72B ↑17 60.5
counterfactual_inference
29 models we track have a score here.
- 1
InternVL3-78B ↑28 85.5
- 2
InternVL3-38B ↑26 84.5
- 3
InternVL2_5-78B ↑23 82.0
- 4
InternVL3-8B ↑20 81.5
- 5 Aria ↑17 80.5
- 6 Qwen2-VL-72B-Instruct ↑19 76.0
- 7
InternVL2.5-8B ↑11 75.5
- 8
Qwen2.5-Omni-7B ↑11 75.5
- 9
Qwen2.5-VL-7B ↑8 72.5
- 10
InternVL2-8B 67.0
Overall
29 models we track have a score here.
- 1
InternVL3-78B ↑28 79.2
- 2
InternVL3-38B ↑26 76.0
- 3
InternVL2_5-78B ↑23 75.6
- 4
InternVL3-8B ↑20 73.2
- 5
Qwen2.5-VL-72B ↑22 71.3
- 6
InternVL2.5-8B ↑12 70.5
- 7 Qwen2-VL-72B-Instruct ↑18 70.2
- 8
Qwen2.5-Omni-7B ↑11 69.0
- 9 Aria ↑13 67.9
- 10
Qwen2.5-VL-7B ↑7 67.5
CP
29 models we track have a score here.
- 1
InternVL2_5-78B ↑25 2.08
- 2 Aria ↑20 2.05
- 3 GPT-4o Aug 2024 ↑17 2.04
- 4
Gemini 2.0 Flash Feb 2025 ↑17 2.04
- 5
Qwen2.5-VL-72B ↑22 1.97
- 6 MiniCPM-o-2_6 ↑7 1.88
- 7
InternVL3-38B ↑21 1.88
- 8 LLaVA-Video-72B-Qwen2 ↑15 1.87
- 9
InternVL3-78B ↑20 1.86
- 10 MiniCPM-V-2_6 ↑1 1.85
FP-S
29 models we track have a score here.
- 1
Gemini 2.0 Flash Feb 2025 ↑20 2.11
- 2
InternVL2_5-78B ↑24 2.10
- 3
InternVL3-78B ↑26 1.97
- 4
InternVL3-38B ↑24 1.96
- 5 MiniCPM-o-2_6 ↑8 1.91
- 6
Qwen2.5-VL-72B ↑21 1.91
- 7 GPT-4o Aug 2024 ↑13 1.89
- 8 Aria ↑14 1.86
- 9
InternVL3-8B ↑15 1.85
- 10 Qwen2-VL-72B-Instruct ↑15 1.84
FP-C
29 models we track have a score here.
- 1
Gemini 2.0 Flash Feb 2025 ↑20 1.80
- 2
InternVL2_5-78B ↑24 1.79
- 3 Aria ↑19 1.67
- 4
InternVL3-78B ↑25 1.67
- 5 GPT-4o Aug 2024 ↑15 1.66
- 6 LLaVA-Video-72B-Qwen2 ↑17 1.59
- 7 MiniCPM-o-2_6 ↑6 1.59
- 8
InternVL3-38B ↑20 1.56
- 9
InternVL2.5-8B ↑9 1.55
- 10
InternVL3-8B ↑14 1.54
HL
29 models we track have a score here.
- 1
Gemini 2.0 Flash Feb 2025 ↑20 2.24
- 2 GPT-4o Aug 2024 ↑18 2.15
- 3 Claude 3.5 Sonnet ↑2 1.40
- 4
Gemini 1.5 Flash May 2024 ↑8 1.29
- 5
Qwen2.5-VL-7B ↑12 1.24
- 6 MiniCPM-V-2_6 ↑5 1.05
- 7
Qwen2.5-VL-72B ↑20 1.00
- 8 Aria ↑14 0.92
- 9 MiniCPM-o-2_6 ↑4 0.92
- 10
InternVL2_5-78B ↑16 0.89
LR
29 models we track have a score here.
- 1
Gemini 2.0 Flash Feb 2025 ↑20 2.10
- 2
Qwen2.5-VL-72B ↑25 1.96
- 3
InternVL2_5-78B ↑23 1.88
- 4
InternVL3-78B ↑25 1.79
- 5 Qwen2-VL-72B-Instruct ↑20 1.77
- 6 GPT-4o Aug 2024 ↑14 1.75
- 7
InternVL3-38B ↑21 1.71
- 8
Qwen2.5-Omni-7B ↑11 1.58
- 9 MiMo-VL-7B-RL ↑5 1.57
- 10
Qwen2.5-VL-7B ↑7 1.54
AR
29 models we track have a score here.
- 1 GPT-4o Aug 2024 ↑19 2.04
- 2
InternVL2_5-78B ↑24 2.00
- 3
Gemini 2.0 Flash Feb 2025 ↑18 1.98
- 4
Qwen2.5-VL-72B ↑23 1.96
- 5 Aria ↑17 1.89
- 6
Gemini 1.5 Flash May 2024 ↑6 1.88
- 7
InternVL3-38B ↑21 1.88
- 8
InternVL3-78B ↑21 1.83
- 9 MiniCPM-V-2_6 ↑2 1.81
- 10
InternVL3-8B ↑14 1.81
RR
29 models we track have a score here.
- 1
InternVL2_5-78B ↑25 2.18
- 2
Qwen2.5-VL-72B ↑25 1.95
- 3
InternVL2.5-8B ↑15 1.94
- 4
InternVL3-38B ↑24 1.92
- 5 Aria ↑17 1.87
- 6
Gemini 2.0 Flash Feb 2025 ↑15 1.84
- 7
InternVL3-78B ↑22 1.84
- 8
InternVL3-8B ↑16 1.83
- 9 GPT-4o Aug 2024 ↑11 1.81
- 10
Gemini 1.5 Flash May 2024 ↑2 1.78
CSR
29 models we track have a score here.
- 1
Qwen2.5-VL-72B ↑26 1.94
- 2 GPT-4o Aug 2024 ↑18 1.93
- 3
InternVL2_5-78B ↑23 1.89
- 4 MiniCPM-o-2_6 ↑9 1.85
- 5 LLaVA-Video-72B-Qwen2 ↑18 1.80
- 6
Gemini 2.0 Flash Feb 2025 ↑15 1.78
- 7
Qwen2.5-Omni-7B ↑12 1.77
- 8
Gemini 1.5 Flash May 2024 ↑4 1.75
- 9
InternVL3-78B ↑20 1.75
- 10 Qwen2-VL-72B-Instruct ↑15 1.73
TR
29 models we track have a score here.
- 1
InternVL2_5-78B ↑25 1.77
- 2
Gemini 2.0 Flash Feb 2025 ↑19 1.67
- 3 Aria ↑19 1.65
- 4 GPT-4o Aug 2024 ↑16 1.62
- 5
Qwen2.5-VL-72B ↑22 1.62
- 6 MiniCPM-V-2_6 ↑5 1.60
- 7 MiniCPM-o-2_6 ↑6 1.60
- 8 Qwen2-VL-72B-Instruct ↑17 1.52
- 9 LLaVA-Video-72B-Qwen2 ↑14 1.51
- 10
InternVL3-38B ↑18 1.51
Perception
29 models we track have a score here.
- 1
Gemini 2.0 Flash Feb 2025 ↑20 2.07
- 2
InternVL2_5-78B ↑24 2.00
- 3 GPT-4o Aug 2024 ↑17 1.89
- 4
InternVL3-78B ↑25 1.84
- 5 Aria ↑17 1.83
- 6 MiniCPM-o-2_6 ↑7 1.83
- 7
Qwen2.5-VL-72B ↑20 1.83
- 8
InternVL3-38B ↑20 1.82
- 9 MiniCPM-V-2_6 ↑2 1.75
- 10 LLaVA-Video-72B-Qwen2 ↑13 1.74
Reasoning
29 models we track have a score here.
- 1
InternVL2_5-78B ↑25 1.94
- 2
Gemini 2.0 Flash Feb 2025 ↑19 1.86
- 3
Qwen2.5-VL-72B ↑24 1.85
- 4 GPT-4o Aug 2024 ↑16 1.81
- 5
InternVL3-38B ↑23 1.75
- 6 Aria ↑16 1.73
- 7
InternVL3-78B ↑22 1.72
- 8 Qwen2-VL-72B-Instruct ↑17 1.65
- 9 MiniCPM-o-2_6 ↑4 1.64
- 10
Gemini 1.5 Flash May 2024 ↑2 1.63
Overall
29 models we track have a score here.
- 1
Gemini 2.0 Flash Feb 2025 ↑20 2.01
- 2
InternVL2_5-78B ↑24 1.98
- 3 GPT-4o Aug 2024 ↑17 1.87
- 4
Qwen2.5-VL-72B ↑23 1.84
- 5 Aria ↑17 1.81
- 6
InternVL3-78B ↑23 1.81
- 7
InternVL3-38B ↑21 1.80
- 8 MiniCPM-o-2_6 ↑5 1.76
- 9 LLaVA-Video-72B-Qwen2 ↑14 1.71
- 10 MiniCPM-V-2_6 ↑1 1.70
action
29 models we track have a score here.
- 1 Qwen2-VL-72B-Instruct ↑24 97.1
- 2
Qwen2.5-VL-72B ↑25 97.0
- 3
InternVL3-38B ↑25 95.8
- 4 GPT-4o Aug 2024 ↑16 95.3
- 5
Qwen2.5-VL-7B ↑12 94.6
- 6
Qwen2.5-Omni-7B ↑13 93.8
- 7
InternVL3-78B ↑22 93.8
- 8 LLaVA-Video-72B-Qwen2 ↑15 93.7
- 9
InternVL3-8B ↑15 93.3
- 10 Qwen2-VL-7B-Instruct ↑5 93.2
multi-choice
29 models we track have a score here.
- 1 Qwen2-VL-72B-Instruct ↑24 77.8
- 2
InternVL3-38B ↑26 77.2
- 3
InternVL3-78B ↑26 76.0
- 4
Qwen2.5-VL-72B ↑23 75.1
- 5
Qwen2.5-VL-7B ↑12 74.2
- 6
InternVL2_5-78B ↑20 73.1
- 7 Qwen2-VL-7B-Instruct ↑8 70.4
- 8
InternVL3-8B ↑16 70.4
- 9 Aria ↑13 69.9
- 10 GPT-4o Aug 2024 ↑10 69.9
overall
29 models we track have a score here.
- 1 Qwen2-VL-72B-Instruct ↑24 79.4
- 2
InternVL3-38B ↑26 78.5
- 3
Qwen2.5-VL-72B ↑24 77.7
- 4
InternVL3-78B ↑25 77.0
- 5 GPT-4o Aug 2024 ↑15 72.7
- 6
Qwen2.5-VL-7B ↑11 72.0
- 7
Qwen2.5-Omni-7B ↑12 70.7
- 8
InternVL3-8B ↑16 70.4
- 9 LLaVA-Video-72B-Qwen2 ↑14 70.4
- 10 Aria ↑12 69.6
direction
29 models we track have a score here.
- 1 Qwen2-VL-72B-Instruct ↑24 65.8
- 2
InternVL3-78B ↑27 60.6
- 3
InternVL3-38B ↑25 60.4
- 4
Qwen2.5-VL-72B ↑23 59.8
- 5
Qwen2.5-VL-7B ↑12 56.0
- 6 GPT-4o Aug 2024 ↑14 55.9
- 7 Qwen2-VL-7B-Instruct ↑8 54.8
- 8
Qwen2.5-Omni-7B ↑11 54.0
- 9 Claude 3.5 Sonnet ↓4 51.7
- 10 Aria ↑12 51.4
speed
29 models we track have a score here.
- 1
InternVL3-38B ↑27 66.7
- 2
InternVL3-78B ↑27 65.0
- 3 Qwen2-VL-72B-Instruct ↑22 64.1
- 4
Qwen2.5-VL-72B ↑23 63.9
- 5 GPT-4o Aug 2024 ↑15 62.0
- 6 Aria ↑16 58.1
- 7
InternVL3-8B ↑17 56.6
- 8 LLaVA-Video-72B-Qwen2 ↑15 56.5
- 9 Claude 3.5 Sonnet ↓4 56.2
- 10
Qwen2.5-Omni-7B ↑9 55.8
order
29 models we track have a score here.
- 1
InternVL3-38B ↑27 87.8
- 2
InternVL3-78B ↑27 85.9
- 3
Qwen2.5-VL-72B ↑24 84.5
- 4 Qwen2-VL-72B-Instruct ↑21 84.0
- 5 LLaVA-Video-72B-Qwen2 ↑18 77.4
- 6
Qwen2.5-VL-7B ↑11 76.3
- 7
InternVL3-8B ↑17 75.9
- 8
Qwen2.5-Omni-7B ↑11 74.9
- 9 Aria ↑13 74.2
- 10
InternVL2.5-8B ↑8 72.5
attribute_change
29 models we track have a score here.
- 1 Qwen2-VL-72B-Instruct ↑24 86.3
- 2
Qwen2.5-VL-72B ↑25 84.0
- 3
InternVL3-38B ↑25 82.7
- 4
InternVL3-78B ↑25 80.3
- 5
Qwen2.5-VL-7B ↑12 78.4
- 6 GPT-4o Aug 2024 ↑14 77.6
- 7 Qwen2-VL-7B-Instruct ↑8 76.4
- 8
InternVL3-8B ↑16 75.9
- 9
Qwen2.5-Omni-7B ↑10 75.1
- 10
InternVL2.5-8B ↑8 74.5
caption_matching
29 models we track have a score here.
- 1 Qwen2-VL-72B-Instruct ↑24 86.4
- 2
InternVL3-38B ↑26 85.8
- 3
Qwen2.5-VL-72B ↑24 84.5
- 4
InternVL3-78B ↑25 84.5
- 5
InternVL2_5-78B ↑21 83.2
- 6 GPT-4o Aug 2024 ↑14 81.4
- 7
Qwen2.5-VL-7B ↑10 80.7
- 8 Qwen2-VL-7B-Instruct ↑7 80.4
- 9
InternVL2.5-8B ↑9 79.8
- 10 Aria ↑12 79.6
captioning
29 models we track have a score here.
- 1
InternVL3-78B ↑28 74.9
- 2
InternVL3-38B ↑26 74.8
- 3
Qwen2.5-VL-72B ↑24 73.9
- 4 Qwen2-VL-72B-Instruct ↑21 73.0
- 5 GPT-4o Aug 2024 ↑15 69.2
- 6 LLaVA-Video-72B-Qwen2 ↑17 64.9
- 7
Gemini 2.0 Flash Feb 2025 ↑14 63.8
- 8 Claude 3.5 Sonnet ↓3 63.4
- 9
Qwen2.5-Omni-7B ↑10 63.0
- 10
InternVL3-8B ↑14 61.1
yes_no
29 models we track have a score here.
- 1 Qwen2-VL-72B-Instruct ↑24 81.3
- 2
Qwen2.5-VL-72B ↑25 78.3
- 3
InternVL3-38B ↑25 77.9
- 4
Qwen2.5-VL-7B ↑13 75.8
- 5
InternVL3-78B ↑24 74.7
- 6
Gemini 2.0 Flash Feb 2025 ↑15 74.4
- 7 Qwen2-VL-7B-Instruct ↑8 74.0
- 8
InternVL2_5-78B ↑18 73.9
- 9
InternVL3-8B ↑15 72.8
- 10
Qwen2.5-Omni-7B ↑9 72.5
plotQA
29 models we track have a score here.
- 1
InternVL3-78B ↑28 84.2
- 2 LLaVA-Video-72B-Qwen2 ↑21 83.3
- 3
InternVL3-38B ↑25 82.0
- 4
InternVL2_5-78B ↑22 81.1
- 5
Gemini 2.0 Flash Feb 2025 ↑16 78.8
- 6 LLaVA-Video-7B-Qwen2 ↑10 78.3
- 7
InternVL3-8B ↑17 78.1
- 8
Qwen2.5-Omni-7B ↑11 74.2
- 9
InternVL2.5-8B ↑9 73.8
- 10 Qwen2-VL-72B-Instruct ↑15 72.5
needle
29 models we track have a score here.
- 1
InternVL3-78B ↑28 84.2
- 2 LLaVA-Video-72B-Qwen2 ↑21 81.4
- 3
InternVL3-38B ↑25 80.8
- 4 LLaVA-Video-7B-Qwen2 ↑12 79.4
- 5 Qwen2-VL-72B-Instruct ↑20 78.9
- 6
InternVL2_5-78B ↑20 78.6
- 7
InternVL3-8B ↑17 77.7
- 8 Aria ↑14 76.6
- 9 Qwen2-VL-7B-Instruct ↑6 76.1
- 10
Qwen2.5-VL-72B ↑17 75.2
ego
29 models we track have a score here.
- 1 LLaVA-Video-72B-Qwen2 ↑22 69.0
- 2
InternVL3-78B ↑27 69.0
- 3
InternVL3-38B ↑25 66.8
- 4 LLaVA-Video-7B-Qwen2 ↑12 65.6
- 5
Gemini 2.0 Flash Feb 2025 ↑16 65.1
- 6
InternVL2_5-78B ↑20 64.2
- 7 Qwen2-VL-72B-Instruct ↑18 63.9
- 8 Aria ↑14 63.1
- 9
InternVL3-8B ↑15 63.1
- 10
Qwen2.5-VL-72B ↑17 59.9
count
29 models we track have a score here.
- 1
InternVL3-78B ↑28 59.7
- 2
InternVL3-38B ↑26 58.7
- 3
InternVL2_5-78B ↑23 51.5
- 4 Aria ↑18 46.6
- 5 LLaVA-Video-7B-Qwen2 ↑11 45.1
- 6
InternVL3-8B ↑18 43.2
- 7 LLaVA-Video-72B-Qwen2 ↑16 40.3
- 8
InternVL2.5-8B ↑10 37.9
- 9
InternVL2-8B ↑1 35.9
- 10
Gemini 2.0 Flash Feb 2025 ↑11 35.0
order
29 models we track have a score here.
- 1
InternVL3-78B ↑28 79.5
- 2
InternVL3-38B ↑26 77.2
- 3
InternVL2_5-78B ↑23 73.4
- 4 LLaVA-Video-72B-Qwen2 ↑19 68.0
- 5 Aria ↑17 61.8
- 6
Gemini 2.0 Flash Feb 2025 ↑15 61.4
- 7 LLaVA-Video-7B-Qwen2 ↑9 60.6
- 8
InternVL3-8B ↑16 60.2
- 9
Qwen2.5-VL-72B ↑18 59.5
- 10 Qwen2-VL-72B-Instruct ↑15 58.7
anomaly_reco
29 models we track have a score here.
- 1
InternVL2_5-78B ↑25 84.0
- 2
InternVL3-38B ↑26 83.0
- 3
InternVL3-78B ↑26 82.0
- 4
InternVL3-8B ↑20 78.5
- 5
Qwen2.5-VL-72B ↑22 77.5
- 6
InternVL2.5-8B ↑12 77.0
- 7
Gemini 2.0 Flash Feb 2025 ↑14 77.0
- 8 Qwen2-VL-72B-Instruct ↑17 75.0
- 9
Qwen2.5-VL-7B ↑8 74.0
- 10
Qwen2.5-Omni-7B ↑9 72.0
topic_reasoning
29 models we track have a score here.
- 1
InternVL3-78B ↑28 90.9
- 2
InternVL2_5-78B ↑24 89.4
- 3
Qwen2.5-VL-72B ↑24 89.4
- 4
Gemini 2.0 Flash Feb 2025 ↑17 89.4
- 5
InternVL3-8B ↑19 89.4
- 6 LLaVA-Video-72B-Qwen2 ↑17 89.0
- 7 Qwen2-VL-72B-Instruct ↑18 88.2
- 8
InternVL3-38B ↑20 87.8
- 9 LLaVA-Video-7B-Qwen2 ↑7 87.1
- 10 Aria ↑12 86.3
sub_scene
29 models we track have a score here.
- 1
InternVL3-38B ↑27 534.3
- 2
InternVL2_5-78B ↑24 533.3
- 3
InternVL3-78B ↑26 516.9
- 4
Gemini 2.0 Flash Feb 2025 ↑17 503.0
- 5
Qwen2.5-Omni-7B ↑14 494.0
- 6 Qwen2-VL-72B-Instruct ↑19 488.1
- 7
Qwen2.5-VL-72B ↑20 481.1
- 8
Qwen2.5-VL-7B ↑9 481.1
- 9
InternVL2.5-8B ↑9 478.6
- 10
InternVL3-8B ↑14 476.1
summary
29 models we track have a score here.
- 1 GPT-4o Aug 2024 ↑19 5.09
- 2
Gemini 2.0 Flash Feb 2025 ↑19 4.71
- 3 Aria ↑19 4.63
- 4
Gemini 1.5 Flash May 2024 ↑8 4.62
- 5 MiniCPM-V-2_6 ↑6 4.59
- 6 Claude 3.5 Sonnet ↓1 4.56
- 7
Qwen2.5-VL-72B ↑20 4.32
- 8
InternVL3-38B ↑20 4.29
- 9
Qwen2.5-VL-7B ↑8 4.26
- 10 MiMo-VL-7B-RL ↑4 3.95
M-Avg
29 models we track have a score here.
- 1
InternVL3-78B ↑28 79.5
- 2
InternVL3-38B ↑26 77.4
- 3
InternVL2_5-78B ↑23 75.8
- 4 LLaVA-Video-72B-Qwen2 ↑19 75.2
- 5
InternVL3-8B ↑19 73.1
- 6 LLaVA-Video-7B-Qwen2 ↑10 73.0
- 7
Gemini 2.0 Flash Feb 2025 ↑14 71.6
- 8 Aria ↑14 70.3
- 9 Qwen2-VL-72B-Instruct ↑16 69.6
- 10
InternVL2.5-8B ↑8 68.5
G-Avg
29 models we track have a score here.
- 1 Aria ↑21 5.00
- 2
Gemini 2.0 Flash Feb 2025 ↑19 4.90
- 3
InternVL3-38B ↑25 4.80
- 4
Qwen2.5-VL-72B ↑23 4.60
- 5
InternVL2_5-78B ↑21 4.50
- 6
Qwen2.5-VL-7B ↑11 4.50
- 7
InternVL3-78B ↑22 4.50
- 8 MiniCPM-V-2_6 ↑3 4.45
- 9 GPT-4o Aug 2024 ↑11 4.37
- 10
Qwen2.5-Omni-7B ↑9 4.30
2 models have held number one on the official leaderboard since its release in September 2024
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official OpenVLM Video Leaderboard leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official OpenVLM Video Leaderboard leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Other
- Artificial Analysis Intelligence Index v5 In use Version 5 of Artificial Analysis's index: nine evaluations in four equal categories, most with private test sets, run three times each in its own agent harness. Claude Fable 5.2 64.6
- Personal Superintelligence Index (Meta) In use Meta's composite of seven personal and knowledge-work evals: Agents' Last Exam, AutomationBench-AA, FocusBench, JobBench, GDP.pdf, VBT and DRACO.
- Vals Science Index In use Agents are scored on the equal average of BioMysteryBench, MysteryMechanism and Terminal-Bench-Science. GPT-6.2 Astra 68.4%
- Decision Index Public v0.3 In use Decision models return probabilities over labeled options in public tasks covering knowledge, language, retrieval, tools, and arts. A chance-corrected weighted aggregate combines the task scores. Granite Switch 4.1 30B 51.3
- SAGE v1.1 (Vals) In use Models grade handwritten undergraduate math and computer science work against rubrics. Scored by class-balanced accuracy of each rubric check against expert grading. Claude Opus 5.5 91.3%
- MTEB Fading Massive Text Embedding Benchmark: retrieval, clustering, classification across languages and modalities.
Qwen3-Embedding-8B 72.2%
Data audit
8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 18 months ago, in May 2025, for MiMo-VL-7B-RL. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (37 entries).
- Pass Models scored 29 models we track, plus 8 other entries on its leaderboard.
- Warning Newest score The newest came 18 months ago, in May 2025, for MiMo-VL-7B-RL.
- Note Frontier models None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147).
- Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on huggingface.co: 29 rows for models we track and 8 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.