Benchmark Almanac

OpenVLM Video Leaderboard

Video-ChatGPT 37.8 tops its leaderboard SmolVLM2-500M-Video-Instruct 35.4 leads the models we track
Not in model cards

Models complete video understanding tasks. OpenCompass compares models across video evaluation datasets.

The leaderboard aggregates results from component evaluations.

Made by
OpenCompass
Released
September 2024
Kind
Composite indices
Level
University
Scored
Settled by events
Answers
Short answer
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Video → Text
Interaction
Single turn
Items from
Another benchmark
Refresh
One fixed set
Run by
Anyone
In views
Vision
Strongest model tested
Gemini 2.0 Flash Feb 2025 #147 on the capability index
Last model added
May 2025
Scores refreshed
November 20th, 2026
37
Entries on its own leaderboard, 29 models we track

Video-ChatGPT tops the leaderboard with 37.8 SmolVLM2-500M-Video-Instruct leads with 35.4 among models released by May 2025

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is SmolVLM2-500M-Video-Instruct, at 35.4. This view shows only models we track, each at its best.

The newest model with a score here came out in May 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 Video-ChatGPT 100 frames ✓ 37.8
  2. 2 SmolVLM2-500M-Video-Instruct 64 frames ✓ 35.4
  3. 3 Phi-4-multimodal 64 frames ✓ 34.8
  4. 4 LLaMA-VID-7B 1 fps frames ✓ 34.2
  5. 5 Chat-UniVi-7B-v1.5 1.0 fps frames ✓ 33.2
  6. 6 Video-LLaVA 8 frames ✓ 33.0
  7. 7 Idefics3-8B-Llama3 16 frames ✓ 31.8
  8. 8 Claude 3.5 Sonnet 8 frames ✓ 28.8
  9. 9 mPLUG-Owl3-7B 16 frames ✓ 28.0
  10. 10 SmolVLM2-2.2B-Instruct 64 frames ✓ 27.2
  11. 11 VILA1.5-40B 14 frames ✓ 27.0
  12. 12 Long-VITA-16K 64 frames ✓ 27.0
  13. 13 Phi-3.5-vision-instruct 16 frames ✓ 26.4
  14. 14 VideoChat2-HD 16 frames ✓ 25.8
  15. 15 PLLaVA-34B 16 frames ✓ 24.0
  16. 16 InternVL2-8B 16 frames ✓ 21.4
  17. 17 MiniCPM-V-2_6 64 frames ✓ 21.4
  18. 18 Gemini 1.5 Flash May 2024, 16 frames ✓ 21.0
  19. 19 MiniCPM-o-2_6 64 frames ✓ 20.4
  20. 20 MiMo-VL-7B-RL 64 frames ✓ 19.6
  21. 21 LLaVA-OneVision-72B-sft 32 frames ✓ 17.4
  22. 22 Qwen2-VL-7B-Instruct 64 frames ✓ 17.2
  23. 23 InternVL2-76B 16 frames ✓ 16.4
  24. 24 LLaVA-Video-7B-Qwen2 64 frames ✓ 15.6
  25. 25 Qwen2.5-VL-7B 64 frames ✓ 14.8
  26. 26 InternVL2.5-8B 64 frames ✓ 12.4
  27. 27 Qwen2.5-Omni-7B 64 frames ✓ 12.2
  28. 28 GPT-4o Aug 2024, 16 frames ✓ 12.0
  29. 29 Gemini 2.0 Flash Feb 2025, 64 frames ✓ 11.0
  30. 30 Aria 64 frames 9.60
  31. 31 LLaVA-Video-72B-Qwen2 64 frames ✓ 9.00
  32. 32 InternVL3-8B 64 frames ✓ 8.80
  33. 33 Qwen2-VL-72B-Instruct 64 frames ✓ 7.80
  34. 34 InternVL2_5-78B 64 frames ✓ 7.20
  35. 35 Qwen2.5-VL-72B 64 frames ✓ 6.40
  36. 36 InternVL3-38B 64 frames ✓ 3.60
  37. 37 InternVL3-78B 64 frames ✓ 3.00

Source: Official OpenVLM Video Leaderboard leaderboard

Source: Official OpenVLM Video Leaderboard leaderboard

#EntryScoreSettingReleasedRun onLinkSource
1 Video-ChatGPT ✓ open weights 37.8 100 frames May 2023 github.com Official leaderboard
2 SmolVLM2-500M-Video-Instruct ✓ 35.4 64 frames Feb 2025 Feb 2025 huggingface.co Official leaderboard
3 Phi-4-multimodal ✓ 34.8 64 frames Feb 2025 Feb 2025 huggingface.co Official leaderboard
4 LLaMA-VID-7B ✓ open weights 34.2 1 fps frames Nov 2023 huggingface.co Official leaderboard
5 Chat-UniVi-7B-v1.5 ✓ 33.2 1.0 fps frames Apr 2024 Apr 2024 huggingface.co Official leaderboard
6 Video-LLaVA ✓ open weights, 7.5B 33.0 8 frames Nov 2023 github.com Official leaderboard
7 Idefics3-8B-Llama3 ✓ 31.8 16 frames Aug 2024 Aug 2024 huggingface.co Official leaderboard
8 Claude 3.5 Sonnet ✓ 28.8 8 frames Jun 2024 Jun 2024 anthropic.com Official leaderboard
9 mPLUG-Owl3-7B ✓ 28.0 16 frames Aug 2024 Aug 2024 huggingface.co Official leaderboard
10 SmolVLM2-2.2B-Instruct ✓ 27.2 64 frames Feb 2025 Feb 2025 huggingface.co Official leaderboard
11 VILA1.5-40B ✓ 27.0 14 frames Apr 2024 May 2024 huggingface.co Official leaderboard
12 Long-VITA-16K ✓ open weights, 14B 27.0 64 frames Jun 2025 huggingface.co Official leaderboard
13 Phi-3.5-vision-instruct ✓ 26.4 16 frames Aug 2024 Aug 2024 huggingface.co Official leaderboard
14 VideoChat2-HD ✓ open weights 25.8 16 frames Jun 2024 huggingface.co Official leaderboard
15 PLLaVA-34B ✓ open weights, 34.9B 24.0 16 frames Apr 2024 huggingface.co Official leaderboard
16 InternVL2-8B ✓ 21.4 16 frames Jul 2024 Jun 2024 huggingface.co Official leaderboard
17 MiniCPM-V-2_6 ✓ 21.4 64 frames Aug 2024 Aug 2024 huggingface.co Official leaderboard
18 Gemini 1.5 Flash May 2024 ✓ 21.0 16 frames May 2024 May 2024 deepmind.google Official leaderboard
19 MiniCPM-o-2_6 ✓ 20.4 64 frames Jan 2025 Jan 2025 huggingface.co Official leaderboard
20 MiMo-VL-7B-RL ✓ 19.6 64 frames May 2025 Jun 2025 huggingface.co Official leaderboard
21 LLaVA-OneVision-72B-sft ✓ open weights, 73.2B 17.4 32 frames Aug 2024 huggingface.co Official leaderboard
22 Qwen2-VL-7B-Instruct ✓ 17.2 64 frames Aug 2024 Jun 2025 huggingface.co Official leaderboard
23 InternVL2-76B ✓ open weights, 76.3B 16.4 16 frames Jul 2024 huggingface.co Official leaderboard
24 LLaVA-Video-7B-Qwen2 ✓ 15.6 64 frames Sep 2024 Sep 2024 huggingface.co Official leaderboard
25 Qwen2.5-VL-7B ✓ 14.8 64 frames Jan 2025 Jun 2025 huggingface.co Official leaderboard
26 InternVL2.5-8B ✓ 12.4 64 frames Dec 2024 Nov 2024 huggingface.co Official leaderboard
27 Qwen2.5-Omni-7B ✓ 12.2 64 frames Mar 2025 Jun 2025 huggingface.co Official leaderboard
28 GPT-4o Aug 2024 ✓ 12.0 16 frames Aug 2024 Aug 2024 openai.com Official leaderboard
29 Gemini 2.0 Flash Feb 2025 ✓ 11.0 64 frames Feb 2025 Jun 2025 deepmind.google Official leaderboard
30 Aria 9.60 64 frames Sep 2024 Oct 2024 huggingface.co Official leaderboard
31 LLaVA-Video-72B-Qwen2 ✓ 9.00 64 frames Sep 2024 Sep 2024 huggingface.co Official leaderboard
32 InternVL3-8B ✓ 8.80 64 frames Apr 2025 Jun 2025 huggingface.co Official leaderboard
33 Qwen2-VL-72B-Instruct ✓ 7.80 64 frames Aug 2024 Jun 2025 huggingface.co Official leaderboard
34 InternVL2_5-78B ✓ 7.20 64 frames Dec 2024 Dec 2024 huggingface.co Official leaderboard
35 Qwen2.5-VL-72B ✓ 6.40 64 frames Jan 2025 Jun 2025 huggingface.co Official leaderboard
36 InternVL3-38B ✓ 3.60 64 frames Apr 2025 Jun 2025 huggingface.co Official leaderboard
37 InternVL3-78B ✓ 3.00 64 frames Apr 2025 Jun 2025 huggingface.co Official leaderboard
#ModelScoreSettingReleasedRun onLinkSource
1 SmolVLM2-500M-Video-Instruct ✓ 35.4 64 frames Feb 2025 Feb 2025 huggingface.co Official leaderboard
2 Phi-4-multimodal ✓ 34.8 64 frames Feb 2025 Feb 2025 huggingface.co Official leaderboard
3 Chat-UniVi-7B-v1.5 ✓ 33.2 1.0 fps frames Apr 2024 Apr 2024 huggingface.co Official leaderboard
4 Idefics3-8B-Llama3 ✓ 31.8 16 frames Aug 2024 Aug 2024 huggingface.co Official leaderboard
5 Claude 3.5 Sonnet ✓ 28.8 8 frames Jun 2024 Jun 2024 anthropic.com Official leaderboard
6 mPLUG-Owl3-7B ✓ 28.0 16 frames Aug 2024 Aug 2024 huggingface.co Official leaderboard
7 SmolVLM2-2.2B-Instruct ✓ 27.2 64 frames Feb 2025 Feb 2025 huggingface.co Official leaderboard
8 VILA1.5-40B ✓ 27.0 14 frames Apr 2024 May 2024 huggingface.co Official leaderboard
9 Phi-3.5-vision-instruct ✓ 26.4 16 frames Aug 2024 Aug 2024 huggingface.co Official leaderboard
10 InternVL2-8B ✓ 21.4 16 frames Jul 2024 Jun 2024 huggingface.co Official leaderboard
11 MiniCPM-V-2_6 ✓ 21.4 64 frames Aug 2024 Aug 2024 huggingface.co Official leaderboard
12 Gemini 1.5 Flash May 2024 ✓ 21.0 16 frames May 2024 May 2024 deepmind.google Official leaderboard
13 MiniCPM-o-2_6 ✓ 20.4 64 frames Jan 2025 Jan 2025 huggingface.co Official leaderboard
14 MiMo-VL-7B-RL ✓ 19.6 64 frames May 2025 Jun 2025 huggingface.co Official leaderboard
15 Qwen2-VL-7B-Instruct ✓ 17.2 64 frames Aug 2024 Jun 2025 huggingface.co Official leaderboard
16 LLaVA-Video-7B-Qwen2 ✓ 15.6 64 frames Sep 2024 Sep 2024 huggingface.co Official leaderboard
17 Qwen2.5-VL-7B ✓ 14.8 64 frames Jan 2025 Jun 2025 huggingface.co Official leaderboard
18 InternVL2.5-8B ✓ 12.4 64 frames Dec 2024 Nov 2024 huggingface.co Official leaderboard
19 Qwen2.5-Omni-7B ✓ 12.2 64 frames Mar 2025 Jun 2025 huggingface.co Official leaderboard
20 GPT-4o Aug 2024 ✓ 12.0 16 frames Aug 2024 Aug 2024 openai.com Official leaderboard
21 Gemini 2.0 Flash Feb 2025 ✓ 11.0 64 frames Feb 2025 Jun 2025 deepmind.google Official leaderboard
22 Aria 9.60 64 frames Sep 2024 Oct 2024 huggingface.co Official leaderboard
23 LLaVA-Video-72B-Qwen2 ✓ 9.00 64 frames Sep 2024 Sep 2024 huggingface.co Official leaderboard
24 InternVL3-8B ✓ 8.80 64 frames Apr 2025 Jun 2025 huggingface.co Official leaderboard
25 Qwen2-VL-72B-Instruct ✓ 7.80 64 frames Aug 2024 Jun 2025 huggingface.co Official leaderboard
26 InternVL2_5-78B ✓ 7.20 64 frames Dec 2024 Dec 2024 huggingface.co Official leaderboard
27 Qwen2.5-VL-72B ✓ 6.40 64 frames Jan 2025 Jun 2025 huggingface.co Official leaderboard
28 InternVL3-38B ✓ 3.60 64 frames Apr 2025 Jun 2025 huggingface.co Official leaderboard
29 InternVL3-78B ✓ 3.00 64 frames Apr 2025 Jun 2025 huggingface.co Official leaderboard

Rankings by part

We read 75 parts from the benchmark’s own leaderboard in 5 groups: video-mme (w/o subs), mvbench, mmbench-video, tempcompass and mlvu. Each ranks models on one part of the test. InternVL3-78B leads 25 of them, and 9 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.

Video-MME (w/o subs)

29 models we track have a score here.

  1. 1 InternVL3-78B ↑28 73.1
  2. 2 InternVL2_5-78B ↑24 72.1
  3. 3 InternVL3-38B ↑25 72.1
  4. 4 Gemini 2.0 Flash Feb 2025 ↑17 71.0
  5. 5 LLaVA-Video-72B-Qwen2 ↑18 70.5
  6. 6 Qwen2.5-VL-72B ↑21 68.6
  7. 7 GPT-4o Aug 2024 ↑13 67.9
  8. 8 Qwen2-VL-72B-Instruct ↑17 67.3
  9. 9 Aria ↑13 66.0
  10. 10 InternVL3-8B ↑14 66.0

2 models have held number one on the official leaderboard since its release in September 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Other labs Hugging Face
Own leaderboard Not tested since May 2025

Source: Official OpenVLM Video Leaderboard leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
0.00 10.0 20.0 30.0 40.0 2024 2025 0.00 10.0 20.0 30.0 40.0 2024 2025

Source: Official OpenVLM Video Leaderboard leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Other

All 69 in Other
  • Artificial Analysis Intelligence Index v5 In use Version 5 of Artificial Analysis's index: nine evaluations in four equal categories, most with private test sets, run three times each in its own agent harness. Claude Fable 5.2 64.6
  • Personal Superintelligence Index (Meta) In use Meta's composite of seven personal and knowledge-work evals: Agents' Last Exam, AutomationBench-AA, FocusBench, JobBench, GDP.pdf, VBT and DRACO.
  • Vals Science Index In use Agents are scored on the equal average of BioMysteryBench, MysteryMechanism and Terminal-Bench-Science. GPT-6.2 Astra 68.4%
  • Decision Index Public v0.3 In use Decision models return probabilities over labeled options in public tasks covering knowledge, language, retrieval, tools, and arts. A chance-corrected weighted aggregate combines the task scores. Granite Switch 4.1 30B 51.3
  • SAGE v1.1 (Vals) In use Models grade handwritten undergraduate math and computer science work against rubrics. Scored by class-balanced accuracy of each rubric check against expert grading. Claude Opus 5.5 91.3%
  • MTEB Fading Massive Text Embedding Benchmark: retrieval, clustering, classification across languages and modalities. Qwen3-Embedding-8B 72.2%

Data audit

8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 18 months ago, in May 2025, for MiMo-VL-7B-RL. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (37 entries).
  • Pass Models scored 29 models we track, plus 8 other entries on its leaderboard.
  • Warning Newest score The newest came 18 months ago, in May 2025, for MiMo-VL-7B-RL.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147).
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on huggingface.co: 29 rows for models we track and 8 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check