Benchmark Almanac

Video-MMMU

Qwen 3.8 Max 88.7% leads since August 2026
GPT-5 84.6% leads since August 2025
Qwen 3.8 Max 88.7% leads since August 2026
In use

300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy.

Frame sampling differs across labs.

Released
January 2025
Kind
Video understanding
Level
University
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Strongest model tested
Qwen 3.8 Max #29 on the capability index
Last in a launch post
August 2026
Last model added
August 2026
Scores refreshed
November 12th, 2026

In the authors’ words

Humans acquire knowledge through three cognitive stages: perceiving information, comprehending knowledge, and adapting knowledge to solve novel problems. Videos serve as an effective medium for this learning process, facilitating a progression through these cognitive stages. However, existing video benchmarks fail to systematically evaluate the knowledge acquisition capabilities in Large Multimodal Models (LMMs). To address this gap, we introduce Video-MMMU, a multi-modal, multi-disciplinary benchmark designed to assess LMMs' ability to acquire and utilize knowledge from videos. Video-MMMU features a curated collection of 300 expert-level videos and 900 human-annotated questions across six disciplines, evaluating knowledge acquisition through stage-aligned question-answer pairs: Perception, Comprehension, and Adaptation. A proposed knowledge gain metric, Δknowledge, quantifies improvement in performance after video viewing. Evaluation of LMMs reveals a steep decline in performance as cognitive demands increase and highlights a significant gap between human and model knowledge acquisition, underscoring the need for methods to enhance LMMs' capability to learn and adapt from videos.

Abstract of Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos (arXiv:2501.13826, CC0)
36
Models scored in lab cards
33
Launch posts list it, from 10 labs
16
System cards report it, from 8 labs

Qwen 3.8 Max leads with 88.7% among models released by August 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

GPT-5 leads with 84.6% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Qwen 3.8 Max leads with 88.7%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

The newest model with a score here came out in August 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

36 models. Scroll sideways for the rest.

  1. 88.7 Qwen 3.8 Max
  2. 88.3 Seed2.0 Lite (0428)
  3. 87.6 Gemini 3 Pro
  4. 86.9 Gemini 3 Flash
  5. 86.9 Seed2.0 Pro
  6. 86.6 Kimi K2.5
  7. 85.4 Qwen3.7-Plus
  8. 84.8 Gemini 3.1 Flash-Lite
  9. 84.7 Qwen3.5 397B-A17B
  10. 84.6 GPT-5
  11. 84.6 MiniMax-M3
  12. 84.4 Qwen3.6 27B
  13. 84.0 Qwen 3.6 Plus
  14. 83.7 Qwen 3.6 35B-A3B
  15. 83.6 Gemini 2.5 Pro
  16. 82.7 Seed1.8
  17. 82.5 GPT-5 mini
  18. 82.3 Qwen3.5-27B
  19. 82.0 Qwen3.5-122B-A10B
  20. 81.4 Seed1.5-VL
  21. 81.1 ERNIE 5.0
  22. 80.4 Qwen3.5-35B-A3B
  23. 80.0 Qwen3-VL-235B-A22B
  24. 80.0 Keye-VL-2.0-30B-A3B
  25. 79.2 Gemini 2.5 Flash
  26. 78.9 Qwen3.5-9B
  27. 74.7 GLM-4.6V
  28. 74.1 Qwen3.5-4B
  29. 72.4 GLM-4.5V
  30. 67.5 LongCat-Flash-Omni
  31. 66.8 GPT-5 nano
  32. 65.2 Kimi-VL-A3B-Thinking-2506
  33. 62.1 Qwen3.5-2B
  34. 61.0 GLM-4.1V-9B-Thinking
  35. 60.2 Qwen2.5-VL-72B
  36. 44.3 Qwen3.5-0.8B
  1. 1 Qwen 3.8 Max 88.7%
  2. 2 Seed2.0 Lite (0428) 88.3%
  3. 3 Gemini 3 Pro 87.6%
  4. 4 Gemini 3 Flash 86.9%
  5. 5 Seed2.0 Pro 86.9%
  6. 6 Kimi K2.5 86.6%
  7. 7 Qwen3.7-Plus 85.4%
  8. 8 Gemini 3.1 Flash-Lite 84.8%
  9. 9 Qwen3.5 397B-A17B 84.7%
  10. 10 GPT-5 84.6%
  11. 11 MiniMax-M3 84.6%
  12. 12 Qwen3.6 27B 84.4%
  13. 13 Qwen 3.6 Plus 84%
  14. 14 Qwen 3.6 35B-A3B 83.7%
  15. 15 Gemini 2.5 Pro Jun 2025 83.6%
  16. 16 Seed1.8 82.7%
  17. 17 GPT-5 mini 82.5%
  18. 18 Qwen3.5-27B 82.3%
  19. 19 Qwen3.5-122B-A10B 82%
  20. 20 Seed1.5-VL 81.4%
  21. 21 ERNIE 5.0 81.1%
  22. 22 Qwen3.5-35B-A3B 80.4%
  23. 23 Qwen3-VL-235B-A22B 80%
  24. 24 Keye-VL-2.0-30B-A3B 80%
  25. 25 Gemini 2.5 Flash Jun 2025 79.2%
  26. 26 Qwen3.5-9B 78.9%
  27. 27 GLM-4.6V 74.7%
  28. 28 Qwen3.5-4B 74.1%
  29. 29 GLM-4.5V 72.4%
  30. 30 LongCat-Flash-Omni 67.5%
  31. 31 GPT-5 nano 66.8%
  32. 32 Kimi-VL-A3B-Thinking-2506 65.2%
  33. 33 Qwen3.5-2B 62.1%
  34. 34 GLM-4.1V-9B-Thinking 61%
  35. 35 Qwen2.5-VL-72B 60.2%
  36. 36 Qwen3.5-0.8B 44.3%

Source: Labs’ model cards

24 models. Scroll sideways for the rest.

  1. 84.6 GPT-5
  2. 83.6 Gemini 2.5 Pro
  3. 83.3 o3
  4. 66.0 Keye-VL-1.5-8B
  5. 65.8 Claude 3.5 Sonnet (October 2024)
  6. 65.2 Kimi-VL-A3B-Thinking-2506
  7. 61.2 GPT-4o
  8. 60.2 Qwen2.5-VL-72B
  9. 53.9 Gemini 1.5 Pro
  10. 50.8 Aria
  11. 49.8 Gemini 1.5 Flash
  12. 49.7 LLaVA-Video-72B-Qwen2
  13. 48.3 LLaVA-OneVision 72B
  14. 47.4 Qwen2.5-VL-7B
  15. 47.0 VideoLLAMA3-7B
  16. 42.0 mPLUG-Owl3-7B
  17. 41.8 MAmmoTH-VL-8B
  18. 37.4 InternVL2-8B
  19. 36.1 LLaVA-Video-7B-Qwen2
  20. 34.0 VILA1.5-40B
  21. 33.9 LLaVA-OneVision 7B
  22. 30.0 Llama 3.2 11B
  23. 24.0 LongVA-7B
  24. 20.9 Llama-3-VILA1.5-8B
  1. 1 GPT-5 84.6%
  2. 2 Gemini 2.5 Pro Jun 2025 83.6%
  3. 3 o3 83.3%
  4. 4 Keye-VL-1.5-8B 66%
  5. 5 Claude 3.5 Sonnet (October 2024) 65.8%
  6. 6 Kimi-VL-A3B-Thinking-2506 65.2%
  7. 7 GPT-4o Aug 2024 61.2%
  8. 8 Qwen2.5-VL-72B 60.2%
  9. 9 Gemini 1.5 Pro Sep 2024 53.9%
  10. 10 Aria 50.8%
  11. 11 Gemini 1.5 Flash Sep 2024 49.8%
  12. 12 LLaVA-Video-72B-Qwen2 49.7%
  13. 13 LLaVA-OneVision 72B 48.3%
  14. 14 Qwen2.5-VL-7B 47.4%
  15. 15 VideoLLAMA3-7B 47%
  16. 16 mPLUG-Owl3-7B 42%
  17. 17 MAmmoTH-VL-8B 41.8%
  18. 18 InternVL2-8B 37.4%
  19. 19 LLaVA-Video-7B-Qwen2 36.1%
  20. 20 VILA1.5-40B 34%
  21. 21 LLaVA-OneVision 7B 33.9%
  22. 22 Llama 3.2 11B 30%
  23. 23 LongVA-7B 24%
  24. 24 Llama-3-VILA1.5-8B 20.9%

Source: Official Video-MMMU leaderboard

56 models. Scroll sideways for the rest.

  1. 88.7 Qwen 3.8 Max
  2. 88.3 Seed2.0 Lite (0428)
  3. 87.6 Gemini 3 Pro
  4. 86.9 Gemini 3 Flash
  5. 86.9 Seed2.0 Pro
  6. 86.6 Kimi K2.5
  7. 85.4 Qwen3.7-Plus
  8. 84.8 Gemini 3.1 Flash-Lite
  9. 84.7 Qwen3.5 397B-A17B
  10. 84.6 GPT-5
  11. 84.6 MiniMax-M3
  12. 84.4 Qwen3.6 27B
  13. 84.0 Qwen 3.6 Plus
  14. 83.7 Qwen 3.6 35B-A3B
  15. 83.6 Gemini 2.5 Pro
  16. 83.3 o3
  17. 82.7 Seed1.8
  18. 82.5 GPT-5 mini
  19. 82.3 Qwen3.5-27B
  20. 82.0 Qwen3.5-122B-A10B
  21. 81.4 Seed1.5-VL
  22. 81.1 ERNIE 5.0
  23. 80.4 Qwen3.5-35B-A3B
  24. 80.0 Qwen3-VL-235B-A22B
  25. 80.0 Keye-VL-2.0-30B-A3B
  26. 79.2 Gemini 2.5 Flash
  27. 78.9 Qwen3.5-9B
  28. 74.7 GLM-4.6V
  29. 74.1 Qwen3.5-4B
  30. 72.4 GLM-4.5V
  31. 67.5 LongCat-Flash-Omni
  32. 66.8 GPT-5 nano
  33. 66.0 Keye-VL-1.5-8B
  34. 65.8 Claude 3.5 Sonnet (October 2024)
  35. 65.2 Kimi-VL-A3B-Thinking-2506
  36. 62.1 Qwen3.5-2B
  37. 61.2 GPT-4o
  38. 61.0 GLM-4.1V-9B-Thinking
  39. 60.2 Qwen2.5-VL-72B
  40. 53.9 Gemini 1.5 Pro
  41. 50.8 Aria
  42. 49.8 Gemini 1.5 Flash
  43. 49.7 LLaVA-Video-72B-Qwen2
  44. 48.3 LLaVA-OneVision 72B
  45. 47.4 Qwen2.5-VL-7B
  46. 47.0 VideoLLAMA3-7B
  47. 44.3 Qwen3.5-0.8B
  48. 42.0 mPLUG-Owl3-7B
  49. 41.8 MAmmoTH-VL-8B
  50. 37.4 InternVL2-8B
  51. 36.1 LLaVA-Video-7B-Qwen2
  52. 34.0 VILA1.5-40B
  53. 33.9 LLaVA-OneVision 7B
  54. 30.0 Llama 3.2 11B
  55. 24.0 LongVA-7B
  56. 20.9 Llama-3-VILA1.5-8B
  1. 1 Qwen 3.8 Max 88.7%
  2. 2 Seed2.0 Lite (0428) 88.3%
  3. 3 Gemini 3 Pro 87.6%
  4. 4 Gemini 3 Flash 86.9%
  5. 5 Seed2.0 Pro 86.9%
  6. 6 Kimi K2.5 86.6%
  7. 7 Qwen3.7-Plus 85.4%
  8. 8 Gemini 3.1 Flash-Lite 84.8%
  9. 9 Qwen3.5 397B-A17B 84.7%
  10. 10 GPT-5 84.6%
  11. 11 MiniMax-M3 84.6%
  12. 12 Qwen3.6 27B 84.4%
  13. 13 Qwen 3.6 Plus 84%
  14. 14 Qwen 3.6 35B-A3B 83.7%
  15. 15 Gemini 2.5 Pro Jun 2025 83.6%
  16. 16 o3 83.3%
  17. 17 Seed1.8 82.7%
  18. 18 GPT-5 mini 82.5%
  19. 19 Qwen3.5-27B 82.3%
  20. 20 Qwen3.5-122B-A10B 82%
  21. 21 Seed1.5-VL 81.4%
  22. 22 ERNIE 5.0 81.1%
  23. 23 Qwen3.5-35B-A3B 80.4%
  24. 24 Qwen3-VL-235B-A22B 80%
  25. 25 Keye-VL-2.0-30B-A3B 80%
  26. 26 Gemini 2.5 Flash Jun 2025 79.2%
  27. 27 Qwen3.5-9B 78.9%
  28. 28 GLM-4.6V 74.7%
  29. 29 Qwen3.5-4B 74.1%
  30. 30 GLM-4.5V 72.4%
  31. 31 LongCat-Flash-Omni 67.5%
  32. 32 GPT-5 nano 66.8%
  33. 33 Keye-VL-1.5-8B 66%
  34. 34 Claude 3.5 Sonnet (October 2024) 65.8%
  35. 35 Kimi-VL-A3B-Thinking-2506 65.2%
  36. 36 Qwen3.5-2B 62.1%
  37. 37 GPT-4o Aug 2024 61.2%
  38. 38 GLM-4.1V-9B-Thinking 61%
  39. 39 Qwen2.5-VL-72B 60.2%
  40. 40 Gemini 1.5 Pro Sep 2024 53.9%
  41. 41 Aria 50.8%
  42. 42 Gemini 1.5 Flash Sep 2024 49.8%
  43. 43 LLaVA-Video-72B-Qwen2 49.7%
  44. 44 LLaVA-OneVision 72B 48.3%
  45. 45 Qwen2.5-VL-7B 47.4%
  46. 46 VideoLLAMA3-7B 47%
  47. 47 Qwen3.5-0.8B 44.3%
  48. 48 mPLUG-Owl3-7B 42%
  49. 49 MAmmoTH-VL-8B 41.8%
  50. 50 InternVL2-8B 37.4%
  51. 51 LLaVA-Video-7B-Qwen2 36.1%
  52. 52 VILA1.5-40B 34%
  53. 53 LLaVA-OneVision 7B 33.9%
  54. 54 Llama 3.2 11B 30%
  55. 55 LongVA-7B 24%
  56. 56 Llama-3-VILA1.5-8B 20.9%

Sources: Labs’ model cards and official Video-MMMU leaderboard

#ModelScoreSettingDateSource
1 Qwen 3.8 Max 88.7% Aug 2026 Lab model card
2 Seed2.0 Lite (0428) 88.3% Apr 2026 Lab model card
3 Gemini 3 Pro 87.6% Nov 2025 Lab model card
4 Gemini 3 Flash 86.9% Dec 2025 Lab model card
5 Seed2.0 Pro 86.9% Feb 2026 Lab model card
6 Kimi K2.5 86.6% Jan 2026 Lab model card
7 Qwen3.7-Plus 85.4% Jun 2026 Lab model card
8 Gemini 3.1 Flash-Lite 84.8% Mar 2026 Lab model card
9 Qwen3.5 397B-A17B 84.7% Feb 2026 Lab model card
10 GPT-5 84.6% Aug 2025 Lab model card
11 MiniMax-M3 84.6% Jun 2026 Lab model card
12 Qwen3.6 27B 84.4% Apr 2026 Lab model card
13 Qwen 3.6 Plus 84% Apr 2026 Lab model card
14 Qwen 3.6 35B-A3B 83.7% Apr 2026 Lab model card
15 Gemini 2.5 Pro Jun 2025 83.6% Jun 2025 Lab model card
16 Seed1.8 82.7% Dec 2025 Lab model card
17 GPT-5 mini 82.5% Aug 2025 Lab model card
18 Qwen3.5-27B 82.3% Feb 2026 Lab model card
19 Qwen3.5-122B-A10B 82% Feb 2026 Lab model card
20 Seed1.5-VL 81.4% May 2025 Lab model card
21 ERNIE 5.0 81.1% Feb 2026 Lab model card
22 Qwen3.5-35B-A3B 80.4% Feb 2026 Lab model card
23 Qwen3-VL-235B-A22B 80% Sep 2025 Lab model card
24 Keye-VL-2.0-30B-A3B 80% May 2026 Lab model card
25 Gemini 2.5 Flash Jun 2025 79.2% Jul 2025 Lab model card
26 Qwen3.5-9B 78.9% Mar 2026 Lab model card
27 GLM-4.6V 74.7% Dec 2025 Lab model card
28 Qwen3.5-4B 74.1% Mar 2026 Lab model card
29 GLM-4.5V 72.4% Aug 2025 Lab model card
30 LongCat-Flash-Omni 67.5% Oct 2025 Lab model card
31 GPT-5 nano 66.8% Aug 2025 Lab model card
32 Kimi-VL-A3B-Thinking-2506 65.2% Jun 2025 Lab model card
33 Qwen3.5-2B 62.1% Mar 2026 Lab model card
34 GLM-4.1V-9B-Thinking 61% Jul 2025 Lab model card
35 Qwen2.5-VL-72B 60.2% Feb 2025 Lab model card
36 Qwen3.5-0.8B 44.3% Mar 2026 Lab model card
#ModelScoreSettingDateSource
1 GPT-5 84.6% Aug 2025 Official leaderboard
2 Gemini 2.5 Pro Jun 2025 83.6% Jun 2025 Official leaderboard
3 o3 83.3% Apr 2025 Official leaderboard
4 Keye-VL-1.5-8B 66% Aug 2025 Official leaderboard
5 Claude 3.5 Sonnet (October 2024) 65.8% Oct 2024 Official leaderboard
6 Kimi-VL-A3B-Thinking-2506 65.2% Jun 2025 Official leaderboard
7 GPT-4o Aug 2024 61.2% Aug 2024 Official leaderboard
8 Qwen2.5-VL-72B 60.2% Jan 2025 Official leaderboard
9 Gemini 1.5 Pro Sep 2024 53.9% Sep 2024 Official leaderboard
10 Aria 50.8% Sep 2024 Official leaderboard
11 Gemini 1.5 Flash Sep 2024 49.8% Aug 2024 Official leaderboard
12 LLaVA-Video-72B-Qwen2 49.7% Sep 2024 Official leaderboard
13 LLaVA-OneVision 72B 48.3% Aug 2024 Official leaderboard
14 Qwen2.5-VL-7B 47.4% Jan 2025 Official leaderboard
15 VideoLLAMA3-7B 47% Jan 2024 Official leaderboard
16 mPLUG-Owl3-7B 42% Aug 2024 Official leaderboard
17 MAmmoTH-VL-8B 41.8% Nov 2024 Official leaderboard
18 InternVL2-8B 37.4% Jul 2024 Official leaderboard
19 LLaVA-Video-7B-Qwen2 36.1% Sep 2024 Official leaderboard
20 VILA1.5-40B 34% Apr 2024 Official leaderboard
21 LLaVA-OneVision 7B 33.9% Aug 2024 Official leaderboard
22 Llama 3.2 11B 30% Sep 2024 Official leaderboard
23 LongVA-7B 24% Jun 2024 Official leaderboard
24 Llama-3-VILA1.5-8B 20.9% Apr 2024 Official leaderboard
#ModelScoreSettingDateSource
1 Qwen 3.8 Max 88.7% Aug 2026 Lab model card
2 Seed2.0 Lite (0428) 88.3% Apr 2026 Lab model card
3 Gemini 3 Pro 87.6% Nov 2025 Lab model card
4 Gemini 3 Flash 86.9% Dec 2025 Lab model card
5 Seed2.0 Pro 86.9% Feb 2026 Lab model card
6 Kimi K2.5 86.6% Jan 2026 Lab model card
7 Qwen3.7-Plus 85.4% Jun 2026 Lab model card
8 Gemini 3.1 Flash-Lite 84.8% Mar 2026 Lab model card
9 Qwen3.5 397B-A17B 84.7% Feb 2026 Lab model card
10 GPT-5 84.6% Aug 2025 Official leaderboard
11 MiniMax-M3 84.6% Jun 2026 Lab model card
12 Qwen3.6 27B 84.4% Apr 2026 Lab model card
13 Qwen 3.6 Plus 84% Apr 2026 Lab model card
14 Qwen 3.6 35B-A3B 83.7% Apr 2026 Lab model card
15 Gemini 2.5 Pro Jun 2025 83.6% Jun 2025 Official leaderboard
16 o3 83.3% Apr 2025 Official leaderboard
17 Seed1.8 82.7% Dec 2025 Lab model card
18 GPT-5 mini 82.5% Aug 2025 Lab model card
19 Qwen3.5-27B 82.3% Feb 2026 Lab model card
20 Qwen3.5-122B-A10B 82% Feb 2026 Lab model card
21 Seed1.5-VL 81.4% May 2025 Lab model card
22 ERNIE 5.0 81.1% Feb 2026 Lab model card
23 Qwen3.5-35B-A3B 80.4% Feb 2026 Lab model card
24 Qwen3-VL-235B-A22B 80% Sep 2025 Lab model card
25 Keye-VL-2.0-30B-A3B 80% May 2026 Lab model card
26 Gemini 2.5 Flash Jun 2025 79.2% Jul 2025 Lab model card
27 Qwen3.5-9B 78.9% Mar 2026 Lab model card
28 GLM-4.6V 74.7% Dec 2025 Lab model card
29 Qwen3.5-4B 74.1% Mar 2026 Lab model card
30 GLM-4.5V 72.4% Aug 2025 Lab model card
31 LongCat-Flash-Omni 67.5% Oct 2025 Lab model card
32 GPT-5 nano 66.8% Aug 2025 Lab model card
33 Keye-VL-1.5-8B 66% Aug 2025 Official leaderboard
34 Claude 3.5 Sonnet (October 2024) 65.8% Oct 2024 Official leaderboard
35 Kimi-VL-A3B-Thinking-2506 65.2% Jun 2025 Official leaderboard
36 Qwen3.5-2B 62.1% Mar 2026 Lab model card
37 GPT-4o Aug 2024 61.2% Aug 2024 Official leaderboard
38 GLM-4.1V-9B-Thinking 61% Jul 2025 Lab model card
39 Qwen2.5-VL-72B 60.2% Jan 2025 Official leaderboard
40 Gemini 1.5 Pro Sep 2024 53.9% Sep 2024 Official leaderboard
41 Aria 50.8% Sep 2024 Official leaderboard
42 Gemini 1.5 Flash Sep 2024 49.8% Aug 2024 Official leaderboard
43 LLaVA-Video-72B-Qwen2 49.7% Sep 2024 Official leaderboard
44 LLaVA-OneVision 72B 48.3% Aug 2024 Official leaderboard
45 Qwen2.5-VL-7B 47.4% Jan 2025 Official leaderboard
46 VideoLLAMA3-7B 47% Jan 2024 Official leaderboard
47 Qwen3.5-0.8B 44.3% Mar 2026 Lab model card
48 mPLUG-Owl3-7B 42% Aug 2024 Official leaderboard
49 MAmmoTH-VL-8B 41.8% Nov 2024 Official leaderboard
50 InternVL2-8B 37.4% Jul 2024 Official leaderboard
51 LLaVA-Video-7B-Qwen2 36.1% Sep 2024 Official leaderboard
52 VILA1.5-40B 34% Apr 2024 Official leaderboard
53 LLaVA-OneVision 7B 33.9% Aug 2024 Official leaderboard
54 Llama 3.2 11B 30% Sep 2024 Official leaderboard
55 LongVA-7B 24% Jun 2024 Official leaderboard
56 Llama-3-VILA1.5-8B 20.9% Apr 2024 Official leaderboard

The benchmark's own leaderboard has 28 entries, 24 of them models we track. Switch the scores above to rank them together.

7 models have held number one in labs’ own numbers since its release in January 2025 4 models have held number one across all independent runs since its release in January 2025 7 models have held number one with claims included since its release in January 2025

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Anthropic OpenAI Google DeepMind Qwen ByteDance Seed
Own leaderboard Not tested since Aug 2025
GPT-5 84.6%
Lab claims Passed 85% in Nov 2025, by Gemini 3 Pro
Qwen 3.8 Max 88.7%

Sources: Official Video-MMMU leaderboard and labs’ model cards

Google DeepMind’s own number for Gemini 3 Pro passed 85% in November 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen 3.8 Max Gemini 3 Pro Gemini 2.5 Pro Claude 3.5 Sonnet (October 2024) GPT-4o VideoLLAMA3-7B 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen 3.8 Max Claude 3.5 Sonnet (October 2024)

Sources: Official Video-MMMU leaderboard and labs’ model cards

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

ByteDance Seed
Google DeepMind
Moonshot AI
Z.ai
OpenAI
Qwen
Meituan
Baidu
Kuaishou
MiniMax
DateLaunch postScoreAs the post writes it
Aug 3, 2026 Qwen 3.8 Max 88.7 VideoMMMU
Jun 1, 2026 Qwen3.7-Plus 85.4 VideoMMMU
Jun 1, 2026 MiniMax-M3 84.6 Video-MMMU
May 25, 2026 Keye-VL-2.0-30B-A3B 80 VideoMMMU
Apr 28, 2026 Seed2.0 Lite (0428) 88.3 VideoMMMU
Apr 22, 2026 Qwen3.6 27B 84.4 VideoMMMU
Apr 15, 2026 Qwen 3.6 35B-A3B 83.7 VideoMMMU
Apr 2, 2026 Qwen 3.6 Plus 84 VideoMMMU
Mar 3, 2026 Gemini 3.1 Flash-Lite 84.8 Video-MMMU
Mar 2, 2026 Qwen3.5-0.8B 44.3 VideoMMMU 44.3/-- (with CI / without CI)
Mar 2, 2026 Qwen3.5-2B 62.1 VideoMMMU 62.1/-- (with CI / without CI)
Mar 2, 2026 Qwen3.5-4B 74.1 VideoMMMU
Mar 2, 2026 Qwen3.5-9B 78.9 VideoMMMU
Feb 24, 2026 Qwen3.5-35B-A3B 80.4 VideoMMMU
Feb 24, 2026 Qwen3.5-27B 82.3 VideoMMMU
Feb 24, 2026 Qwen3.5-122B-A10B 82 VideoMMMU
Feb 16, 2026 Qwen3.5 397B-A17B 84.7 VideoMMMU
Feb 6, 2026 ERNIE 5.0 81.1 Video-MMMU
Jan 27, 2026 Kimi K2.5 86.6 VideoMMMU
Dec 18, 2025 Seed1.8 82.7 VideoMMMU
Dec 17, 2025 Gemini 3 Flash 86.9 Video-MMMU
Dec 8, 2025 GLM-4.6V 74.7 VideoMMMU
Nov 18, 2025 Gemini 3 Pro 87.6 Video-MMMU
Oct 31, 2025 LongCat-Flash-Omni 67.5 Video-MMMU
Sep 23, 2025 Qwen3-VL-235B-A22B 80 VideoMMMU
Aug 11, 2025 GLM-4.5V 72.4 VideoMMMU
Aug 7, 2025 GPT-5 nano 66.8 VideoMMMU, max frame 256
Aug 7, 2025 GPT-5 mini 82.5 VideoMMMU, max frame 256
Aug 7, 2025 GPT-5 84.6 VideoMMMU Video-based multimodal reasoning (max frame 256)
Jul 1, 2025 GLM-4.1V-9B-Thinking 61 VideoMMMU
Jun 21, 2025 Kimi-VL-A3B-Thinking-2506 65.2 VideoMMMU (Pass@1)
Jun 17, 2025 Gemini 2.5 Pro 83.6 VideoMMMU
May 12, 2025 Seed1.5-VL 81.4 Video-MMMU

16 system cards report it, from 8 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Jun 9, 2026 Keye-VL-2.0-30B-A3B technical report 80 Video-MMMU (Table 2) Capabilities
Mar 21, 2026 Seed1.8 model card 82.7 VideoMMMU (Table 3) Capabilities
Mar 3, 2026 Gemini 3.1 Flash-Lite model card 84.8 Video-MMMU (results table, p.4) Capabilities
Feb 14, 2026 Seed2.0 Pro model card 86.9 VideoMMMU (Table 9) Capabilities
Feb 4, 2026 ERNIE 5.0 technical report 81.1 Video-MMMU (Table 3) Capabilities
Jan 30, 2026 Kimi K2.5 technical report 86.6 VideoMMMU (Table 4) Capabilities
Jan 1, 2026 GLM-4.6V technical report 74.7 VideoMMMU, GLM-4.6V thinking (Table 2) Capabilities
Nov 26, 2025 Qwen3-VL-235B-A22B technical report 80 VideoMMMU (Table 2) Capabilities
Oct 31, 2025 LongCat-Flash-Omni technical report 67.5 Video-MMMU (Table 6) Capabilities
Aug 13, 2025 GLM-4.5V technical report 72.4 VideoMMMU, GLM-4.5V thinking (Table 2) Capabilities
Jul 7, 2025 Gemini 2.5 Pro technical report 83.6 VideoMMMU, test, 0-shot (Table 6) Capabilities
Jul 7, 2025 Gemini 2.5 Flash technical report 79.2 VideoMMMU, test, 0-shot (Table 6) Capabilities
Jul 1, 2025 GLM-4.1V-9B-Thinking technical report 61 VideoMMMU, GLM-4.1V-9B-Thinking thinking (Table 2) Capabilities
Jun 23, 2025 Kimi-VL-A3B-Thinking-2506 technical report 65.2 VideoMMMU (Pass@1) (Table 4) Capabilities
May 11, 2025 Seed1.5-VL technical report 81.4 Video-MMMU, thinking (Table 7). Non-thinking 72.1 Capabilities
Feb 19, 2025 Qwen2.5-VL-72B technical report 60.2 Video-MMMU (Table 8) Capabilities

4 lab claims next to the leaderboard

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 4 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
Kimi-VL-A3B-Thinking-2506 65.2% VideoMMMU (Pass@1) 65.2% Leaderboard Same
Qwen2.5-VL-72B 60.2% Video-MMMU (Table 8) 60.2% Leaderboard Same
Gemini 2.5 Pro Jun 2025 83.6% VideoMMMU 83.6% Leaderboard Same
GPT-5 84.6% VideoMMMU Video-based multimodal reasoning (max frame 256) 84.6% Leaderboard Same

More in Audio and video

All 233 in Audio and video
  • LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization. Gemini 4 Argon 91.7%
  • MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy. Qwen 3.8 Max 82.4%
  • MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. Qwen 3.8 Max 90.8%
  • FLEURS In use Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better. Phi-4-multimodal 4.00%
  • VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. Qwen3.5-Omni-Plus 93.1%
  • TVBench In use Models answer multiple-choice questions on 10 temporal video tasks, such as action counting and ordering, that static frames cannot solve. Scored by accuracy. Qwen 3.8 Max 81.9%