Benchmark Almanac

LongShOTBench

LongShOTAgent 66.7% tops its leaderboard Qwen3-Omni-30B-A3B-Thinking 64.2% leads the models we track
Not in model cards

Models answer single-turn and multi-turn questions about long videos using visual, spoken, and ambient audio evidence. Weighted rubrics award partial credit.

The suite contains 3,401 questions over 274 videos totaling 188 hours. Report the modality and question track with each score.

Made by
MBZUAI
Released
December 2025
Kind
Long videos
Level
Everyday
Scored
Graded by a model
Answers
Open-ended
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Video, Audio → Text
Interaction
Multi-turn
Refresh
One fixed set
Run by
Anyone
Strongest model tested
Gemini 3.1 Pro #40 on the capability index
Last model added
April 2026
Scores refreshed
November 20th, 2026

In the authors’ words

Long-form omni-modal video understanding requires integrating vision, speech, and ambient audio with coherent long-context reasoning. Existing video benchmarks often trade off temporal scale, modality coverage, open-ended interaction, and interpretable scoring. To address this gap, we introduce LongShOTBench, a long video understanding benchmark designed around three coupled goals: holistic omni-modal integration, intent-driven open-ended interaction, and rubric-level diagnosis. It builds single- and multi-turn questions from real viewing scenarios, with systematic tasks probing visual, speech, ambient-audio, temporal, and cross-modal reasoning. Each item includes a reference answer and a weighted criterion-level rubric, letting evaluation identify which perceptual facts, temporal links, modality-grounding requirements, and reasoning steps are satisfied or missed. All samples are manually verified to improve grounding, clarity, and rubric reliability. We also introduce LongShOTAgent, a training-free omni-modal evidence-seeking agent coupling full-video preprocessing with targeted retrieval, query-adaptive segment refinement, and explicit claim verification over visual, speech, and non-speech audio evidence. Its iterative search-refine-verify loop exposes intermediate evidence and lets modality-specific specialists re-analyze relevant moments before answering. We evaluate 105 video-capable models spanning open-source omni-modal models, vision-language systems, audio LLMs, agentic pipelines and closed-source APIs. Current MLLMs remain far from saturating LongShOTBench, while our LongShOTAgent is the strongest training-free system, reaching 66.64% overall. By releasing the benchmark, leaderboard, and method, we provide a shared, interpretable testbed for advancing long-form omni-modal video reasoning. Code, data, and the leaderboard are available at https://longshot.cvmbzuai.com/.

Abstract of A Benchmark for Omni-Modal Reasoning in Long Videos (arXiv:2512.16978, CC0)
106
Entries on its own leaderboard, 67 models we track

LongShOTAgent tops the leaderboard with 66.7% Qwen3-Omni-30B-A3B-Thinking leads with 64.2% among models released by April 2026

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Qwen3-Omni-30B-A3B-Thinking, at 64.2%. This view shows only models we track, each at its best.

The newest model with a score here came out in April 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

106 entries. Scroll sideways for the rest.

  1. 66.7 LongShOTAgent
  2. 64.2 Qwen3-Omni-30B-A3B-Thinking
  3. 57.0 Gemini 3.1 Pro
  4. 49.6 Voxtral-Mini-3B-2507
  5. 48.6 Voxtral Small
  6. 47.6 Qwen3-VL-235B-A22B
  7. 46.7 Qwen3-VL-32B-Thinking
  8. 46.2 Vero-Qwen3I-8B
  9. 45.1 Kimi K2.6
  10. 43.7 Qwen3-VL-30B-A3B-Thinking
  11. 42.6 Qwen3-VL-235B-A22B-Instruct
  12. 42.2 Qwen3-VL-8B-Thinking
  13. 42.0 Gemma 3 27B
  14. 41.5 Intern-S1-mini
  15. 41.2 Nemotron-3-Nano-Omni-30B-A3B-R
  16. 40.6 Qwen3-VL-32B-Instruct
  17. 40.1 Gemma 3 12B
  18. 38.9 Qwen3-Omni-30B-A3B
  19. 38.9 Gemma 4 31B IT
  20. 38.7 STEP3-VL-10B
  21. 37.8 GLM-4.5V
  22. 37.7 Qwen3-VL-4B-Thinking
  23. 36.9 MiMo-VL-7B-RL-2508
  24. 36.1 Qwen2.5-VL-32B
  25. 35.8 Gemma 3 4B
  26. 34.8 Gemma 4 26B A4B
  27. 34.5 Qwen3-VL-8B-Instruct
  28. 33.8 GLM-4.6V-Flash
  29. 32.9 Ovis2.6-30B-A3B
  30. 32.8 Qwen3.6 27B
  31. 32.3 Qwen2.5-VL-72B
  32. 31.9 Qwen 3.6 35B-A3B
  33. 31.8 Qwen3.5-27B
  34. 31.1 Qwen3.5-35B-A3B
  35. 30.8 Qwen3-VL-30B-A3B-Instruct
  36. 30.5 R-4B
  37. 30.4 Fun-Audio-Chat-8B
  38. 30.4 Qwen3.5-122B-A10B
  39. 29.9 InternVL3.5-241B-A28B
  40. 29.6 InternVL3-38B
  41. 29.4 GLM-4.1V-9B-Thinking
  42. 29.1 InternVL3-78B
  43. 28.4 Qwen3-VL-4B-Instruct
  44. 28.0 VideoRFT-7B
  45. 28.0 ERNIE-4.5-VL-28B-A3B
  46. 27.8 LongVT-RL
  47. 27.8 DeepEyes-7B
  48. 27.7 Ovis2.5-9B
  49. 27.7 Video-R2
  50. 27.6 Molmo2-8B
  51. 27.4 InternVL3-14B
  52. 27.3 Qwen3.5-9B
  53. 27.1 RePlan-Qwen2.5-VL-7B
  54. 26.9 Gemma 4 E4B
  55. 26.9 Video-R1-7B
  56. 26.9 TimeSearch-R
  57. 26.1 InternVL3.5-38B
  58. 26.1 InternVL3.5-8B
  59. 26.0 PixelReasoner-RL-v1
  60. 25.9 VideoChat-R1-7B
  61. 25.7 Qwen2.5-VL-7B
  62. 25.4 InternVL2_5-78B
  63. 25.0 Ovis2-16B
  64. 24.4 InternVL3-8B
  65. 23.9 Kimi-VL-A3B-Thinking
  66. 23.9 InternVL2.5-38B
  67. 23.4 Qwen2.5-VL-7B-VRAG
  68. 23.3 MiniCPM-o 4.5
  69. 23.1 Phi-4-Reasoning-Vision-15B
  70. 22.8 R1-Onevision-7B
  71. 22.6 Qwen3.5-4B
  72. 22.3 AURA-32B
  73. 22.2 InternVL3.5-4B-I
  74. 22.1 Keye-VL-1.5-8B
  75. 22.0 Audio Flamingo 3
  76. 21.8 Gemma 4 E2B
  77. 21.7 NVIDIA Nemotron Nano 12B VL
  78. 21.6 HumanSense Omni-R
  79. 21.5 EchoInk-R1-7B
  80. 21.3 MiniCPM-V 4.5
  81. 21.3 Qwen2.5-Omni-7B
  82. 20.6 SpaceThinker-3B
  83. 20.4 Qwen3-VL-2B-Instruct
  84. 19.9 Ovis2-8B
  85. 19.9 Qwen2.5-VL-3B
  86. 19.4 MiniCPM-o-2_6
  87. 19.0 Qwen3.5-2B
  88. 19.0 Baichuan-Omni-1.5
  89. 18.1 Qwen2.5-Omni-3B
  90. 17.7 SALMONN2+ 7B
  91. 17.7 Ovis2-4B
  92. 16.9 Qwen3.5-0.8B
  93. 16.0 OmniAtlas-Qwen2.5-7B
  94. 15.9 VideoLLAMA3-7B
  95. 15.8 Kimi-VL-A3B-Instruct
  96. 15.5 OmniVinci-9B
  97. 13.6 Monet-7B
  98. 13.2 LLaVA-OneVision 7B
  99. 13.1 Qwen2-Audio-7B-I
  100. 11.8 VideoMind
  101. 9.70 Tempo-6B
  102. 9.47 SenseNova-SI-1.3-8B
  103. 8.17 Video-RAG
  104. 7.99 Vgent
  105. 7.26 OneThinker-8B
  106. 3.17 VideoScore2
  1. 1 LongShOTAgent input: V+A 66.7%
  2. 2 Qwen3-Omni-30B-A3B-Thinking input: V+A, thinking 64.2%
  3. 3 Gemini 3.1 Pro input: V+A 57%
  4. 4 Voxtral-Mini-3B-2507 input: Audio 49.6%
  5. 5 Voxtral Small input: Audio 48.6%
  6. 6 Qwen3-VL-235B-A22B input: Video, thinking 47.6%
  7. 7 Qwen3-VL-32B-Thinking input: Video, thinking 46.7%
  8. 8 Vero-Qwen3I-8B input: Video 46.2%
  9. 9 Kimi K2.6 input: Video 45.1%
  10. 10 Qwen3-VL-30B-A3B-Thinking input: Video, thinking 43.7%
  11. 11 Qwen3-VL-235B-A22B-Instruct input: Video, instruct 42.6%
  12. 12 Qwen3-VL-8B-Thinking input: Video, thinking 42.2%
  13. 13 Gemma 3 27B input: Frames 42%
  14. 14 Intern-S1-mini input: Video 41.5%
  15. 15 Nemotron-3-Nano-Omni-30B-A3B-R input: V+A 41.2%
  16. 16 Qwen3-VL-32B-Instruct input: Video, instruct 40.6%
  17. 17 Gemma 3 12B input: Frames 40.1%
  18. 18 Qwen3-Omni-30B-A3B input: V+A, instruct 38.9%
  19. 19 Gemma 4 31B IT input: Frames 38.9%
  20. 20 STEP3-VL-10B input: Video 38.7%
  21. 21 GLM-4.5V input: Video 37.8%
  22. 22 Qwen3-VL-4B-Thinking input: Video, thinking 37.7%
  23. 23 MiMo-VL-7B-RL-2508 input: Video 36.9%
  24. 24 Qwen2.5-VL-32B input: Video, instruct 36.1%
  25. 25 Gemma 3 4B input: Frames 35.8%
  26. 26 Gemma 4 26B A4B input: Frames 34.8%
  27. 27 Qwen3-VL-8B-Instruct input: Video, instruct 34.5%
  28. 28 GLM-4.6V-Flash input: Video 33.8%
  29. 29 Ovis2.6-30B-A3B input: Video 32.9%
  30. 30 Qwen3.6 27B input: Video 32.8%
  31. 31 Qwen2.5-VL-72B input: Video, instruct 32.3%
  32. 32 Qwen 3.6 35B-A3B input: Video 31.9%
  33. 33 Qwen3.5-27B input: Video 31.8%
  34. 34 Qwen3.5-35B-A3B input: Video 31.1%
  35. 35 Qwen3-VL-30B-A3B-Instruct input: Video, instruct 30.8%
  36. 36 R-4B input: Video 30.5%
  37. 37 Fun-Audio-Chat-8B input: Audio 30.4%
  38. 38 Qwen3.5-122B-A10B input: Video 30.4%
  39. 39 InternVL3.5-241B-A28B input: Video 29.9%
  40. 40 InternVL3-38B input: Video 29.6%
  41. 41 GLM-4.1V-9B-Thinking input: Video, thinking 29.4%
  42. 42 InternVL3-78B input: Video 29.1%
  43. 43 Qwen3-VL-4B-Instruct input: Video, instruct 28.4%
  44. 44 VideoRFT-7B input: Video 28%
  45. 45 ERNIE-4.5-VL-28B-A3B input: Video 28%
  46. 46 LongVT-RL input: Video 27.8%
  47. 47 DeepEyes-7B input: Video 27.8%
  48. 48 Ovis2.5-9B input: Video 27.7%
  49. 49 Video-R2 input: Video 27.7%
  50. 50 Molmo2-8B input: Video 27.6%
  51. 51 InternVL3-14B input: Video 27.4%
  52. 52 Qwen3.5-9B input: Video 27.3%
  53. 53 RePlan-Qwen2.5-VL-7B input: Video 27.1%
  54. 54 Gemma 4 E4B input: V+A 26.9%
  55. 55 Video-R1-7B input: Video 26.9%
  56. 56 TimeSearch-R input: Video 26.9%
  57. 57 InternVL3.5-38B input: Video 26.1%
  58. 58 InternVL3.5-8B input: Video 26.1%
  59. 59 PixelReasoner-RL-v1 input: Video 26%
  60. 60 VideoChat-R1-7B input: Video 25.9%
  61. 61 Qwen2.5-VL-7B input: Video, instruct 25.7%
  62. 62 InternVL2_5-78B input: Video 25.4%
  63. 63 Ovis2-16B input: Frames 25%
  64. 64 InternVL3-8B input: Video 24.4%
  65. 65 Kimi-VL-A3B-Thinking input: Frames, thinking 23.9%
  66. 66 InternVL2.5-38B input: Video 23.9%
  67. 67 Qwen2.5-VL-7B-VRAG input: Video 23.4%
  68. 68 MiniCPM-o 4.5 input: V+A 23.3%
  69. 69 Phi-4-Reasoning-Vision-15B input: Video 23.1%
  70. 70 R1-Onevision-7B input: Video 22.8%
  71. 71 Qwen3.5-4B input: Video 22.6%
  72. 72 AURA-32B input: Video 22.3%
  73. 73 InternVL3.5-4B-I input: Video, instruct 22.2%
  74. 74 Keye-VL-1.5-8B input: Video 22.1%
  75. 75 Audio Flamingo 3 input: Audio 22%
  76. 76 Gemma 4 E2B input: V+A 21.8%
  77. 77 NVIDIA Nemotron Nano 12B VL input: Video 21.7%
  78. 78 HumanSense Omni-R input: V+A 21.6%
  79. 79 EchoInk-R1-7B input: V+A 21.5%
  80. 80 MiniCPM-V 4.5 input: Video 21.3%
  81. 81 Qwen2.5-Omni-7B input: V+A 21.3%
  82. 82 SpaceThinker-3B input: Video 20.6%
  83. 83 Qwen3-VL-2B-Instruct input: Video, instruct 20.4%
  84. 84 Ovis2-8B input: Frames 19.9%
  85. 85 Qwen2.5-VL-3B input: Video, instruct 19.9%
  86. 86 MiniCPM-o-2_6 input: V+A 19.4%
  87. 87 Qwen3.5-2B input: Video 19%
  88. 88 Baichuan-Omni-1.5 input: V+A 19%
  89. 89 Qwen2.5-Omni-3B input: V+A 18.1%
  90. 90 SALMONN2+ 7B input: V+A 17.7%
  91. 91 Ovis2-4B input: Frames 17.7%
  92. 92 Qwen3.5-0.8B input: Video 16.9%
  93. 93 OmniAtlas-Qwen2.5-7B input: V+A 16%
  94. 94 VideoLLAMA3-7B input: Video 15.9%
  95. 95 Kimi-VL-A3B-Instruct input: Frames, instruct 15.8%
  96. 96 OmniVinci-9B input: V+A 15.5%
  97. 97 Monet-7B input: Video 13.6%
  98. 98 LLaVA-OneVision 7B input: Video 13.2%
  99. 99 Qwen2-Audio-7B-I input: Audio, instruct 13.1%
  100. 100 VideoMind input: V+A 11.8%
  101. 101 Tempo-6B input: Video 9.70%
  102. 102 SenseNova-SI-1.3-8B input: Video 9.47%
  103. 103 Video-RAG input: V+A 8.17%
  104. 104 Vgent input: V+A 7.99%
  105. 105 OneThinker-8B input: Video 7.26%
  106. 106 VideoScore2 input: Video 3.17%

Source: Official LongShOTBench leaderboard

  1. 1 Qwen3-Omni-30B-A3B-Thinking input: V+A, thinking 64.2%
  2. 2 Gemini 3.1 Pro input: V+A 57%
  3. 3 Voxtral Small input: Audio 48.6%
  4. 4 Qwen3-VL-235B-A22B input: Video, thinking 47.6%
  5. 5 Qwen3-VL-32B-Thinking input: Video, thinking 46.7%
  6. 6 Kimi K2.6 input: Video 45.1%
  7. 7 Qwen3-VL-30B-A3B-Thinking input: Video, thinking 43.7%
  8. 8 Qwen3-VL-235B-A22B-Instruct input: Video, instruct 42.6%
  9. 9 Qwen3-VL-8B-Thinking input: Video, thinking 42.2%
  10. 10 Gemma 3 27B input: Frames 42%
  11. 11 Qwen3-VL-32B-Instruct input: Video, instruct 40.6%
  12. 12 Gemma 3 12B input: Frames 40.1%
  13. 13 Qwen3-Omni-30B-A3B input: V+A, instruct 38.9%
  14. 14 Gemma 4 31B IT input: Frames 38.9%
  15. 15 STEP3-VL-10B input: Video 38.7%
  16. 16 GLM-4.5V input: Video 37.8%
  17. 17 Qwen3-VL-4B-Thinking input: Video, thinking 37.7%
  18. 18 MiMo-VL-7B-RL-2508 input: Video 36.9%
  19. 19 Qwen2.5-VL-32B input: Video, instruct 36.1%
  20. 20 Gemma 3 4B input: Frames 35.8%
  21. 21 Gemma 4 26B A4B input: Frames 34.8%
  22. 22 Qwen3-VL-8B-Instruct input: Video, instruct 34.5%
  23. 23 GLM-4.6V-Flash input: Video 33.8%
  24. 24 Qwen3.6 27B input: Video 32.8%
  25. 25 Qwen2.5-VL-72B input: Video, instruct 32.3%
  26. 26 Qwen 3.6 35B-A3B input: Video 31.9%
  27. 27 Qwen3.5-27B input: Video 31.8%
  28. 28 Qwen3.5-35B-A3B input: Video 31.1%
  29. 29 Qwen3-VL-30B-A3B-Instruct input: Video, instruct 30.8%
  30. 30 Qwen3.5-122B-A10B input: Video 30.4%
  31. 31 InternVL3.5-241B-A28B input: Video 29.9%
  32. 32 InternVL3-38B input: Video 29.6%
  33. 33 GLM-4.1V-9B-Thinking input: Video, thinking 29.4%
  34. 34 InternVL3-78B input: Video 29.1%
  35. 35 Qwen3-VL-4B-Instruct input: Video, instruct 28.4%
  36. 36 Ovis2.5-9B input: Video 27.7%
  37. 37 Molmo2-8B input: Video 27.6%
  38. 38 InternVL3-14B input: Video 27.4%
  39. 39 Qwen3.5-9B input: Video 27.3%
  40. 40 Gemma 4 E4B input: V+A 26.9%
  41. 41 InternVL3.5-38B input: Video 26.1%
  42. 42 InternVL3.5-8B input: Video 26.1%
  43. 43 Qwen2.5-VL-7B input: Video, instruct 25.7%
  44. 44 InternVL2_5-78B input: Video 25.4%
  45. 45 Ovis2-16B input: Frames 25%
  46. 46 InternVL3-8B input: Video 24.4%
  47. 47 Kimi-VL-A3B-Thinking input: Frames, thinking 23.9%
  48. 48 InternVL2.5-38B input: Video 23.9%
  49. 49 MiniCPM-o 4.5 input: V+A 23.3%
  50. 50 Phi-4-Reasoning-Vision-15B input: Video 23.1%
  51. 51 Qwen3.5-4B input: Video 22.6%
  52. 52 Keye-VL-1.5-8B input: Video 22.1%
  53. 53 Audio Flamingo 3 input: Audio 22%
  54. 54 Gemma 4 E2B input: V+A 21.8%
  55. 55 MiniCPM-V 4.5 input: Video 21.3%
  56. 56 Qwen2.5-Omni-7B input: V+A 21.3%
  57. 57 Qwen3-VL-2B-Instruct input: Video, instruct 20.4%
  58. 58 Ovis2-8B input: Frames 19.9%
  59. 59 Qwen2.5-VL-3B input: Video, instruct 19.9%
  60. 60 MiniCPM-o-2_6 input: V+A 19.4%
  61. 61 Qwen3.5-2B input: Video 19%
  62. 62 Baichuan-Omni-1.5 input: V+A 19%
  63. 63 Ovis2-4B input: Frames 17.7%
  64. 64 Qwen3.5-0.8B input: Video 16.9%
  65. 65 VideoLLAMA3-7B input: Video 15.9%
  66. 66 Kimi-VL-A3B-Instruct input: Frames, instruct 15.8%
  67. 67 LLaVA-OneVision 7B input: Video 13.2%

Source: Official LongShOTBench leaderboard

#EntryScoreSettingDateSource
1 LongShOTAgent 66.7% input: V+A, LongShOTAgent Official leaderboard
2 Qwen3-Omni-30B-A3B-Thinking 64.2% input: V+A, thinking Sep 2025 Official leaderboard
3 Gemini 3.1 Pro 57% input: V+A Feb 2026 Official leaderboard
4 Voxtral-Mini-3B-2507 3B 49.6% input: Audio Official leaderboard
5 Voxtral Small 48.6% input: Audio Jul 2025 Official leaderboard
6 Qwen3-VL-235B-A22B 47.6% input: Video, thinking Sep 2025 Official leaderboard
7 Qwen3-VL-32B-Thinking 46.7% input: Video, thinking Oct 2025 Official leaderboard
8 Vero-Qwen3I-8B 8B 46.2% input: Video Official leaderboard
9 Kimi K2.6 45.1% input: Video Apr 2026 Official leaderboard
10 Qwen3-VL-30B-A3B-Thinking 43.7% input: Video, thinking Oct 2025 Official leaderboard
11 Qwen3-VL-235B-A22B-Instruct 42.6% input: Video, instruct Sep 2025 Official leaderboard
12 Qwen3-VL-8B-Thinking 42.2% input: Video, thinking Oct 2025 Official leaderboard
13 Gemma 3 27B 42% input: Frames Mar 2025 Official leaderboard
14 Intern-S1-mini 14B 41.5% input: Video Official leaderboard
15 Nemotron-3-Nano-Omni-30B-A3B-R 30B 41.2% input: V+A Official leaderboard
16 Qwen3-VL-32B-Instruct 40.6% input: Video, instruct Oct 2025 Official leaderboard
17 Gemma 3 12B 40.1% input: Frames Mar 2025 Official leaderboard
18 Qwen3-Omni-30B-A3B 38.9% input: V+A, instruct Sep 2025 Official leaderboard
19 Gemma 4 31B IT 38.9% input: Frames Apr 2026 Official leaderboard
20 STEP3-VL-10B 38.7% input: Video Jan 2026 Official leaderboard
21 GLM-4.5V 37.8% input: Video Aug 2025 Official leaderboard
22 Qwen3-VL-4B-Thinking 37.7% input: Video, thinking Oct 2025 Official leaderboard
23 MiMo-VL-7B-RL-2508 36.9% input: Video Aug 2025 Official leaderboard
24 Qwen2.5-VL-32B 36.1% input: Video, instruct Mar 2025 Official leaderboard
25 Gemma 3 4B 35.8% input: Frames Mar 2025 Official leaderboard
26 Gemma 4 26B A4B 34.8% input: Frames Apr 2026 Official leaderboard
27 Qwen3-VL-8B-Instruct 34.5% input: Video, instruct Oct 2025 Official leaderboard
28 GLM-4.6V-Flash 33.8% input: Video Dec 2025 Official leaderboard
29 Ovis2.6-30B-A3B 30B 32.9% input: Video Official leaderboard
30 Qwen3.6 27B 32.8% input: Video Apr 2026 Official leaderboard
31 Qwen2.5-VL-72B 32.3% input: Video, instruct Jan 2025 Official leaderboard
32 Qwen 3.6 35B-A3B 31.9% input: Video Apr 2026 Official leaderboard
33 Qwen3.5-27B 31.8% input: Video Feb 2026 Official leaderboard
34 Qwen3.5-35B-A3B 31.1% input: Video Feb 2026 Official leaderboard
35 Qwen3-VL-30B-A3B-Instruct 30.8% input: Video, instruct Oct 2025 Official leaderboard
36 R-4B 4B 30.5% input: Video Official leaderboard
37 Fun-Audio-Chat-8B 8B 30.4% input: Audio Official leaderboard
38 Qwen3.5-122B-A10B 30.4% input: Video Feb 2026 Official leaderboard
39 InternVL3.5-241B-A28B 29.9% input: Video Aug 2025 Official leaderboard
40 InternVL3-38B 29.6% input: Video Apr 2025 Official leaderboard
41 GLM-4.1V-9B-Thinking 29.4% input: Video, thinking Jul 2025 Official leaderboard
42 InternVL3-78B 29.1% input: Video Apr 2025 Official leaderboard
43 Qwen3-VL-4B-Instruct 28.4% input: Video, instruct Oct 2025 Official leaderboard
44 VideoRFT-7B 7B 28% input: Video Official leaderboard
45 ERNIE-4.5-VL-28B-A3B 28B 28% input: Video Official leaderboard
46 LongVT-RL 7B 27.8% input: Video Official leaderboard
47 DeepEyes-7B 7B 27.8% input: Video Official leaderboard
48 Ovis2.5-9B 27.7% input: Video Aug 2025 Official leaderboard
49 Video-R2 7B 27.7% input: Video Official leaderboard
50 Molmo2-8B 27.6% input: Video Dec 2025 Official leaderboard
51 InternVL3-14B 27.4% input: Video Apr 2025 Official leaderboard
52 Qwen3.5-9B 27.3% input: Video Feb 2026 Official leaderboard
53 RePlan-Qwen2.5-VL-7B 7B 27.1% input: Video Official leaderboard
54 Gemma 4 E4B 26.9% input: V+A Mar 2026 Official leaderboard
55 Video-R1-7B 7B 26.9% input: Video Official leaderboard
56 TimeSearch-R 7B 26.9% input: Video Official leaderboard
57 InternVL3.5-38B 26.1% input: Video Aug 2025 Official leaderboard
58 InternVL3.5-8B 26.1% input: Video Aug 2025 Official leaderboard
59 PixelReasoner-RL-v1 7B 26% input: Video Official leaderboard
60 VideoChat-R1-7B 7B 25.9% input: Video Official leaderboard
61 Qwen2.5-VL-7B 25.7% input: Video, instruct Jan 2025 Official leaderboard
62 InternVL2_5-78B 25.4% input: Video Dec 2024 Official leaderboard
63 Ovis2-16B 25% input: Frames Feb 2025 Official leaderboard
64 InternVL3-8B 24.4% input: Video Apr 2025 Official leaderboard
65 Kimi-VL-A3B-Thinking 23.9% input: Frames, thinking Apr 2025 Official leaderboard
66 InternVL2.5-38B 23.9% input: Video Dec 2024 Official leaderboard
67 Qwen2.5-VL-7B-VRAG 7B 23.4% input: Video Official leaderboard
68 MiniCPM-o 4.5 23.3% input: V+A Feb 2026 Official leaderboard
69 Phi-4-Reasoning-Vision-15B 23.1% input: Video Mar 2026 Official leaderboard
70 R1-Onevision-7B 7B 22.8% input: Video Official leaderboard
71 Qwen3.5-4B 22.6% input: Video Feb 2026 Official leaderboard
72 AURA-32B 32B 22.3% input: Video Official leaderboard
73 InternVL3.5-4B-I 4B 22.2% input: Video, instruct Official leaderboard
74 Keye-VL-1.5-8B 22.1% input: Video Aug 2025 Official leaderboard
75 Audio Flamingo 3 22% input: Audio Jul 2025 Official leaderboard
76 Gemma 4 E2B 21.8% input: V+A Mar 2026 Official leaderboard
77 NVIDIA Nemotron Nano 12B VL 12B 21.7% input: Video Official leaderboard
78 HumanSense Omni-R 7B 21.6% input: V+A Official leaderboard
79 EchoInk-R1-7B 7B 21.5% input: V+A Official leaderboard
80 MiniCPM-V 4.5 21.3% input: Video Aug 2025 Official leaderboard
81 Qwen2.5-Omni-7B 21.3% input: V+A Mar 2025 Official leaderboard
82 SpaceThinker-3B 3B 20.6% input: Video Official leaderboard
83 Qwen3-VL-2B-Instruct 20.4% input: Video, instruct Oct 2025 Official leaderboard
84 Ovis2-8B 19.9% input: Frames Feb 2025 Official leaderboard
85 Qwen2.5-VL-3B 19.9% input: Video, instruct Jan 2025 Official leaderboard
86 MiniCPM-o-2_6 19.4% input: V+A Jan 2025 Official leaderboard
87 Qwen3.5-2B 19% input: Video Feb 2026 Official leaderboard
88 Baichuan-Omni-1.5 19% input: V+A Jan 2025 Official leaderboard
89 Qwen2.5-Omni-3B 3B 18.1% input: V+A Official leaderboard
90 SALMONN2+ 7B 7B 17.7% input: V+A Official leaderboard
91 Ovis2-4B 17.7% input: Frames Feb 2025 Official leaderboard
92 Qwen3.5-0.8B 16.9% input: Video Feb 2026 Official leaderboard
93 OmniAtlas-Qwen2.5-7B 7B 16% input: V+A Official leaderboard
94 VideoLLAMA3-7B 15.9% input: Video Jan 2024 Official leaderboard
95 Kimi-VL-A3B-Instruct 15.8% input: Frames, instruct Apr 2025 Official leaderboard
96 OmniVinci-9B 9B 15.5% input: V+A Official leaderboard
97 Monet-7B 7B 13.6% input: Video Official leaderboard
98 LLaVA-OneVision 7B 13.2% input: Video Aug 2024 Official leaderboard
99 Qwen2-Audio-7B-I 7B 13.1% input: Audio, instruct Official leaderboard
100 VideoMind 11.8% input: V+A, VideoMind Official leaderboard
101 Tempo-6B 6B 9.70% input: Video Official leaderboard
102 SenseNova-SI-1.3-8B 8B 9.47% input: Video Official leaderboard
103 Video-RAG 8.17% input: V+A, Video-RAG Official leaderboard
104 Vgent 7.99% input: V+A, Vgent Official leaderboard
105 OneThinker-8B 8B 7.26% input: Video Official leaderboard
106 VideoScore2 7B 3.17% input: Video Official leaderboard
#ModelScoreSettingDateSource
1 Qwen3-Omni-30B-A3B-Thinking 64.2% input: V+A, thinking Sep 2025 Official leaderboard
2 Gemini 3.1 Pro 57% input: V+A Feb 2026 Official leaderboard
3 Voxtral Small 48.6% input: Audio Jul 2025 Official leaderboard
4 Qwen3-VL-235B-A22B 47.6% input: Video, thinking Sep 2025 Official leaderboard
5 Qwen3-VL-32B-Thinking 46.7% input: Video, thinking Oct 2025 Official leaderboard
6 Kimi K2.6 45.1% input: Video Apr 2026 Official leaderboard
7 Qwen3-VL-30B-A3B-Thinking 43.7% input: Video, thinking Oct 2025 Official leaderboard
8 Qwen3-VL-235B-A22B-Instruct 42.6% input: Video, instruct Sep 2025 Official leaderboard
9 Qwen3-VL-8B-Thinking 42.2% input: Video, thinking Oct 2025 Official leaderboard
10 Gemma 3 27B 42% input: Frames Mar 2025 Official leaderboard
11 Qwen3-VL-32B-Instruct 40.6% input: Video, instruct Oct 2025 Official leaderboard
12 Gemma 3 12B 40.1% input: Frames Mar 2025 Official leaderboard
13 Qwen3-Omni-30B-A3B 38.9% input: V+A, instruct Sep 2025 Official leaderboard
14 Gemma 4 31B IT 38.9% input: Frames Apr 2026 Official leaderboard
15 STEP3-VL-10B 38.7% input: Video Jan 2026 Official leaderboard
16 GLM-4.5V 37.8% input: Video Aug 2025 Official leaderboard
17 Qwen3-VL-4B-Thinking 37.7% input: Video, thinking Oct 2025 Official leaderboard
18 MiMo-VL-7B-RL-2508 36.9% input: Video Aug 2025 Official leaderboard
19 Qwen2.5-VL-32B 36.1% input: Video, instruct Mar 2025 Official leaderboard
20 Gemma 3 4B 35.8% input: Frames Mar 2025 Official leaderboard
21 Gemma 4 26B A4B 34.8% input: Frames Apr 2026 Official leaderboard
22 Qwen3-VL-8B-Instruct 34.5% input: Video, instruct Oct 2025 Official leaderboard
23 GLM-4.6V-Flash 33.8% input: Video Dec 2025 Official leaderboard
24 Qwen3.6 27B 32.8% input: Video Apr 2026 Official leaderboard
25 Qwen2.5-VL-72B 32.3% input: Video, instruct Jan 2025 Official leaderboard
26 Qwen 3.6 35B-A3B 31.9% input: Video Apr 2026 Official leaderboard
27 Qwen3.5-27B 31.8% input: Video Feb 2026 Official leaderboard
28 Qwen3.5-35B-A3B 31.1% input: Video Feb 2026 Official leaderboard
29 Qwen3-VL-30B-A3B-Instruct 30.8% input: Video, instruct Oct 2025 Official leaderboard
30 Qwen3.5-122B-A10B 30.4% input: Video Feb 2026 Official leaderboard
31 InternVL3.5-241B-A28B 29.9% input: Video Aug 2025 Official leaderboard
32 InternVL3-38B 29.6% input: Video Apr 2025 Official leaderboard
33 GLM-4.1V-9B-Thinking 29.4% input: Video, thinking Jul 2025 Official leaderboard
34 InternVL3-78B 29.1% input: Video Apr 2025 Official leaderboard
35 Qwen3-VL-4B-Instruct 28.4% input: Video, instruct Oct 2025 Official leaderboard
36 Ovis2.5-9B 27.7% input: Video Aug 2025 Official leaderboard
37 Molmo2-8B 27.6% input: Video Dec 2025 Official leaderboard
38 InternVL3-14B 27.4% input: Video Apr 2025 Official leaderboard
39 Qwen3.5-9B 27.3% input: Video Feb 2026 Official leaderboard
40 Gemma 4 E4B 26.9% input: V+A Mar 2026 Official leaderboard
41 InternVL3.5-38B 26.1% input: Video Aug 2025 Official leaderboard
42 InternVL3.5-8B 26.1% input: Video Aug 2025 Official leaderboard
43 Qwen2.5-VL-7B 25.7% input: Video, instruct Jan 2025 Official leaderboard
44 InternVL2_5-78B 25.4% input: Video Dec 2024 Official leaderboard
45 Ovis2-16B 25% input: Frames Feb 2025 Official leaderboard
46 InternVL3-8B 24.4% input: Video Apr 2025 Official leaderboard
47 Kimi-VL-A3B-Thinking 23.9% input: Frames, thinking Apr 2025 Official leaderboard
48 InternVL2.5-38B 23.9% input: Video Dec 2024 Official leaderboard
49 MiniCPM-o 4.5 23.3% input: V+A Feb 2026 Official leaderboard
50 Phi-4-Reasoning-Vision-15B 23.1% input: Video Mar 2026 Official leaderboard
51 Qwen3.5-4B 22.6% input: Video Feb 2026 Official leaderboard
52 Keye-VL-1.5-8B 22.1% input: Video Aug 2025 Official leaderboard
53 Audio Flamingo 3 22% input: Audio Jul 2025 Official leaderboard
54 Gemma 4 E2B 21.8% input: V+A Mar 2026 Official leaderboard
55 MiniCPM-V 4.5 21.3% input: Video Aug 2025 Official leaderboard
56 Qwen2.5-Omni-7B 21.3% input: V+A Mar 2025 Official leaderboard
57 Qwen3-VL-2B-Instruct 20.4% input: Video, instruct Oct 2025 Official leaderboard
58 Ovis2-8B 19.9% input: Frames Feb 2025 Official leaderboard
59 Qwen2.5-VL-3B 19.9% input: Video, instruct Jan 2025 Official leaderboard
60 MiniCPM-o-2_6 19.4% input: V+A Jan 2025 Official leaderboard
61 Qwen3.5-2B 19% input: Video Feb 2026 Official leaderboard
62 Baichuan-Omni-1.5 19% input: V+A Jan 2025 Official leaderboard
63 Ovis2-4B 17.7% input: Frames Feb 2025 Official leaderboard
64 Qwen3.5-0.8B 16.9% input: Video Feb 2026 Official leaderboard
65 VideoLLAMA3-7B 15.9% input: Video Jan 2024 Official leaderboard
66 Kimi-VL-A3B-Instruct 15.8% input: Frames, instruct Apr 2025 Official leaderboard
67 LLaVA-OneVision 7B 13.2% input: Video Aug 2024 Official leaderboard

Rankings by part

We read 10 parts from the benchmark’s own leaderboard in 3 groups: modality, length and category. Each ranks models on one part of the test. Qwen3-Omni-30B-A3B-Thinking leads every one of them. Arrows show how far a model moved from its overall rank among the models we track.

Visual

67 models we track have a score here.

  1. 1 Qwen3-Omni-30B-A3B-Thinking 62.5%
  2. 2 Gemini 3.1 Pro 53.2%
  3. 3 Voxtral Small 47.2%
  4. 4 Qwen3-VL-235B-A22B 46.2%
  5. 5 Qwen3-VL-32B-Thinking 44.9%
  6. 6 Kimi K2.6 43.1%
  7. 7 Qwen3-VL-30B-A3B-Thinking 41.8%
  8. 8 Qwen3-VL-235B-A22B-Instruct 40.5%
  9. 9 Gemma 3 27B ↑1 40.3%
  10. 10 Qwen3-VL-8B-Thinking ↓1 39.9%

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Qwen
Own leaderboard Not tested since Apr 2026

Source: Official LongShOTBench leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen3-Omni-30B-A3B-Thinking Voxtral Small Gemma 3 27B Qwen2.5-VL-72B InternVL2_5-78B VideoLLAMA3-7B 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen3-Omni-30B-A3B-Thinking Voxtral Small

Source: Official LongShOTBench leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Audio and video

All 695 in Audio and video
  • Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy. Qwen 3.8 Max 88.7%
  • LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization. Gemini 4 Argon 91.7%
  • MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy. Qwen 3.8 Max 82.4%
  • MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. Qwen 3.8 Max 90.8%
  • FLEURS In use Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better. Phi-4-multimodal 4.00%
  • VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. Qwen3.5-Omni-Plus 93.1%

Data audit

9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 7 months ago, in April 2026, for Qwen3.6 27B. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (106 entries).
  • Pass Models scored 67 models we track, plus 39 other entries on its leaderboard.
  • Warning Newest score The newest came 7 months ago, in April 2026, for Qwen3.6 27B.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40).
  • Pass Headroom The best is 64.2%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on longshot.cvmbzuai.com: 67 rows for models we track and 39 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check