LongShOTBench
Models answer single-turn and multi-turn questions about long videos using visual, spoken, and ambient audio evidence. Weighted rubrics award partial credit.
The suite contains 3,401 questions over 274 videos totaling 188 hours. Report the modality and question track with each score.
- Made by
- MBZUAI
- Released
- December 2025
- Kind
- Long videos
- Level
- Everyday
- Scored
- Graded by a model
- Answers
- Open-ended
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Multi-turn
- Refresh
- One fixed set
- Run by
- Anyone
- In views
- Vision, Audio and speech
- Strongest model tested
-
Gemini 3.1 Pro #40 on the capability index
- Last model added
- April 2026
- Scores refreshed
- November 20th, 2026
In the authors’ words
Long-form omni-modal video understanding requires integrating vision, speech, and ambient audio with coherent long-context reasoning. Existing video benchmarks often trade off temporal scale, modality coverage, open-ended interaction, and interpretable scoring. To address this gap, we introduce LongShOTBench, a long video understanding benchmark designed around three coupled goals: holistic omni-modal integration, intent-driven open-ended interaction, and rubric-level diagnosis. It builds single- and multi-turn questions from real viewing scenarios, with systematic tasks probing visual, speech, ambient-audio, temporal, and cross-modal reasoning. Each item includes a reference answer and a weighted criterion-level rubric, letting evaluation identify which perceptual facts, temporal links, modality-grounding requirements, and reasoning steps are satisfied or missed. All samples are manually verified to improve grounding, clarity, and rubric reliability. We also introduce LongShOTAgent, a training-free omni-modal evidence-seeking agent coupling full-video preprocessing with targeted retrieval, query-adaptive segment refinement, and explicit claim verification over visual, speech, and non-speech audio evidence. Its iterative search-refine-verify loop exposes intermediate evidence and lets modality-specific specialists re-analyze relevant moments before answering. We evaluate 105 video-capable models spanning open-source omni-modal models, vision-language systems, audio LLMs, agentic pipelines and closed-source APIs. Current MLLMs remain far from saturating LongShOTBench, while our LongShOTAgent is the strongest training-free system, reaching 66.64% overall. By releasing the benchmark, leaderboard, and method, we provide a shared, interpretable testbed for advancing long-form omni-modal video reasoning. Code, data, and the leaderboard are available at https://longshot.cvmbzuai.com/.
LongShOTAgent tops the leaderboard with 66.7% Qwen3-Omni-30B-A3B-Thinking leads with 64.2% among models released by April 2026
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Qwen3-Omni-30B-A3B-Thinking, at 64.2%. This view shows only models we track, each at its best.
The newest model with a score here came out in April 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 66.7 LongShOTAgent
- 64.2
Qwen3-Omni-30B-A3B-Thinking
- 57.0
Gemini 3.1 Pro
- 49.6 Voxtral-Mini-3B-2507
- 48.6
Voxtral Small
- 47.6
Qwen3-VL-235B-A22B
- 46.7
Qwen3-VL-32B-Thinking
- 46.2 Vero-Qwen3I-8B
- 45.1 Kimi K2.6
- 43.7
Qwen3-VL-30B-A3B-Thinking
- 42.6
Qwen3-VL-235B-A22B-Instruct
- 42.2
Qwen3-VL-8B-Thinking
- 42.0
Gemma 3 27B
- 41.5 Intern-S1-mini
- 41.2 Nemotron-3-Nano-Omni-30B-A3B-R
- 40.6
Qwen3-VL-32B-Instruct
- 40.1
Gemma 3 12B
- 38.9
Qwen3-Omni-30B-A3B
- 38.9
Gemma 4 31B IT
- 38.7
STEP3-VL-10B
- 37.8 GLM-4.5V
- 37.7
Qwen3-VL-4B-Thinking
- 36.9 MiMo-VL-7B-RL-2508
- 36.1
Qwen2.5-VL-32B
- 35.8
Gemma 3 4B
- 34.8
Gemma 4 26B A4B
- 34.5
Qwen3-VL-8B-Instruct
- 33.8 GLM-4.6V-Flash
- 32.9 Ovis2.6-30B-A3B
- 32.8
Qwen3.6 27B
- 32.3
Qwen2.5-VL-72B
- 31.9
Qwen 3.6 35B-A3B
- 31.8
Qwen3.5-27B
- 31.1
Qwen3.5-35B-A3B
- 30.8
Qwen3-VL-30B-A3B-Instruct
- 30.5 R-4B
- 30.4 Fun-Audio-Chat-8B
- 30.4
Qwen3.5-122B-A10B
- 29.9
InternVL3.5-241B-A28B
- 29.6
InternVL3-38B
- 29.4 GLM-4.1V-9B-Thinking
- 29.1
InternVL3-78B
- 28.4
Qwen3-VL-4B-Instruct
- 28.0 VideoRFT-7B
- 28.0 ERNIE-4.5-VL-28B-A3B
- 27.8 LongVT-RL
- 27.8 DeepEyes-7B
- 27.7
Ovis2.5-9B
- 27.7 Video-R2
- 27.6
Molmo2-8B
- 27.4
InternVL3-14B
- 27.3
Qwen3.5-9B
- 27.1 RePlan-Qwen2.5-VL-7B
- 26.9
Gemma 4 E4B
- 26.9 Video-R1-7B
- 26.9 TimeSearch-R
- 26.1
InternVL3.5-38B
- 26.1
InternVL3.5-8B
- 26.0 PixelReasoner-RL-v1
- 25.9 VideoChat-R1-7B
- 25.7
Qwen2.5-VL-7B
- 25.4
InternVL2_5-78B
- 25.0
Ovis2-16B
- 24.4
InternVL3-8B
- 23.9 Kimi-VL-A3B-Thinking
- 23.9
InternVL2.5-38B
- 23.4 Qwen2.5-VL-7B-VRAG
- 23.3
MiniCPM-o 4.5
- 23.1
Phi-4-Reasoning-Vision-15B
- 22.8 R1-Onevision-7B
- 22.6
Qwen3.5-4B
- 22.3 AURA-32B
- 22.2 InternVL3.5-4B-I
- 22.1
Keye-VL-1.5-8B
- 22.0
Audio Flamingo 3
- 21.8
Gemma 4 E2B
- 21.7 NVIDIA Nemotron Nano 12B VL
- 21.6 HumanSense Omni-R
- 21.5 EchoInk-R1-7B
- 21.3
MiniCPM-V 4.5
- 21.3
Qwen2.5-Omni-7B
- 20.6 SpaceThinker-3B
- 20.4
Qwen3-VL-2B-Instruct
- 19.9
Ovis2-8B
- 19.9
Qwen2.5-VL-3B
- 19.4 MiniCPM-o-2_6
- 19.0
Qwen3.5-2B
- 19.0
Baichuan-Omni-1.5
- 18.1 Qwen2.5-Omni-3B
- 17.7 SALMONN2+ 7B
- 17.7
Ovis2-4B
- 16.9
Qwen3.5-0.8B
- 16.0 OmniAtlas-Qwen2.5-7B
- 15.9 VideoLLAMA3-7B
- 15.8 Kimi-VL-A3B-Instruct
- 15.5 OmniVinci-9B
- 13.6 Monet-7B
- 13.2
LLaVA-OneVision 7B
- 13.1 Qwen2-Audio-7B-I
- 11.8 VideoMind
- 9.70 Tempo-6B
- 9.47 SenseNova-SI-1.3-8B
- 8.17 Video-RAG
- 7.99 Vgent
- 7.26 OneThinker-8B
- 3.17 VideoScore2
- 1 LongShOTAgent input: V+A 66.7%
- 2
Qwen3-Omni-30B-A3B-Thinking input: V+A, thinking 64.2%
- 3
Gemini 3.1 Pro input: V+A 57%
- 4 Voxtral-Mini-3B-2507 input: Audio 49.6%
- 5
Voxtral Small input: Audio 48.6%
- 6
Qwen3-VL-235B-A22B input: Video, thinking 47.6%
- 7
Qwen3-VL-32B-Thinking input: Video, thinking 46.7%
- 8 Vero-Qwen3I-8B input: Video 46.2%
- 9 Kimi K2.6 input: Video 45.1%
- 10
Qwen3-VL-30B-A3B-Thinking input: Video, thinking 43.7%
- 11
Qwen3-VL-235B-A22B-Instruct input: Video, instruct 42.6%
- 12
Qwen3-VL-8B-Thinking input: Video, thinking 42.2%
- 13
Gemma 3 27B input: Frames 42%
- 14 Intern-S1-mini input: Video 41.5%
- 15 Nemotron-3-Nano-Omni-30B-A3B-R input: V+A 41.2%
- 16
Qwen3-VL-32B-Instruct input: Video, instruct 40.6%
- 17
Gemma 3 12B input: Frames 40.1%
- 18
Qwen3-Omni-30B-A3B input: V+A, instruct 38.9%
- 19
Gemma 4 31B IT input: Frames 38.9%
- 20
STEP3-VL-10B input: Video 38.7%
- 21 GLM-4.5V input: Video 37.8%
- 22
Qwen3-VL-4B-Thinking input: Video, thinking 37.7%
- 23 MiMo-VL-7B-RL-2508 input: Video 36.9%
- 24
Qwen2.5-VL-32B input: Video, instruct 36.1%
- 25
Gemma 3 4B input: Frames 35.8%
- 26
Gemma 4 26B A4B input: Frames 34.8%
- 27
Qwen3-VL-8B-Instruct input: Video, instruct 34.5%
- 28 GLM-4.6V-Flash input: Video 33.8%
- 29 Ovis2.6-30B-A3B input: Video 32.9%
- 30
Qwen3.6 27B input: Video 32.8%
- 31
Qwen2.5-VL-72B input: Video, instruct 32.3%
- 32
Qwen 3.6 35B-A3B input: Video 31.9%
- 33
Qwen3.5-27B input: Video 31.8%
- 34
Qwen3.5-35B-A3B input: Video 31.1%
- 35
Qwen3-VL-30B-A3B-Instruct input: Video, instruct 30.8%
- 36 R-4B input: Video 30.5%
- 37 Fun-Audio-Chat-8B input: Audio 30.4%
- 38
Qwen3.5-122B-A10B input: Video 30.4%
- 39
InternVL3.5-241B-A28B input: Video 29.9%
- 40
InternVL3-38B input: Video 29.6%
- 41 GLM-4.1V-9B-Thinking input: Video, thinking 29.4%
- 42
InternVL3-78B input: Video 29.1%
- 43
Qwen3-VL-4B-Instruct input: Video, instruct 28.4%
- 44 VideoRFT-7B input: Video 28%
- 45 ERNIE-4.5-VL-28B-A3B input: Video 28%
- 46 LongVT-RL input: Video 27.8%
- 47 DeepEyes-7B input: Video 27.8%
- 48
Ovis2.5-9B input: Video 27.7%
- 49 Video-R2 input: Video 27.7%
- 50
Molmo2-8B input: Video 27.6%
- 51
InternVL3-14B input: Video 27.4%
- 52
Qwen3.5-9B input: Video 27.3%
- 53 RePlan-Qwen2.5-VL-7B input: Video 27.1%
- 54
Gemma 4 E4B input: V+A 26.9%
- 55 Video-R1-7B input: Video 26.9%
- 56 TimeSearch-R input: Video 26.9%
- 57
InternVL3.5-38B input: Video 26.1%
- 58
InternVL3.5-8B input: Video 26.1%
- 59 PixelReasoner-RL-v1 input: Video 26%
- 60 VideoChat-R1-7B input: Video 25.9%
- 61
Qwen2.5-VL-7B input: Video, instruct 25.7%
- 62
InternVL2_5-78B input: Video 25.4%
- 63
Ovis2-16B input: Frames 25%
- 64
InternVL3-8B input: Video 24.4%
- 65 Kimi-VL-A3B-Thinking input: Frames, thinking 23.9%
- 66
InternVL2.5-38B input: Video 23.9%
- 67 Qwen2.5-VL-7B-VRAG input: Video 23.4%
- 68
MiniCPM-o 4.5 input: V+A 23.3%
- 69
Phi-4-Reasoning-Vision-15B input: Video 23.1%
- 70 R1-Onevision-7B input: Video 22.8%
- 71
Qwen3.5-4B input: Video 22.6%
- 72 AURA-32B input: Video 22.3%
- 73 InternVL3.5-4B-I input: Video, instruct 22.2%
- 74
Keye-VL-1.5-8B input: Video 22.1%
- 75
Audio Flamingo 3 input: Audio 22%
- 76
Gemma 4 E2B input: V+A 21.8%
- 77 NVIDIA Nemotron Nano 12B VL input: Video 21.7%
- 78 HumanSense Omni-R input: V+A 21.6%
- 79 EchoInk-R1-7B input: V+A 21.5%
- 80
MiniCPM-V 4.5 input: Video 21.3%
- 81
Qwen2.5-Omni-7B input: V+A 21.3%
- 82 SpaceThinker-3B input: Video 20.6%
- 83
Qwen3-VL-2B-Instruct input: Video, instruct 20.4%
- 84
Ovis2-8B input: Frames 19.9%
- 85
Qwen2.5-VL-3B input: Video, instruct 19.9%
- 86 MiniCPM-o-2_6 input: V+A 19.4%
- 87
Qwen3.5-2B input: Video 19%
- 88
Baichuan-Omni-1.5 input: V+A 19%
- 89 Qwen2.5-Omni-3B input: V+A 18.1%
- 90 SALMONN2+ 7B input: V+A 17.7%
- 91
Ovis2-4B input: Frames 17.7%
- 92
Qwen3.5-0.8B input: Video 16.9%
- 93 OmniAtlas-Qwen2.5-7B input: V+A 16%
- 94 VideoLLAMA3-7B input: Video 15.9%
- 95 Kimi-VL-A3B-Instruct input: Frames, instruct 15.8%
- 96 OmniVinci-9B input: V+A 15.5%
- 97 Monet-7B input: Video 13.6%
- 98
LLaVA-OneVision 7B input: Video 13.2%
- 99 Qwen2-Audio-7B-I input: Audio, instruct 13.1%
- 100 VideoMind input: V+A 11.8%
- 101 Tempo-6B input: Video 9.70%
- 102 SenseNova-SI-1.3-8B input: Video 9.47%
- 103 Video-RAG input: V+A 8.17%
- 104 Vgent input: V+A 7.99%
- 105 OneThinker-8B input: Video 7.26%
- 106 VideoScore2 input: Video 3.17%
Source: Official LongShOTBench leaderboard
- 64.2
Qwen3-Omni-30B-A3B-Thinking
- 57.0
Gemini 3.1 Pro
- 48.6
Voxtral Small
- 47.6
Qwen3-VL-235B-A22B
- 46.7
Qwen3-VL-32B-Thinking
- 45.1 Kimi K2.6
- 43.7
Qwen3-VL-30B-A3B-Thinking
- 42.6
Qwen3-VL-235B-A22B-Instruct
- 42.2
Qwen3-VL-8B-Thinking
- 42.0
Gemma 3 27B
- 40.6
Qwen3-VL-32B-Instruct
- 40.1
Gemma 3 12B
- 38.9
Qwen3-Omni-30B-A3B
- 38.9
Gemma 4 31B IT
- 38.7
STEP3-VL-10B
- 37.8 GLM-4.5V
- 37.7
Qwen3-VL-4B-Thinking
- 36.9 MiMo-VL-7B-RL-2508
- 36.1
Qwen2.5-VL-32B
- 35.8
Gemma 3 4B
- 34.8
Gemma 4 26B A4B
- 34.5
Qwen3-VL-8B-Instruct
- 33.8 GLM-4.6V-Flash
- 32.8
Qwen3.6 27B
- 32.3
Qwen2.5-VL-72B
- 31.9
Qwen 3.6 35B-A3B
- 31.8
Qwen3.5-27B
- 31.1
Qwen3.5-35B-A3B
- 30.8
Qwen3-VL-30B-A3B-Instruct
- 30.4
Qwen3.5-122B-A10B
- 29.9
InternVL3.5-241B-A28B
- 29.6
InternVL3-38B
- 29.4 GLM-4.1V-9B-Thinking
- 29.1
InternVL3-78B
- 28.4
Qwen3-VL-4B-Instruct
- 27.7
Ovis2.5-9B
- 27.6
Molmo2-8B
- 27.4
InternVL3-14B
- 27.3
Qwen3.5-9B
- 26.9
Gemma 4 E4B
- 26.1
InternVL3.5-38B
- 26.1
InternVL3.5-8B
- 25.7
Qwen2.5-VL-7B
- 25.4
InternVL2_5-78B
- 25.0
Ovis2-16B
- 24.4
InternVL3-8B
- 23.9 Kimi-VL-A3B-Thinking
- 23.9
InternVL2.5-38B
- 23.3
MiniCPM-o 4.5
- 23.1
Phi-4-Reasoning-Vision-15B
- 22.6
Qwen3.5-4B
- 22.1
Keye-VL-1.5-8B
- 22.0
Audio Flamingo 3
- 21.8
Gemma 4 E2B
- 21.3
MiniCPM-V 4.5
- 21.3
Qwen2.5-Omni-7B
- 20.4
Qwen3-VL-2B-Instruct
- 19.9
Ovis2-8B
- 19.9
Qwen2.5-VL-3B
- 19.4 MiniCPM-o-2_6
- 19.0
Qwen3.5-2B
- 19.0
Baichuan-Omni-1.5
- 17.7
Ovis2-4B
- 16.9
Qwen3.5-0.8B
- 15.9 VideoLLAMA3-7B
- 15.8 Kimi-VL-A3B-Instruct
- 13.2
LLaVA-OneVision 7B
- 1
Qwen3-Omni-30B-A3B-Thinking input: V+A, thinking 64.2%
- 2
Gemini 3.1 Pro input: V+A 57%
- 3
Voxtral Small input: Audio 48.6%
- 4
Qwen3-VL-235B-A22B input: Video, thinking 47.6%
- 5
Qwen3-VL-32B-Thinking input: Video, thinking 46.7%
- 6 Kimi K2.6 input: Video 45.1%
- 7
Qwen3-VL-30B-A3B-Thinking input: Video, thinking 43.7%
- 8
Qwen3-VL-235B-A22B-Instruct input: Video, instruct 42.6%
- 9
Qwen3-VL-8B-Thinking input: Video, thinking 42.2%
- 10
Gemma 3 27B input: Frames 42%
- 11
Qwen3-VL-32B-Instruct input: Video, instruct 40.6%
- 12
Gemma 3 12B input: Frames 40.1%
- 13
Qwen3-Omni-30B-A3B input: V+A, instruct 38.9%
- 14
Gemma 4 31B IT input: Frames 38.9%
- 15
STEP3-VL-10B input: Video 38.7%
- 16 GLM-4.5V input: Video 37.8%
- 17
Qwen3-VL-4B-Thinking input: Video, thinking 37.7%
- 18 MiMo-VL-7B-RL-2508 input: Video 36.9%
- 19
Qwen2.5-VL-32B input: Video, instruct 36.1%
- 20
Gemma 3 4B input: Frames 35.8%
- 21
Gemma 4 26B A4B input: Frames 34.8%
- 22
Qwen3-VL-8B-Instruct input: Video, instruct 34.5%
- 23 GLM-4.6V-Flash input: Video 33.8%
- 24
Qwen3.6 27B input: Video 32.8%
- 25
Qwen2.5-VL-72B input: Video, instruct 32.3%
- 26
Qwen 3.6 35B-A3B input: Video 31.9%
- 27
Qwen3.5-27B input: Video 31.8%
- 28
Qwen3.5-35B-A3B input: Video 31.1%
- 29
Qwen3-VL-30B-A3B-Instruct input: Video, instruct 30.8%
- 30
Qwen3.5-122B-A10B input: Video 30.4%
- 31
InternVL3.5-241B-A28B input: Video 29.9%
- 32
InternVL3-38B input: Video 29.6%
- 33 GLM-4.1V-9B-Thinking input: Video, thinking 29.4%
- 34
InternVL3-78B input: Video 29.1%
- 35
Qwen3-VL-4B-Instruct input: Video, instruct 28.4%
- 36
Ovis2.5-9B input: Video 27.7%
- 37
Molmo2-8B input: Video 27.6%
- 38
InternVL3-14B input: Video 27.4%
- 39
Qwen3.5-9B input: Video 27.3%
- 40
Gemma 4 E4B input: V+A 26.9%
- 41
InternVL3.5-38B input: Video 26.1%
- 42
InternVL3.5-8B input: Video 26.1%
- 43
Qwen2.5-VL-7B input: Video, instruct 25.7%
- 44
InternVL2_5-78B input: Video 25.4%
- 45
Ovis2-16B input: Frames 25%
- 46
InternVL3-8B input: Video 24.4%
- 47 Kimi-VL-A3B-Thinking input: Frames, thinking 23.9%
- 48
InternVL2.5-38B input: Video 23.9%
- 49
MiniCPM-o 4.5 input: V+A 23.3%
- 50
Phi-4-Reasoning-Vision-15B input: Video 23.1%
- 51
Qwen3.5-4B input: Video 22.6%
- 52
Keye-VL-1.5-8B input: Video 22.1%
- 53
Audio Flamingo 3 input: Audio 22%
- 54
Gemma 4 E2B input: V+A 21.8%
- 55
MiniCPM-V 4.5 input: Video 21.3%
- 56
Qwen2.5-Omni-7B input: V+A 21.3%
- 57
Qwen3-VL-2B-Instruct input: Video, instruct 20.4%
- 58
Ovis2-8B input: Frames 19.9%
- 59
Qwen2.5-VL-3B input: Video, instruct 19.9%
- 60 MiniCPM-o-2_6 input: V+A 19.4%
- 61
Qwen3.5-2B input: Video 19%
- 62
Baichuan-Omni-1.5 input: V+A 19%
- 63
Ovis2-4B input: Frames 17.7%
- 64
Qwen3.5-0.8B input: Video 16.9%
- 65 VideoLLAMA3-7B input: Video 15.9%
- 66 Kimi-VL-A3B-Instruct input: Frames, instruct 15.8%
- 67
LLaVA-OneVision 7B input: Video 13.2%
Source: Official LongShOTBench leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | LongShOTAgent | 66.7% | input: V+A, LongShOTAgent | Official leaderboard | |
| 2 | | 64.2% | input: V+A, thinking | Sep 2025 | Official leaderboard |
| 3 | | 57% | input: V+A | Feb 2026 | Official leaderboard |
| 4 | Voxtral-Mini-3B-2507 3B | 49.6% | input: Audio | Official leaderboard | |
| 5 | | 48.6% | input: Audio | Jul 2025 | Official leaderboard |
| 6 | | 47.6% | input: Video, thinking | Sep 2025 | Official leaderboard |
| 7 | | 46.7% | input: Video, thinking | Oct 2025 | Official leaderboard |
| 8 | Vero-Qwen3I-8B 8B | 46.2% | input: Video | Official leaderboard | |
| 9 | Kimi K2.6 | 45.1% | input: Video | Apr 2026 | Official leaderboard |
| 10 | | 43.7% | input: Video, thinking | Oct 2025 | Official leaderboard |
| 11 | | 42.6% | input: Video, instruct | Sep 2025 | Official leaderboard |
| 12 | | 42.2% | input: Video, thinking | Oct 2025 | Official leaderboard |
| 13 | | 42% | input: Frames | Mar 2025 | Official leaderboard |
| 14 | Intern-S1-mini 14B | 41.5% | input: Video | Official leaderboard | |
| 15 | Nemotron-3-Nano-Omni-30B-A3B-R 30B | 41.2% | input: V+A | Official leaderboard | |
| 16 | | 40.6% | input: Video, instruct | Oct 2025 | Official leaderboard |
| 17 | | 40.1% | input: Frames | Mar 2025 | Official leaderboard |
| 18 | | 38.9% | input: V+A, instruct | Sep 2025 | Official leaderboard |
| 19 | | 38.9% | input: Frames | Apr 2026 | Official leaderboard |
| 20 | | 38.7% | input: Video | Jan 2026 | Official leaderboard |
| 21 | GLM-4.5V | 37.8% | input: Video | Aug 2025 | Official leaderboard |
| 22 | | 37.7% | input: Video, thinking | Oct 2025 | Official leaderboard |
| 23 | MiMo-VL-7B-RL-2508 | 36.9% | input: Video | Aug 2025 | Official leaderboard |
| 24 | | 36.1% | input: Video, instruct | Mar 2025 | Official leaderboard |
| 25 | | 35.8% | input: Frames | Mar 2025 | Official leaderboard |
| 26 | | 34.8% | input: Frames | Apr 2026 | Official leaderboard |
| 27 | | 34.5% | input: Video, instruct | Oct 2025 | Official leaderboard |
| 28 | GLM-4.6V-Flash | 33.8% | input: Video | Dec 2025 | Official leaderboard |
| 29 | Ovis2.6-30B-A3B 30B | 32.9% | input: Video | Official leaderboard | |
| 30 | | 32.8% | input: Video | Apr 2026 | Official leaderboard |
| 31 | | 32.3% | input: Video, instruct | Jan 2025 | Official leaderboard |
| 32 | | 31.9% | input: Video | Apr 2026 | Official leaderboard |
| 33 | | 31.8% | input: Video | Feb 2026 | Official leaderboard |
| 34 | | 31.1% | input: Video | Feb 2026 | Official leaderboard |
| 35 | | 30.8% | input: Video, instruct | Oct 2025 | Official leaderboard |
| 36 | R-4B 4B | 30.5% | input: Video | Official leaderboard | |
| 37 | Fun-Audio-Chat-8B 8B | 30.4% | input: Audio | Official leaderboard | |
| 38 | | 30.4% | input: Video | Feb 2026 | Official leaderboard |
| 39 | | 29.9% | input: Video | Aug 2025 | Official leaderboard |
| 40 | | 29.6% | input: Video | Apr 2025 | Official leaderboard |
| 41 | GLM-4.1V-9B-Thinking | 29.4% | input: Video, thinking | Jul 2025 | Official leaderboard |
| 42 | | 29.1% | input: Video | Apr 2025 | Official leaderboard |
| 43 | | 28.4% | input: Video, instruct | Oct 2025 | Official leaderboard |
| 44 | VideoRFT-7B 7B | 28% | input: Video | Official leaderboard | |
| 45 | ERNIE-4.5-VL-28B-A3B 28B | 28% | input: Video | Official leaderboard | |
| 46 | LongVT-RL 7B | 27.8% | input: Video | Official leaderboard | |
| 47 | DeepEyes-7B 7B | 27.8% | input: Video | Official leaderboard | |
| 48 | | 27.7% | input: Video | Aug 2025 | Official leaderboard |
| 49 | Video-R2 7B | 27.7% | input: Video | Official leaderboard | |
| 50 | | 27.6% | input: Video | Dec 2025 | Official leaderboard |
| 51 | | 27.4% | input: Video | Apr 2025 | Official leaderboard |
| 52 | | 27.3% | input: Video | Feb 2026 | Official leaderboard |
| 53 | RePlan-Qwen2.5-VL-7B 7B | 27.1% | input: Video | Official leaderboard | |
| 54 | | 26.9% | input: V+A | Mar 2026 | Official leaderboard |
| 55 | Video-R1-7B 7B | 26.9% | input: Video | Official leaderboard | |
| 56 | TimeSearch-R 7B | 26.9% | input: Video | Official leaderboard | |
| 57 | | 26.1% | input: Video | Aug 2025 | Official leaderboard |
| 58 | | 26.1% | input: Video | Aug 2025 | Official leaderboard |
| 59 | PixelReasoner-RL-v1 7B | 26% | input: Video | Official leaderboard | |
| 60 | VideoChat-R1-7B 7B | 25.9% | input: Video | Official leaderboard | |
| 61 | | 25.7% | input: Video, instruct | Jan 2025 | Official leaderboard |
| 62 | | 25.4% | input: Video | Dec 2024 | Official leaderboard |
| 63 | | 25% | input: Frames | Feb 2025 | Official leaderboard |
| 64 | | 24.4% | input: Video | Apr 2025 | Official leaderboard |
| 65 | Kimi-VL-A3B-Thinking | 23.9% | input: Frames, thinking | Apr 2025 | Official leaderboard |
| 66 | | 23.9% | input: Video | Dec 2024 | Official leaderboard |
| 67 | Qwen2.5-VL-7B-VRAG 7B | 23.4% | input: Video | Official leaderboard | |
| 68 | | 23.3% | input: V+A | Feb 2026 | Official leaderboard |
| 69 | | 23.1% | input: Video | Mar 2026 | Official leaderboard |
| 70 | R1-Onevision-7B 7B | 22.8% | input: Video | Official leaderboard | |
| 71 | | 22.6% | input: Video | Feb 2026 | Official leaderboard |
| 72 | AURA-32B 32B | 22.3% | input: Video | Official leaderboard | |
| 73 | InternVL3.5-4B-I 4B | 22.2% | input: Video, instruct | Official leaderboard | |
| 74 | | 22.1% | input: Video | Aug 2025 | Official leaderboard |
| 75 | | 22% | input: Audio | Jul 2025 | Official leaderboard |
| 76 | | 21.8% | input: V+A | Mar 2026 | Official leaderboard |
| 77 | NVIDIA Nemotron Nano 12B VL 12B | 21.7% | input: Video | Official leaderboard | |
| 78 | HumanSense Omni-R 7B | 21.6% | input: V+A | Official leaderboard | |
| 79 | EchoInk-R1-7B 7B | 21.5% | input: V+A | Official leaderboard | |
| 80 | | 21.3% | input: Video | Aug 2025 | Official leaderboard |
| 81 | | 21.3% | input: V+A | Mar 2025 | Official leaderboard |
| 82 | SpaceThinker-3B 3B | 20.6% | input: Video | Official leaderboard | |
| 83 | | 20.4% | input: Video, instruct | Oct 2025 | Official leaderboard |
| 84 | | 19.9% | input: Frames | Feb 2025 | Official leaderboard |
| 85 | | 19.9% | input: Video, instruct | Jan 2025 | Official leaderboard |
| 86 | MiniCPM-o-2_6 | 19.4% | input: V+A | Jan 2025 | Official leaderboard |
| 87 | | 19% | input: Video | Feb 2026 | Official leaderboard |
| 88 | | 19% | input: V+A | Jan 2025 | Official leaderboard |
| 89 | Qwen2.5-Omni-3B 3B | 18.1% | input: V+A | Official leaderboard | |
| 90 | SALMONN2+ 7B 7B | 17.7% | input: V+A | Official leaderboard | |
| 91 | | 17.7% | input: Frames | Feb 2025 | Official leaderboard |
| 92 | | 16.9% | input: Video | Feb 2026 | Official leaderboard |
| 93 | OmniAtlas-Qwen2.5-7B 7B | 16% | input: V+A | Official leaderboard | |
| 94 | VideoLLAMA3-7B | 15.9% | input: Video | Jan 2024 | Official leaderboard |
| 95 | Kimi-VL-A3B-Instruct | 15.8% | input: Frames, instruct | Apr 2025 | Official leaderboard |
| 96 | OmniVinci-9B 9B | 15.5% | input: V+A | Official leaderboard | |
| 97 | Monet-7B 7B | 13.6% | input: Video | Official leaderboard | |
| 98 | | 13.2% | input: Video | Aug 2024 | Official leaderboard |
| 99 | Qwen2-Audio-7B-I 7B | 13.1% | input: Audio, instruct | Official leaderboard | |
| 100 | VideoMind | 11.8% | input: V+A, VideoMind | Official leaderboard | |
| 101 | Tempo-6B 6B | 9.70% | input: Video | Official leaderboard | |
| 102 | SenseNova-SI-1.3-8B 8B | 9.47% | input: Video | Official leaderboard | |
| 103 | Video-RAG | 8.17% | input: V+A, Video-RAG | Official leaderboard | |
| 104 | Vgent | 7.99% | input: V+A, Vgent | Official leaderboard | |
| 105 | OneThinker-8B 8B | 7.26% | input: Video | Official leaderboard | |
| 106 | VideoScore2 7B | 3.17% | input: Video | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 64.2% | input: V+A, thinking | Sep 2025 | Official leaderboard |
| 2 | | 57% | input: V+A | Feb 2026 | Official leaderboard |
| 3 | | 48.6% | input: Audio | Jul 2025 | Official leaderboard |
| 4 | | 47.6% | input: Video, thinking | Sep 2025 | Official leaderboard |
| 5 | | 46.7% | input: Video, thinking | Oct 2025 | Official leaderboard |
| 6 | Kimi K2.6 | 45.1% | input: Video | Apr 2026 | Official leaderboard |
| 7 | | 43.7% | input: Video, thinking | Oct 2025 | Official leaderboard |
| 8 | | 42.6% | input: Video, instruct | Sep 2025 | Official leaderboard |
| 9 | | 42.2% | input: Video, thinking | Oct 2025 | Official leaderboard |
| 10 | | 42% | input: Frames | Mar 2025 | Official leaderboard |
| 11 | | 40.6% | input: Video, instruct | Oct 2025 | Official leaderboard |
| 12 | | 40.1% | input: Frames | Mar 2025 | Official leaderboard |
| 13 | | 38.9% | input: V+A, instruct | Sep 2025 | Official leaderboard |
| 14 | | 38.9% | input: Frames | Apr 2026 | Official leaderboard |
| 15 | | 38.7% | input: Video | Jan 2026 | Official leaderboard |
| 16 | GLM-4.5V | 37.8% | input: Video | Aug 2025 | Official leaderboard |
| 17 | | 37.7% | input: Video, thinking | Oct 2025 | Official leaderboard |
| 18 | MiMo-VL-7B-RL-2508 | 36.9% | input: Video | Aug 2025 | Official leaderboard |
| 19 | | 36.1% | input: Video, instruct | Mar 2025 | Official leaderboard |
| 20 | | 35.8% | input: Frames | Mar 2025 | Official leaderboard |
| 21 | | 34.8% | input: Frames | Apr 2026 | Official leaderboard |
| 22 | | 34.5% | input: Video, instruct | Oct 2025 | Official leaderboard |
| 23 | GLM-4.6V-Flash | 33.8% | input: Video | Dec 2025 | Official leaderboard |
| 24 | | 32.8% | input: Video | Apr 2026 | Official leaderboard |
| 25 | | 32.3% | input: Video, instruct | Jan 2025 | Official leaderboard |
| 26 | | 31.9% | input: Video | Apr 2026 | Official leaderboard |
| 27 | | 31.8% | input: Video | Feb 2026 | Official leaderboard |
| 28 | | 31.1% | input: Video | Feb 2026 | Official leaderboard |
| 29 | | 30.8% | input: Video, instruct | Oct 2025 | Official leaderboard |
| 30 | | 30.4% | input: Video | Feb 2026 | Official leaderboard |
| 31 | | 29.9% | input: Video | Aug 2025 | Official leaderboard |
| 32 | | 29.6% | input: Video | Apr 2025 | Official leaderboard |
| 33 | GLM-4.1V-9B-Thinking | 29.4% | input: Video, thinking | Jul 2025 | Official leaderboard |
| 34 | | 29.1% | input: Video | Apr 2025 | Official leaderboard |
| 35 | | 28.4% | input: Video, instruct | Oct 2025 | Official leaderboard |
| 36 | | 27.7% | input: Video | Aug 2025 | Official leaderboard |
| 37 | | 27.6% | input: Video | Dec 2025 | Official leaderboard |
| 38 | | 27.4% | input: Video | Apr 2025 | Official leaderboard |
| 39 | | 27.3% | input: Video | Feb 2026 | Official leaderboard |
| 40 | | 26.9% | input: V+A | Mar 2026 | Official leaderboard |
| 41 | | 26.1% | input: Video | Aug 2025 | Official leaderboard |
| 42 | | 26.1% | input: Video | Aug 2025 | Official leaderboard |
| 43 | | 25.7% | input: Video, instruct | Jan 2025 | Official leaderboard |
| 44 | | 25.4% | input: Video | Dec 2024 | Official leaderboard |
| 45 | | 25% | input: Frames | Feb 2025 | Official leaderboard |
| 46 | | 24.4% | input: Video | Apr 2025 | Official leaderboard |
| 47 | Kimi-VL-A3B-Thinking | 23.9% | input: Frames, thinking | Apr 2025 | Official leaderboard |
| 48 | | 23.9% | input: Video | Dec 2024 | Official leaderboard |
| 49 | | 23.3% | input: V+A | Feb 2026 | Official leaderboard |
| 50 | | 23.1% | input: Video | Mar 2026 | Official leaderboard |
| 51 | | 22.6% | input: Video | Feb 2026 | Official leaderboard |
| 52 | | 22.1% | input: Video | Aug 2025 | Official leaderboard |
| 53 | | 22% | input: Audio | Jul 2025 | Official leaderboard |
| 54 | | 21.8% | input: V+A | Mar 2026 | Official leaderboard |
| 55 | | 21.3% | input: Video | Aug 2025 | Official leaderboard |
| 56 | | 21.3% | input: V+A | Mar 2025 | Official leaderboard |
| 57 | | 20.4% | input: Video, instruct | Oct 2025 | Official leaderboard |
| 58 | | 19.9% | input: Frames | Feb 2025 | Official leaderboard |
| 59 | | 19.9% | input: Video, instruct | Jan 2025 | Official leaderboard |
| 60 | MiniCPM-o-2_6 | 19.4% | input: V+A | Jan 2025 | Official leaderboard |
| 61 | | 19% | input: Video | Feb 2026 | Official leaderboard |
| 62 | | 19% | input: V+A | Jan 2025 | Official leaderboard |
| 63 | | 17.7% | input: Frames | Feb 2025 | Official leaderboard |
| 64 | | 16.9% | input: Video | Feb 2026 | Official leaderboard |
| 65 | VideoLLAMA3-7B | 15.9% | input: Video | Jan 2024 | Official leaderboard |
| 66 | Kimi-VL-A3B-Instruct | 15.8% | input: Frames, instruct | Apr 2025 | Official leaderboard |
| 67 | | 13.2% | input: Video | Aug 2024 | Official leaderboard |
Rankings by part
We read 10 parts from the benchmark’s own leaderboard in 3 groups: modality, length and category. Each ranks models on one part of the test. Qwen3-Omni-30B-A3B-Thinking leads every one of them. Arrows show how far a model moved from its overall rank among the models we track.
Visual
67 models we track have a score here.
- 1
Qwen3-Omni-30B-A3B-Thinking 62.5%
- 2
Gemini 3.1 Pro 53.2%
- 3
Voxtral Small 47.2%
- 4
Qwen3-VL-235B-A22B 46.2%
- 5
Qwen3-VL-32B-Thinking 44.9%
- 6 Kimi K2.6 43.1%
- 7
Qwen3-VL-30B-A3B-Thinking 41.8%
- 8
Qwen3-VL-235B-A22B-Instruct 40.5%
- 9
Gemma 3 27B ↑1 40.3%
- 10
Qwen3-VL-8B-Thinking ↓1 39.9%
Audio
67 models we track have a score here.
- 1
Qwen3-Omni-30B-A3B-Thinking 57.2%
- 2
Gemini 3.1 Pro 48.1%
- 3
Qwen3-VL-235B-A22B ↑1 44.9%
- 4
Qwen3-VL-32B-Thinking ↑1 44.1%
- 5
Voxtral Small ↓2 41.9%
- 6 Kimi K2.6 41.3%
- 7
Qwen3-VL-30B-A3B-Thinking 40.4%
- 8
Qwen3-VL-235B-A22B-Instruct 39.4%
- 9
Qwen3-VL-8B-Thinking 38.7%
- 10
Gemma 3 27B 38.7%
Speech
67 models we track have a score here.
- 1
Qwen3-Omni-30B-A3B-Thinking 63.9%
- 2
Gemini 3.1 Pro 54.6%
- 3
Voxtral Small 48.4%
- 4
Qwen3-VL-235B-A22B 46.9%
- 5
Qwen3-VL-32B-Thinking 45.5%
- 6 Kimi K2.6 43.8%
- 7
Qwen3-VL-30B-A3B-Thinking 42.5%
- 8
Qwen3-VL-235B-A22B-Instruct 41%
- 9
Gemma 3 27B ↑1 40.9%
- 10
Qwen3-VL-8B-Thinking ↓1 40.5%
Short
67 models we track have a score here.
- 1
Qwen3-Omni-30B-A3B-Thinking 58%
- 2
Gemini 3.1 Pro 48.5%
- 3
Qwen3-VL-235B-A22B-Instruct ↑5 43.5%
- 4
Qwen3-VL-235B-A22B 42.6%
- 5
Voxtral Small ↓2 42.1%
- 6
Qwen3-VL-32B-Thinking ↓1 41.7%
- 7
Qwen3-VL-32B-Instruct ↑4 40.2%
- 8
Qwen3-VL-8B-Thinking ↑1 40.1%
- 9 Kimi K2.6 ↓3 40%
- 10
Gemma 3 27B 38.3%
Long
67 models we track have a score here.
- 1
Qwen3-Omni-30B-A3B-Thinking 60.8%
- 2
Gemini 3.1 Pro 53.8%
- 3
Qwen3-VL-235B-A22B ↑1 48.7%
- 4
Voxtral Small ↓1 47.2%
- 5
Qwen3-VL-32B-Thinking 46.6%
- 6
Qwen3-VL-30B-A3B-Thinking ↑1 43.4%
- 7
Gemma 3 27B ↑3 42.9%
- 8 Kimi K2.6 ↓2 42.5%
- 9
Qwen3-VL-235B-A22B-Instruct ↓1 42.2%
- 10
Qwen3-VL-8B-Thinking ↓1 41%
Medium
67 models we track have a score here.
- 1
Qwen3-Omni-30B-A3B-Thinking 63.6%
- 2
Gemini 3.1 Pro 53.1%
- 3
Voxtral Small 47.5%
- 4
Qwen3-VL-235B-A22B 44.8%
- 5
Qwen3-VL-32B-Thinking 44%
- 6 Kimi K2.6 43.5%
- 7
Qwen3-VL-30B-A3B-Thinking 41%
- 8
Qwen3-VL-235B-A22B-Instruct 39.5%
- 9
Qwen3-VL-8B-Thinking 39.3%
- 10
Gemma 3 27B 38.9%
Core perception
67 models we track have a score here.
- 1
Qwen3-Omni-30B-A3B-Thinking 51.2%
- 2
Gemini 3.1 Pro 44.7%
- 3
Qwen3-VL-235B-A22B ↑1 40.5%
- 4
Voxtral Small ↓1 39.7%
- 5
Qwen3-VL-32B-Thinking 39.4%
- 6 Kimi K2.6 38.2%
- 7
Qwen3-VL-235B-A22B-Instruct ↑1 36.6%
- 8
Qwen3-VL-30B-A3B-Thinking ↓1 36.5%
- 9
Gemma 3 27B ↑1 35.4%
- 10
Qwen3-VL-32B-Instruct ↑1 35.3%
Reasoning
67 models we track have a score here.
- 1
Qwen3-Omni-30B-A3B-Thinking 72.4%
- 2
Gemini 3.1 Pro 66%
- 3
Voxtral Small 56.1%
- 4
Qwen3-VL-235B-A22B 54.6%
- 5
Qwen3-VL-32B-Thinking 52.1%
- 6 Kimi K2.6 51.5%
- 7
Qwen3-VL-30B-A3B-Thinking 50.1%
- 8
Gemma 3 27B ↑2 48.4%
- 9
Qwen3-VL-8B-Thinking 48%
- 10
Qwen3-Omni-30B-A3B ↑3 47.4%
Information
67 models we track have a score here.
- 1
Qwen3-Omni-30B-A3B-Thinking 66.2%
- 2
Gemini 3.1 Pro 57.6%
- 3
Voxtral Small 49.9%
- 4
Qwen3-VL-235B-A22B 46.7%
- 5
Qwen3-VL-32B-Thinking 45.7%
- 6 Kimi K2.6 43.8%
- 7
Qwen3-VL-30B-A3B-Thinking 42.4%
- 8
Qwen3-VL-235B-A22B-Instruct 41.2%
- 9
Gemma 3 27B ↑1 40.5%
- 10
Qwen3-VL-8B-Thinking ↓1 40.3%
Multimodal
67 models we track have a score here.
- 1
Qwen3-Omni-30B-A3B-Thinking 67%
- 2
Gemini 3.1 Pro 59.6%
- 3
Qwen3-VL-32B-Thinking ↑2 49.7%
- 4
Qwen3-VL-235B-A22B 48.6%
- 5
Voxtral Small ↓2 48.5%
- 6 Kimi K2.6 46.8%
- 7
Qwen3-VL-235B-A22B-Instruct ↑1 46%
- 8
Qwen3-VL-30B-A3B-Thinking ↓1 45.9%
- 9
Qwen3-VL-8B-Thinking 45.3%
- 10
Qwen3-VL-32B-Instruct ↑1 44.3%
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official LongShOTBench leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official LongShOTBench leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Audio and video
- Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy.
Qwen 3.8 Max 88.7%
- LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization.
Gemini 4 Argon 91.7%
- MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy.
Qwen 3.8 Max 82.4%
- MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy.
Qwen 3.8 Max 90.8%
- FLEURS In use Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better.
Phi-4-multimodal 4.00%
- VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules.
Qwen3.5-Omni-Plus 93.1%
Data audit
9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 7 months ago, in April 2026, for Qwen3.6 27B. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
- Pass Independent runs Scores from the official leaderboard (106 entries).
- Pass Models scored 67 models we track, plus 39 other entries on its leaderboard.
- Warning Newest score The newest came 7 months ago, in April 2026, for Qwen3.6 27B.
- Note Frontier models None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40).
- Pass Headroom The best is 64.2%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on longshot.cvmbzuai.com: 67 rows for models we track and 39 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.