Benchmark Almanac

LongVideoBench

Seed2.1 Pro 80.6% leads since June 2026
GPT-4o May 2024 66.7% leads since July 2024
Seed2.1 Pro 80.6% leads since June 2026
Fading

The official leaderboard hasn’t scored a model released after TimeMarker (November 2024). Labs report their own numbers for 11 newer models, such as Seed2.1 Pro at 80.6%.

Models answer referring reasoning questions over 3,763 web videos up to an hour long, with interleaved subtitles. Scored by multiple-choice accuracy.

Released
July 2024
Kind
Long videos
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Tasks
Public, plus a private test set
Measures
Capability
Language
English
In and out
Text, Video → Text
Interaction
Single turn
Items from
Real work and use
Refresh
One fixed set
Run by
A host
In views
Vision
Strongest model tested
Kimi K2.5 #76 on the capability index
Last in a launch post
October 2026, dropped before it saturated
Last model added
October 2026
Scores refreshed
November 20th, 2026

In the authors’ words

Large multimodal models (LMMs) are processing increasingly longer and richer inputs. Albeit the progress, few public benchmark is available to measure such development. To mitigate this gap, we introduce LongVideoBench, a question-answering benchmark that features video-language interleaved inputs up to an hour long. Our benchmark includes 3,763 varying-length web-collected videos with their subtitles across diverse themes, designed to comprehensively evaluate LMMs on long-term multimodal understanding. To achieve this, we interpret the primary challenge as to accurately retrieve and reason over detailed multimodal information from long inputs. As such, we formulate a novel video question-answering task termed referring reasoning. Specifically, as part of the question, it contains a referring query that references related video contexts, called referred context. The model is then required to reason over relevant video details from the referred context. Following the paradigm of referring reasoning, we curate 6,678 human-annotated multiple-choice questions in 17 fine-grained categories, establishing one of the most comprehensive benchmarks for long-form video understanding. Evaluations suggest that the LongVideoBench presents significant challenges even for the most advanced proprietary models (e.g. GPT-4o, Gemini-1.5-Pro, GPT-4-Turbo), while their open-source counterparts show an even larger performance gap. In addition, our results indicate that model performance on the benchmark improves only when they are capable of processing more frames, positioning LongVideoBench as a valuable benchmark for evaluating future-generation long-context LMMs.

Abstract of LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding (arXiv:2407.15754, CC0)
12
Models scored in lab cards
8
Launch posts list it, from 6 labs
10
System cards report it, from 7 labs

Seed2.1 Pro leads with 80.6%

Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.

GPT-4o leads with 66.7% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.

Seed2.1 Pro leads with 80.6%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.

  1. 1 Seed2.1 Pro 80.6%
  2. 2 Seed2.0 Pro 80.3%
  3. 3 Kimi K2.5 79.8%
  4. 4 Seed2.0 Lite (0428) 79%
  5. 5 Seed1.8 77.4%
  6. 6 Keye-VL-2.0-30B-A3B 74.1%
  7. 7 Seed1.5-VL 74%
  8. 8 LongCat-Flash-Omni 69.3%
  9. 9 MiniCPM-V-4.7-35B-A3B 68.9%
  10. 10 InternVL3.5-241B-A28B 67.1%
  11. 11 Qwen2.5-VL-72B 60.7%
  12. 12 Nemotron Nano 12B v2 VL 57%

Source: Labs’ model cards

Source: Official LongVideoBench leaderboard

Sources: Labs’ model cards and official LongVideoBench leaderboard

#ModelScoreSettingDateSource
1 Seed2.1 Pro 80.6% Jun 2026 Lab model card
2 Seed2.0 Pro 80.3% Feb 2026 Lab model card
3 Kimi K2.5 79.8% Jan 2026 Lab model card
4 Seed2.0 Lite (0428) 79% Apr 2026 Lab model card
5 Seed1.8 77.4% Dec 2025 Lab model card
6 Keye-VL-2.0-30B-A3B 74.1% May 2026 Lab model card
7 Seed1.5-VL 74% May 2025 Lab model card
8 LongCat-Flash-Omni 69.3% Oct 2025 Lab model card
9 MiniCPM-V-4.7-35B-A3B 68.9% Oct 2026 Lab model card
10 InternVL3.5-241B-A28B 67.1% Aug 2025 Lab model card
11 Qwen2.5-VL-72B 60.7% Feb 2025 Lab model card
12 Nemotron Nano 12B v2 VL 57% Nov 2025 Lab model card
#ModelScoreSettingDateSource
1 GPT-4o May 2024 66.7% max_frames=256 May 2024 Official leaderboard
2 Aria 65% max_frames=256 Sep 2024 Official leaderboard
3 LLaVA-Video-72B-Qwen2 64.9% max_frames=128 Sep 2024 Official leaderboard
4 Gemini 1.5 Pro May 2024 64.4% max_frames=256 May 2024 Official leaderboard
5 LLaVA-Video-7B-Qwen2 62.7% max_frames=128 Sep 2024 Official leaderboard
6 Gemini 1.5 Flash May 2024 62.4% max_frames=256 May 2024 Official leaderboard
7 GPT-4 Turbo Apr 2024 60.7% max_frames=256 Apr 2024 Official leaderboard
8 InternVL2-40B 60.6% max_frames=16 Jul 2024 Official leaderboard
9 mPLUG-Owl3-7B 60.1% max_frames=128 Aug 2024 Official leaderboard
10 TimeMarker 59.6% max_frames=128 Nov 2024 Official leaderboard
11 GPT-4o mini 58.8% max_frames=250 Jul 2024 Official leaderboard
12 MiniCPM-V-2_6 57.7% max_frames=64 Aug 2024 Official leaderboard
13 Qwen2-VL-7B-Instruct 56.8% max_frames=256 Aug 2024 Official leaderboard
14 kangaroo 54.8% max_frames=64 Nov 2024 Official leaderboard
15 Phi-3-vision-128k-instruct 49.9% max_frames=16 May 2024 Official leaderboard
16 Idefics2-8B 49.4% max_frames=16 Apr 2024 Official leaderboard
17 llava-v1.6-mistral-7b 47.1% max_frames=8 Jan 2024 Official leaderboard
18 InstructBLIP FLAN-T5-XXL 43.8% max_frames=8 Jun 2023 Official leaderboard
19 BLIP-2 FLAN-T5-XXL 43.5% max_frames=8 Feb 2023 Official leaderboard
20 LLaVA-v1.5-13B 43.1% max_frames=8 Oct 2023 Official leaderboard
21 llava-v1.5-7b 40.4% max_frames=8 Oct 2023 Official leaderboard
22 mPLUG-Owl2 39.4% max_frames=8 Nov 2023 Official leaderboard
23 Video-LLaVA-7B 37.6% max_frames=8 Nov 2023 Official leaderboard
#ModelScoreSettingDateSource
1 Seed2.1 Pro 80.6% Jun 2026 Lab model card
2 Seed2.0 Pro 80.3% Feb 2026 Lab model card
3 Kimi K2.5 79.8% Jan 2026 Lab model card
4 Seed2.0 Lite (0428) 79% Apr 2026 Lab model card
5 Seed1.8 77.4% Dec 2025 Lab model card
6 Keye-VL-2.0-30B-A3B 74.1% May 2026 Lab model card
7 Seed1.5-VL 74% May 2025 Lab model card
8 LongCat-Flash-Omni 69.3% Oct 2025 Lab model card
9 MiniCPM-V-4.7-35B-A3B 68.9% Oct 2026 Lab model card
10 InternVL3.5-241B-A28B 67.1% Aug 2025 Lab model card
11 GPT-4o May 2024 66.7% max_frames=256 May 2024 Official leaderboard
12 Aria 65% max_frames=256 Sep 2024 Official leaderboard
13 LLaVA-Video-72B-Qwen2 64.9% max_frames=128 Sep 2024 Official leaderboard
14 Gemini 1.5 Pro May 2024 64.4% max_frames=256 May 2024 Official leaderboard
15 LLaVA-Video-7B-Qwen2 62.7% max_frames=128 Sep 2024 Official leaderboard
16 Gemini 1.5 Flash May 2024 62.4% max_frames=256 May 2024 Official leaderboard
17 GPT-4 Turbo Apr 2024 60.7% max_frames=256 Apr 2024 Official leaderboard
18 Qwen2.5-VL-72B 60.7% Feb 2025 Lab model card
19 InternVL2-40B 60.6% max_frames=16 Jul 2024 Official leaderboard
20 mPLUG-Owl3-7B 60.1% max_frames=128 Aug 2024 Official leaderboard
21 TimeMarker 59.6% max_frames=128 Nov 2024 Official leaderboard
22 GPT-4o mini 58.8% max_frames=250 Jul 2024 Official leaderboard
23 MiniCPM-V-2_6 57.7% max_frames=64 Aug 2024 Official leaderboard
24 Nemotron Nano 12B v2 VL 57% Nov 2025 Lab model card
25 Qwen2-VL-7B-Instruct 56.8% max_frames=256 Aug 2024 Official leaderboard
26 kangaroo 54.8% max_frames=64 Nov 2024 Official leaderboard
27 Phi-3-vision-128k-instruct 49.9% max_frames=16 May 2024 Official leaderboard
28 Idefics2-8B 49.4% max_frames=16 Apr 2024 Official leaderboard
29 llava-v1.6-mistral-7b 47.1% max_frames=8 Jan 2024 Official leaderboard
30 InstructBLIP FLAN-T5-XXL 43.8% max_frames=8 Jun 2023 Official leaderboard
31 BLIP-2 FLAN-T5-XXL 43.5% max_frames=8 Feb 2023 Official leaderboard
32 LLaVA-v1.5-13B 43.1% max_frames=8 Oct 2023 Official leaderboard
33 llava-v1.5-7b 40.4% max_frames=8 Oct 2023 Official leaderboard
34 mPLUG-Owl2 39.4% max_frames=8 Nov 2023 Official leaderboard
35 Video-LLaVA-7B 37.6% max_frames=8 Nov 2023 Official leaderboard

The benchmark's own leaderboard has 35 entries, 23 of them models we track. Switch the scores above to rank them together.

Rankings by part

We read 5 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as Test 8 to 15 seconds, Test 15 to 60 seconds and Test 180 to 600 seconds. GPT-4o leads 3 of them, and LLaVA-Video-72B-Qwen2 leads the other 2. Arrows show how far a model moved from its overall rank among the models we track.

Test 8 to 15 seconds

23 models we track have a score here.

  1. 1 LLaVA-Video-72B-Qwen2 ↑2 72.4%
  2. 2 GPT-4o May 2024 ↓1 71.6%
  3. 3 InternVL2-40B ↑5 71.4%
  4. 4 Gemini 1.5 Pro May 2024 70.2%
  5. 5 LLaVA-Video-7B-Qwen2 69.7%
  6. 6 Aria ↓4 69.4%
  7. 7 mPLUG-Owl3-7B ↑2 69.3%
  8. 8 TimeMarker ↑2 67.3%
  9. 9 GPT-4o mini ↑2 66.6%
  10. 10 GPT-4 Turbo Apr 2024 ↓3 66.4%

6 models have held number one in labs’ own numbers since its release in July 2024 Number one over time 6 models have held number one with claims included since its release in July 2024

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

OpenAI Qwen ByteDance Seed Moonshot AI
Own leaderboard Not tested since Nov 2024
GPT-4o 66.7%
Lab claims 6 leaders
Seed2.1 Pro 80.6%

Sources: Official LongVideoBench leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 Near the ceiling (85%) Seed2.1 Pro Seed1.5-VL GPT-4o GPT-4 Turbo llava-v1.6-mistral-7b InstructBLIP FLAN-T5-XXL 0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 Near the ceiling (85%) Seed2.1 Pro Seed1.5-VL

Sources: Official LongVideoBench leaderboard and labs’ model cards

Labs headline it less often now, last in October 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

ByteDance Seed
Shanghai AI Lab
Meituan
Moonshot AI
Kuaishou
OpenBMB (ModelBest)
DateLaunch postScoreAs the post writes it
Oct 7, 2026 MiniCPM-V-4.7-35B-A3B 68.9 LongVideoBench
May 25, 2026 Keye-VL-2.0-30B-A3B 74.1 LongVideoBench
Apr 28, 2026 Seed2.0 Lite (0428) 79 LongVideoBench
Jan 27, 2026 Kimi K2.5 79.8 LongVideoBench
Dec 18, 2025 Seed1.8 77.4 LongVideoBench
Oct 31, 2025 LongCat-Flash-Omni 69.3 LongVideoBench
Aug 25, 2025 InternVL3.5-241B-A28B 67.1 LongVideoBench
May 12, 2025 Seed1.5-VL 74 LongVideoBench

10 system cards report it, from 7 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Jun 23, 2026 Seed2.1 Pro model card 80.6 LongVideoBench (Table 10) Capabilities
Jun 9, 2026 Keye-VL-2.0-30B-A3B technical report 74.1 LongVideoBench (Table 2) Capabilities
Mar 21, 2026 Seed1.8 model card 77.4 LongVideoBench (Table 3) Capabilities
Feb 14, 2026 Seed2.0 Pro model card 80.3 LongVideoBench (Table 9) Capabilities
Jan 30, 2026 Kimi K2.5 technical report 79.8 LongVideoBench (Table 4) Capabilities
Nov 6, 2025 Nemotron Nano 12B v2 VL technical report 57 LongVideoBench, reasoning on (Table 3). Reasoning off 63.6 Capabilities
Oct 31, 2025 LongCat-Flash-Omni technical report 69.3 LongVideoBench (Table 6) Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 67.1 LongVideoBench (val total) (Table 2, Table 9) Capabilities
May 11, 2025 Seed1.5-VL technical report 74 LongVideoBench, thinking (Table 7). Non-thinking 74.4 Capabilities
Feb 19, 2025 Qwen2.5-VL-72B technical report 60.7 LongVideoBench val (Table 8) Capabilities

More in Audio and video

All 695 in Audio and video
  • Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy. Qwen 3.8 Max 88.7%
  • LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization. Gemini 4 Argon 91.7%
  • MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy. Qwen 3.8 Max 82.4%
  • MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. Qwen 3.8 Max 90.8%
  • FLEURS In use Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better. Phi-4-multimodal 4.00%
  • VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. Qwen3.5-Omni-Plus 93.1%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access Public tasks, plus a private test set.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (35 entries), launch posts (8 scores) and system cards (10 scores).
  • Pass Models scored 35 models we track, plus 12 other entries on its leaderboard.
  • Pass Newest score The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Kimi K2.5 (#76).
  • Pass Headroom The best is 80.6%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on longvideobench.github.io: 23 rows for models we track and 12 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check