Benchmark Almanac

DREAM-1K

Kangaroo 8B 56% tops its leaderboard GPT-4V 34.4% leads the models we track
Seed1.5-VL 43.9% leads since May 2025
Dropped

Video models write fine-grained descriptions of 1,000 videos from varied sources. An automatic method scores how many described events match the reference.

Released with the Tarsier video description models. The 1,000 count follows the name.

Released
July 2024
Kind
Video captioning
Level
Everyday
Scored
Graded by a model
Answers
Open-ended
Tasks
All public
Strongest model tested
GPT-4o mini #177 on the capability index
Last in a launch post
May 2025, dropped before it saturated
Last model added
May 2025
Scores refreshed
November 12th, 2026
31
Entries on its own leaderboard, 8 models we track
1
Launch post lists it, from 1 lab
1
System card reports it, from 1 lab

Kangaroo 8B tops the leaderboard with 56%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is GPT-4V, at 34.4%.

Seed1.5-VL leads with 43.9%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

The newest model with a score here came out in August 2024, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

31 entries. Scroll sideways for the rest.

  1. 56.0 Kangaroo 8B
  2. 42.0 Tarsier2-7B-0115
  3. 40.7 Tarsier2-Recap-7B
  4. 40.1 Tarsier2-7B-1105
  5. 39.2 GPT-4o (snapshot unspecified)
  6. 36.3 Tarsier-34B
  7. 36.2 Gemini 1.5 Pro (snapshot unspecified)
  8. 34.8 Gemini 1.5 Flash (snapshot unspecified)
  9. 34.6 Tarsier-7B
  10. 34.4 GPT-4V
  11. 34.0 GPT-4o mini
  12. 34.0 LLaVA-Video 72B
  13. 33.2 VILA-1.5 34B
  14. 33.2 LLaVA-OneVision 72B
  15. 33.2 Qwen2-VL 72B
  16. 32.9 LLaVA-Video 32B
  17. 32.5 LLaVA-Video 7B
  18. 31.7 InternVL2 72B
  19. 31.7 LLaVA-OneVision 7B
  20. 31.2 Claude 3.5 Sonnet (snapshot unspecified)
  21. 30.5 MiniCPM-V 2.6 7B
  22. 29.9 VILA-1.5 7B
  23. 29.6 Qwen2-VL-7B-Instruct
  24. 28.9 VITA 8×7B
  25. 28.2 PLLaVA 34B
  26. 27.1 VideoLLaMA 2 72B
  27. 26.9 InternVL2 7B
  28. 26.6 VideoChat2-Mistral 7B
  29. 26.2 VideoLLAMA2-7B
  30. 20.4 Video-LLaVA-7B
  31. 19.5 sharegpt4video-8b
  1. 1 Kangaroo 8B frames=64 56%
  2. 2 Tarsier2-7B-0115 frames=16 42%
  3. 3 Tarsier2-Recap-7B frames=16 40.7%
  4. 4 Tarsier2-7B-1105 frames=16 40.1%
  5. 5 GPT-4o (snapshot unspecified) frames=16 39.2%
  6. 6 Tarsier-34B frames=8 36.3%
  7. 7 Gemini 1.5 Pro (snapshot unspecified) frames=1 fps 36.2%
  8. 8 Gemini 1.5 Flash (snapshot unspecified) frames=1 fps 34.8%
  9. 9 Tarsier-7B frames=8 34.6%
  10. 10 GPT-4V frames=8 34.4%
  11. 11 GPT-4o mini frames=8 34%
  12. 12 LLaVA-Video 72B frames=32 34%
  13. 13 VILA-1.5 34B frames=8 33.2%
  14. 14 LLaVA-OneVision 72B frames=16 33.2%
  15. 15 Qwen2-VL 72B frames=2 fps 33.2%
  16. 16 LLaVA-Video 32B frames=32 32.9%
  17. 17 LLaVA-Video 7B frames=32 32.5%
  18. 18 InternVL2 72B frames=8 31.7%
  19. 19 LLaVA-OneVision 7B frames=16 31.7%
  20. 20 Claude 3.5 Sonnet (snapshot unspecified) frames=16 31.2%
  21. 21 MiniCPM-V 2.6 7B frames=64 30.5%
  22. 22 VILA-1.5 7B frames=8 29.9%
  23. 23 Qwen2-VL-7B-Instruct frames=2 fps 29.6%
  24. 24 VITA 8×7B frames=16 28.9%
  25. 25 PLLaVA 34B frames=16 28.2%
  26. 26 VideoLLaMA 2 72B frames=16 27.1%
  27. 27 InternVL2 7B frames=8 26.9%
  28. 28 VideoChat2-Mistral 7B frames=16 26.6%
  29. 29 VideoLLAMA2-7B frames=16 26.2%
  30. 30 Video-LLaVA-7B frames=8 20.4%
  31. 31 sharegpt4video-8b frames=16 19.5%

Source: Official DREAM-1K leaderboard

Sources: Labs’ model cards and official DREAM-1K leaderboard

#EntryScoreSettingDateSource
1 Kangaroo 8B 56% frames=64 Official leaderboard
2 Tarsier2-7B-0115 42% frames=16 Jan 2025 Official leaderboard
3 Tarsier2-Recap-7B 40.7% frames=16 Feb 2025 Official leaderboard
4 Tarsier2-7B-1105 40.1% frames=16 Nov 2024 Official leaderboard
5 GPT-4o (snapshot unspecified) 39.2% frames=16 May 2024 Official leaderboard
6 Tarsier-34B 36.3% frames=8 Jul 2024 Official leaderboard
7 Gemini 1.5 Pro (snapshot unspecified) 36.2% frames=1 fps May 2024 Official leaderboard
8 Gemini 1.5 Flash (snapshot unspecified) 34.8% frames=1 fps May 2024 Official leaderboard
9 Tarsier-7B 34.6% frames=8 Jul 2024 Official leaderboard
10 GPT-4V 34.4% frames=8 Nov 2023 Official leaderboard
11 GPT-4o mini 34% frames=8 Jul 2024 Official leaderboard
12 LLaVA-Video 72B 34% frames=32 Sep 2024 Official leaderboard
13 VILA-1.5 34B 33.2% frames=8 May 2024 Official leaderboard
14 LLaVA-OneVision 72B 33.2% frames=16 Aug 2024 Official leaderboard
15 Qwen2-VL 72B 33.2% frames=2 fps Aug 2024 Official leaderboard
16 LLaVA-Video 32B 32.9% frames=32 Sep 2024 Official leaderboard
17 LLaVA-Video 7B 32.5% frames=32 Sep 2024 Official leaderboard
18 InternVL2 72B 31.7% frames=8 Jul 2024 Official leaderboard
19 LLaVA-OneVision 7B 31.7% frames=16 Aug 2024 Official leaderboard
20 Claude 3.5 Sonnet (snapshot unspecified) 31.2% frames=16 Jun 2024 Official leaderboard
21 MiniCPM-V 2.6 7B 30.5% frames=64 Aug 2024 Official leaderboard
22 VILA-1.5 7B 29.9% frames=8 May 2024 Official leaderboard
23 Qwen2-VL-7B-Instruct 29.6% frames=2 fps Aug 2024 Official leaderboard
24 VITA 8×7B 28.9% frames=16 Aug 2024 Official leaderboard
25 PLLaVA 34B 28.2% frames=16 Apr 2024 Official leaderboard
26 VideoLLaMA 2 72B 27.1% frames=16 Aug 2024 Official leaderboard
27 InternVL2 7B 26.9% frames=8 Jul 2024 Official leaderboard
28 VideoChat2-Mistral 7B 26.6% frames=16 May 2024 Official leaderboard
29 VideoLLAMA2-7B 26.2% frames=16 Jun 2024 Official leaderboard
30 Video-LLaVA-7B 20.4% frames=8 Nov 2023 Official leaderboard
31 sharegpt4video-8b 19.5% frames=16 May 2024 Official leaderboard
#ModelScoreSettingDateSource
1 Seed1.5-VL 43.9% May 2025 Lab model card
2 GPT-4V 34.4% frames=8 Nov 2023 Official leaderboard
3 GPT-4o mini 34% frames=8 Jul 2024 Official leaderboard
4 LLaVA-OneVision 72B 33.2% frames=16 Aug 2024 Official leaderboard
5 LLaVA-OneVision 7B 31.7% frames=16 Aug 2024 Official leaderboard
6 Qwen2-VL-7B-Instruct 29.6% frames=2 fps Aug 2024 Official leaderboard
7 VideoLLAMA2-7B 26.2% frames=16 Jun 2024 Official leaderboard
8 Video-LLaVA-7B 20.4% frames=8 Nov 2023 Official leaderboard
9 sharegpt4video-8b 19.5% frames=16 May 2024 Official leaderboard

Labs report scores for 1 model in their cards. Switch the scores above to rank them together.

Number one over time 2 models have held number one with claims included since its release in July 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Not tested since Aug 2024
GPT-4V 34.4%

Source: Official DREAM-1K leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) Seed1.5-VL GPT-4V 0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) Seed1.5-VL GPT-4V

Sources: Official DREAM-1K leaderboard and labs’ model cards

No launch post has listed it since May 2025

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

ByteDance Seed
DateLaunch postScoreAs the post writes it
May 12, 2025 Seed1.5-VL 43.9 Dream-1K

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
May 11, 2025 Seed1.5-VL technical report 43.9 Dream-1K, thinking (Table 7). Non-thinking 42.6 Capabilities

More in Audio and video

All 233 in Audio and video
  • Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy. Qwen 3.8 Max 88.7%
  • LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization. Gemini 4 Argon 91.7%
  • MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy. Qwen 3.8 Max 82.4%
  • MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. Qwen 3.8 Max 90.8%
  • FLEURS In use Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better. Phi-4-multimodal 4.00%
  • VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. Qwen3.5-Omni-Plus 93.1%