Benchmark Almanac

Seed1.5-VL

Seed1.5-VL ranks first on 4 of the 56 benchmarks with five or more models scored.

ByteDance Seed Released May 12th, 2025 Family Seed 1 The lab's model card Compare with others
66
Benchmarks with a score
4
Ranked first, out of five or more models
15
In the top three
ByteDance Seed May 2025

Seed1.5-VL

Closed weights Flagship

138.7 #186 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
  • MMMU-Pro 67.6% #56 /90
  • SimpleVQA 63.4% #16 /30
  • MMVU 70.1% #15 /26

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
SimpleVQA 63.4% #16 of 30 Lab model card
HallusionBench 60.3% #23 of 31 Lab model card

Math and science

BenchmarkScoreRankSource
OlympiadBench 65% #3 of 3 Lab model card

Agents and tools

BenchmarkScoreRankSource
ScreenSpot-v2 95.2% #1 of 14 Lab model card
WebVoyager 87.2% #4 of 12 Lab model card
Online-Mind2Web 76.4% #5 of 10 Lab model card
VisualWebBench 87.3% #1 of 2 Lab model card
Windows Agent Arena 39.6% #3 of 5 Lab model card
OSWorld-MCP 38.2% #6 of 9 Official leaderboard
OSWorld 36.7% #9 of 13 Lab model card
ScreenSpot-Pro 60.9% #23 of 32 Lab model card
OSWorld-Verified 33.8% #25 of 33 Official leaderboard
AndroidWorld 62.1% #8 of 9 Official leaderboard

Vision, audio and robotics

BenchmarkScoreRankSource
DocVQA 96.9% #1 of 36 Lab model card
We-Math 77.5% #1 of 21 Official leaderboard
MME-Reasoning 59.9% #1 of 11 Official leaderboard
MMMU 77.9% #13 of 136 Official leaderboard
InfographicVQA 91.2% #3 of 26 Lab model card
BLINK 72.1% #5 of 29 Lab model card
MM-HELIX 66.9% #3 of 17 Official leaderboard
MMBench 89.9% #2 of 11 Lab model card
MathVista 85.6% #15 of 70 Lab model card
VideoReasonBench 11.5% #3 of 14 Official leaderboard
ChartQA 89.1% #9 of 37 Lab model card
Charades-STA 64% #1 of 4 Lab model card
RefCOCO / RefCOCO+ / RefCOCOg 91.3% #9 of 29 Lab model card
MMSI-Bench 29.7% #9 of 28 Official leaderboard
All-Angles Bench 58.6% #2 of 6 Lab model card
ViVerBench 73.1% #3 of 8 Official leaderboard
TextVQA 81.8% #7 of 17 Lab model card
MMStar 77.8% #15 of 36 Lab model card
CharXiv (DQ) 92.6% #3 of 7 Lab model card
TempCompass 83.7% #3 of 7 Lab model card
VisuLogic 35% #3 of 7 Lab model card
CountBench 93.7% #13 of 28 Lab model card
RealWorldQA 78.4% #19 of 38 Lab model card
Video-MMMU 81.4% #20 of 36 Lab model card
MVBench 74.4% #13 of 23 Lab model card
MMVU 70.1% #15 of 26 Lab model card
VisFactor 23.7% #15 of 26 Official leaderboard
OCRBench 86.1% #20 of 34 Lab model card
V* 89% #10 of 17 Lab model card
MMVP 69.3% #3 of 5 Lab model card
AI2D 87.3% #32 of 52 Lab model card
MMMU-Pro 67.6% #56 of 90 Lab model card
LongVideoBench 74% #7 of 11 Lab model card
VLMs Are Blind 92.1% #9 of 14 Lab model card
MATH-Vision (MathVision) 68.7% #33 of 51 Lab model card
MotionBench 68.4% #6 of 9 Lab model card
MLVU 82.1% #15 of 22 Lab model card
Video-MME 77.9% #35 of 51 Lab model card
LVBench 64.6% #22 of 31 Lab model card
SUNRGBD 32.0 #7 of 9 Lab model card
CharXiv 60.2% #63 of 75 Lab model card
ZeroBench 2.00% #29 of 31 Lab model card
TVBench 63.6% #7 of 7 Lab model card
OVBench 60% #5 of 5 Lab model card
OVO-Bench 72.3% #5 of 5 Lab model card
TOMATO 44.7% #5 of 5 Lab model card
StreamingBench 68% #3 of 3 Lab model card
DREAM-1K 43.9% #1 of 1 Lab model card
LVIS-MG 72.5% #1 of 1 Lab model card
Seed1.5-VL internal benchmark (ByteDance Seed in-house) 59.3% #1 of 1 Lab model card
StreamBench (StreamChat) 72.8% #1 of 1 Lab model card
TACoS 49.6% #1 of 1 Lab model card
TemporalBench 79.8% #1 of 1 Lab model card

Its lab's numbers put it at 138.7, number 186 of 404

The Self-Reported Index fits one score to the 33 capability results ByteDance Seed reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 136.8 to 140.5. How we compute it.

Its lab's numbers next to 8 leaderboards

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark Lab says Board says Gap
MotionBench 68.4% MotionBench 59.4% Leaderboard, frames: 2fps +9.0 pts
ScreenSpot-Pro 60.9% ScreenSpot-Pro 60.9% Leaderboard Same
AndroidWorld 62.1% Android World 62.1% Leaderboard Same
LVBench 64.6% LVBench 64.6% Leaderboard, thinking Same
MATH-Vision (MathVision) 68.7% MathVision 68.7% Leaderboard Same
MMMU 77.9% MMMU 77.9% Leaderboard, thinking Same
MMMU-Pro 67.6% MMMU-Pro 67.6% Leaderboard, thinking Same
VisuLogic 35% VisuLogic 35% Leaderboard, thinking Same

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Seed1.5-VL GitHub README and Technical Report (arXiv 2505.07062) May 12, 2025

Its system card

Every result in the longer documents the lab published with it. 3 of them aren't in the launch post.

Seed1.5-VL Technical Report May 11, 2025

Capabilities