Benchmark Almanac

Kimi-VL-A3B-Thinking-2506

Kimi-VL-A3B-Thinking-2506 places in the top three on VRBench.

Moonshot AI Released June 21st, 2025 Family Kimi-VL The lab's model card Compare with others
25
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Moonshot AI Jun 2025

Kimi-VL-A3B-Thinking-2506

Open weights Small

127.7 #293 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
  • MMMU-Pro 46.3% #81 /90
  • MMVU 57.5% #24 /26
  • MMLongBench-Doc 42.1% #13 /17

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
MMBench-CN v1.1 79.8% #46 of 178 Official leaderboard
MMLU 82% #65 of 131 Lab model card
MMLU-Pro 68.5% #128 of 167 Lab model card

Math and science

BenchmarkScoreRankSource
MATH 91.8% #8 of 99 Lab model card
GPQA Diamond 42.3% #266 of 302 Lab model card

Agents and tools

BenchmarkScoreRankSource
ScreenSpot-v2 91.4% #9 of 14 Lab model card
ScreenSpot-Pro 52.8% #27 of 32 Lab model card
OSWorld-G 52.5% #5 of 5 Lab model card

Vision, audio and robotics

BenchmarkScoreRankSource
MM-Vet 77% #9 of 164 Official leaderboard
VRBench 61.8% #3 of 19 Official leaderboard
MMBench-EN v1.1 80.4% #50 of 178 Official leaderboard
OCRBench 86.9% #14 of 34 Lab model card
MathVista 80.1% #30 of 70 Lab model card
MMLongBench-Doc 42.1% #13 of 17 Official leaderboard
MMMU 64% #80 of 103 Lab model card
MATH-Vision (MathVision) 56.9% #41 of 51 Lab model card
MMStar 70.4% #29 of 36 Lab model card
RealWorldQA 70% #32 of 38 Lab model card
Video-MME 71.9% #44 of 51 Lab model card
V* 83.2% #15 of 17 Lab model card
Video-MMMU 65.2% #32 of 36 Lab model card
MMMU-Pro 46.3% #81 of 90 Lab model card
GSM8K-V 11.6% #22 of 24 Official leaderboard
MMVU 57.5% #24 of 26 Lab model card

Other

BenchmarkScoreRankSource
OpenVLM Leaderboard 74.3 #15 of 173 Official leaderboard

Its lab's numbers put it at 127.7, number 293 of 404

The Self-Reported Index fits one score to the 19 capability results Moonshot AI reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 125.3 to 130.1. How we compute it.

Its lab's numbers next to 7 leaderboards

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark Lab says Board says Gap
MMBench-EN v1.1 84.4% MMBench-EN-v1.1 (Acc) 80.4% Leaderboard +4.0 pts
ScreenSpot-Pro 52.8% ScreenSpot-Pro (Acc) 51% Leaderboard +1.8 pts
MM-Vet 78.1% MMVet (Acc) 77% Leaderboard +1.1 pts
MMStar 70.4% MMStar (Acc) 70% Leaderboard +0.4 pts
Video-MMMU 65.2% VideoMMMU (Pass@1) 65.2% Leaderboard Same
MATH-Vision (MathVision) 56.9% MATH-Vision (Pass@1) 56.9% Leaderboard Same
MMLongBench-Doc 42.1% MMLongBench-DOC (Acc) 42.1% Leaderboard, VL; 2.8B activated (16B total) Same

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Kimi-VL-A3B-Thinking-2506 model card (Hugging Face) Jun 21, 2025

Its system card

Every result in the longer documents the lab published with it.