Benchmark Almanac

BLINK

Seed2.1 Pro 81.4% leads since June 2026
GPT-4V 51.3% leads since April 2024
Seed2.1 Pro 81.4% leads since June 2026
In use

The official leaderboard hasn’t scored a model released after Claude 3 Opus (February 2024). Labs report their own numbers for 29 newer models, such as Seed2.1 Pro at 81.4%.

Models answer 3,807 multiple choice questions built from 14 classic computer vision tasks that use one or more images. Scoring is accuracy.

Test answers are hidden on EvalAI, so model cards usually report the 1,901 question validation split.

Released
April 2024
Announced on X
106K views, 405 likes
Kind
General visual questions
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Tasks
Public, plus a private test set
Measures
Capability
Language
English
In and out
Text, Images → Text
Interaction
Single turn
Refresh
One fixed set
Run by
A host
In views
Vision
Strongest model tested
Kimi K2.5 #76 on the capability index
Last in a launch post
October 2026
Last model added
October 2026
Scores refreshed
November 18th, 2026

In the authors’ words

We introduce Blink, a new benchmark for multimodal language models (LLMs) that focuses on core visual perception abilities not found in other evaluations. Most of the Blink tasks can be solved by humans "within a blink" (e.g., relative depth estimation, visual correspondence, forensics detection, and multi-view reasoning). However, we find these perception-demanding tasks cast significant challenges for current multimodal LLMs because they resist mediation through natural language. Blink reformats 14 classic computer vision tasks into 3,807 multiple-choice questions, paired with single or multiple images and visual prompting. While humans get 95.70% accuracy on average, Blink is surprisingly challenging for existing multimodal LLMs: even the best-performing GPT-4V and Gemini achieve accuracies of 51.26% and 45.72%, only 13.17% and 7.63% higher than random guessing, indicating that such perception abilities have not "emerged" yet in recent multimodal LLMs. Our analysis also highlights that specialist CV models could solve these problems much better, suggesting potential pathways for future improvements. We believe Blink will stimulate the community to help multimodal LLMs catch up with human-level visual perception.

Abstract of BLINK: Multimodal Large Language Models Can See but Not Perceive (arXiv:2404.12390, CC0)
30
Models scored in lab cards
19
Launch posts list it, from 13 labs
24
System cards report it, from 13 labs

Seed2.1 Pro leads with 81.4%

Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.

GPT-4V leads with 51.3% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.

Seed2.1 Pro leads with 81.4%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.

  1. 1 Seed2.1 Pro 81.4%
  2. 2 Seed2.0 Pro 79.5%
  3. 3 Kimi K2.5 78.9%
  4. 4 Seed1.8 74.3%
  5. 5 Seed1.5-VL 72.1%
  6. 6 Nemotron 3.5 Super VL 71.3%
  7. 7 MiniCPM-V-4.7-35B-A3B 70.3%
  8. 8 ERNIE 5.0 70%
  9. 9 EXAONE 4.5 33B 68.8%
  10. 10 SenseNova-SI-1.3-InternVL3-8B 68%
  11. 11 Qwen3-VL-235B-A22B 67.1%
  12. 12 STEP3-VL-10B 66.8%
  13. 13 GLM-4.6V 65.5%
  14. 14 GLM-4.5V 65.3%
  15. 15 GLM-4.1V-9B-Thinking 65.1%
  16. 16 Qwen2.5-VL-72B 64.4%
  17. 17 LongCat-Flash-Omni 63.1%
  18. 18 ERNIE-4.5-VL-28B-A3B-Thinking 62.8%
  19. 19 LFM2.5-VL-3B 61.5%
  20. 20 Gemini 1.5 Pro May 2024 61.4%
  21. 21 InternVL3.5-241B-A28B 61.4%
  22. 22 Phi-4-multimodal 61.3%
  23. 23 gemini-robotics-er-1.5-preview 57.8%
  24. 24 Phi-3.5-vision-instruct 57%
  25. 25 Nemotron Nano 12B v2 VL 56.7%
  26. 26 North Micro Vision Instruct 52.7%
  27. 27 LFM2-VL-1.6B 44.5%
  28. 28 Gemma 3 27B 39.6%
  29. 29 Gemma 3 4B 38%
  30. 30 Gemma 3 12B 35.9%

Source: Labs’ model cards

Sources: Labs’ model cards and official BLINK leaderboard

#ModelScoreSettingDateSource
1 Seed2.1 Pro 81.4% Jun 2026 Lab model card
2 Seed2.0 Pro 79.5% Feb 2026 Lab model card
3 Kimi K2.5 78.9% Jan 2026 Lab model card
4 Seed1.8 74.3% Dec 2025 Lab model card
5 Seed1.5-VL 72.1% May 2025 Lab model card
6 Nemotron 3.5 Super VL 71.3% Oct 2026 Lab model card
7 MiniCPM-V-4.7-35B-A3B 70.3% Oct 2026 Lab model card
8 ERNIE 5.0 70% Feb 2026 Lab model card
9 EXAONE 4.5 33B 68.8% Apr 2026 Lab model card
10 SenseNova-SI-1.3-InternVL3-8B 68% Jan 2026 Lab model card
11 Qwen3-VL-235B-A22B 67.1% Sep 2025 Lab model card
12 STEP3-VL-10B 66.8% Jan 2026 Lab model card
13 GLM-4.6V 65.5% Dec 2025 Lab model card
14 GLM-4.5V 65.3% Aug 2025 Lab model card
15 GLM-4.1V-9B-Thinking 65.1% Jul 2025 Lab model card
16 Qwen2.5-VL-72B 64.4% Feb 2025 Lab model card
17 LongCat-Flash-Omni 63.1% Oct 2025 Lab model card
18 ERNIE-4.5-VL-28B-A3B-Thinking 62.8% Nov 2025 Lab model card
19 LFM2.5-VL-3B 61.5% Aug 2026 Lab model card
20 Gemini 1.5 Pro May 2024 61.4% May 2024 Lab model card
21 InternVL3.5-241B-A28B 61.4% Aug 2025 Lab model card
22 Phi-4-multimodal 61.3% Feb 2025 Lab model card
23 gemini-robotics-er-1.5-preview 57.8% Sep 2025 Lab model card
24 Phi-3.5-vision-instruct 57% Aug 2024 Lab model card
25 Nemotron Nano 12B v2 VL 56.7% Nov 2025 Lab model card
26 North Micro Vision Instruct 52.7% Aug 2026 Lab model card
27 LFM2-VL-1.6B 44.5% Nov 2025 Lab model card
28 Gemma 3 27B 39.6% Mar 2025 Lab model card
29 Gemma 3 4B 38% Mar 2025 Lab model card
30 Gemma 3 12B 35.9% Mar 2025 Lab model card
#ModelScoreSettingDateSource
1 GPT-4V 51.3% Nov 2023 Official leaderboard
2 Gemini 1.0 Pro 45.7% Dec 2023 Official leaderboard
3 LLaVA-v1.6-34B 45% Jan 2024 Official leaderboard
4 Claude 3 Opus 44.1% Feb 2024 Official leaderboard
5 Yi-VL-34B 42.8% Dec 2023 Official leaderboard
6 Qwen-VL-Max 41.9% Jan 2024 Official leaderboard
7 Yi-VL-6B 41.2% Dec 2023 Official leaderboard
8 LLaVA-v1.5-13B 40.5% Oct 2023 Official leaderboard
9 CogVLM-17B-Chat 39.4% Nov 2023 Official leaderboard
10 instructblip-vicuna-7b 38.6% May 2023 Official leaderboard
11 llava-v1.5-7b 38% Oct 2023 Official leaderboard
#ModelScoreSettingDateSource
1 Seed2.1 Pro 81.4% Jun 2026 Lab model card
2 Seed2.0 Pro 79.5% Feb 2026 Lab model card
3 Kimi K2.5 78.9% Jan 2026 Lab model card
4 Seed1.8 74.3% Dec 2025 Lab model card
5 Seed1.5-VL 72.1% May 2025 Lab model card
6 Nemotron 3.5 Super VL 71.3% Oct 2026 Lab model card
7 MiniCPM-V-4.7-35B-A3B 70.3% Oct 2026 Lab model card
8 ERNIE 5.0 70% Feb 2026 Lab model card
9 EXAONE 4.5 33B 68.8% Apr 2026 Lab model card
10 SenseNova-SI-1.3-InternVL3-8B 68% Jan 2026 Lab model card
11 Qwen3-VL-235B-A22B 67.1% Sep 2025 Lab model card
12 STEP3-VL-10B 66.8% Jan 2026 Lab model card
13 GLM-4.6V 65.5% Dec 2025 Lab model card
14 GLM-4.5V 65.3% Aug 2025 Lab model card
15 GLM-4.1V-9B-Thinking 65.1% Jul 2025 Lab model card
16 Qwen2.5-VL-72B 64.4% Feb 2025 Lab model card
17 LongCat-Flash-Omni 63.1% Oct 2025 Lab model card
18 ERNIE-4.5-VL-28B-A3B-Thinking 62.8% Nov 2025 Lab model card
19 LFM2.5-VL-3B 61.5% Aug 2026 Lab model card
20 Gemini 1.5 Pro May 2024 61.4% May 2024 Lab model card
21 InternVL3.5-241B-A28B 61.4% Aug 2025 Lab model card
22 Phi-4-multimodal 61.3% Feb 2025 Lab model card
23 gemini-robotics-er-1.5-preview 57.8% Sep 2025 Lab model card
24 Phi-3.5-vision-instruct 57% Aug 2024 Lab model card
25 Nemotron Nano 12B v2 VL 56.7% Nov 2025 Lab model card
26 North Micro Vision Instruct 52.7% Aug 2026 Lab model card
27 GPT-4V 51.3% Nov 2023 Official leaderboard
28 Gemini 1.0 Pro 45.7% Dec 2023 Official leaderboard
29 LLaVA-v1.6-34B 45% Jan 2024 Official leaderboard
30 LFM2-VL-1.6B 44.5% Nov 2025 Lab model card
31 Claude 3 Opus 44.1% Feb 2024 Official leaderboard
32 Yi-VL-34B 42.8% Dec 2023 Official leaderboard
33 Qwen-VL-Max 41.9% Jan 2024 Official leaderboard
34 Yi-VL-6B 41.2% Dec 2023 Official leaderboard
35 LLaVA-v1.5-13B 40.5% Oct 2023 Official leaderboard
36 Gemma 3 27B 39.6% Mar 2025 Lab model card
37 CogVLM-17B-Chat 39.4% Nov 2023 Official leaderboard
38 instructblip-vicuna-7b 38.6% May 2023 Official leaderboard
39 llava-v1.5-7b 38% Oct 2023 Official leaderboard
40 Gemma 3 4B 38% Mar 2025 Lab model card
41 Gemma 3 12B 35.9% Mar 2025 Lab model card

The benchmark's own leaderboard has 38 entries, 11 of them models we track. Switch the scores above to rank them together.

7 models have held number one in labs’ own numbers since its release in April 2024 Number one over time 8 models have held number one with claims included since its release in April 2024

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

OpenAI Google DeepMind Qwen ByteDance Seed Moonshot AI
Own leaderboard Not tested since Apr 2024
GPT-4V 51.3%
Lab claims 7 leaders
Seed2.1 Pro 81.4%

Sources: Official BLINK leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 Near the ceiling (85%) Seed2.1 Pro Seed1.8 Seed1.5-VL Qwen2.5-VL-72B Gemini 1.5 Pro GPT-4V LLaVA-v1.5-13B 0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 Near the ceiling (85%) Seed2.1 Pro Seed1.8

Sources: Official BLINK leaderboard and labs’ model cards

3 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
Microsoft
ByteDance Seed
Z.ai
Liquid AI
Qwen
Meituan
Baidu
SenseTime
StepFun
LG AI Research
Cohere
OpenBMB (ModelBest)
DateLaunch postScoreAs the post writes it
Oct 7, 2026 MiniCPM-V-4.7-35B-A3B 70.3 BLINK
Aug 12, 2026 LFM2.5-VL-3B 61.5 BLINK
Aug 12, 2026 North Micro Vision Instruct 52.7 BLINK
Apr 9, 2026 EXAONE 4.5 33B 68.8 BLINK
Feb 6, 2026 ERNIE 5.0 70 BLINK
Jan 13, 2026 STEP3-VL-10B 66.8 BLINK
Jan 8, 2026 SenseNova-SI-1.3-InternVL3-8B 68 BLINK
Dec 18, 2025 Seed1.8 74.3 BLINK
Dec 8, 2025 GLM-4.6V 65.5 BLINK (Val)
Nov 11, 2025 ERNIE-4.5-VL-28B-A3B-Thinking 62.8 BLINK
Oct 31, 2025 LongCat-Flash-Omni 63.1 BLINK
Sep 23, 2025 Qwen3-VL-235B-A22B 67.1 BLINK
Aug 12, 2025 LFM2-VL-1.6B 44.4 BLINK
Aug 11, 2025 GLM-4.5V 65.3 BLINK (val)
Jul 1, 2025 GLM-4.1V-9B-Thinking 65.1 BLINK
May 12, 2025 Seed1.5-VL 72.1 BLINK
Mar 12, 2025 Gemma 3 27B 39.6 BLINK (PT)
Feb 26, 2025 Phi-4-multimodal 61.3 BLINK
May 14, 2024 Gemini 1.5 Pro 61.4 Blink (val), 0-shot

24 system cards report it, from 13 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 15, 2026 Nemotron 3.5 Super VL technical report 71.3 BLINK, SpatialClaw harness, with agentic vision training, Table 7 Capabilities
Jun 23, 2026 Seed2.1 Pro model card 81.4 BLINK (Table 9) Capabilities
Apr 9, 2026 EXAONE 4.5 33B technical report 68.8 BLINK (Table 2, Reasoning mode). The 3.2 text says 68.7 Capabilities
Mar 21, 2026 Seed1.8 model card 74.3 BLINK (Table 2) Capabilities
Feb 14, 2026 Seed2.0 Pro model card 79.5 BLINK (Table 8) Capabilities
Feb 4, 2026 ERNIE 5.0 technical report 70 BLINK (Table 3) Capabilities
Jan 30, 2026 Kimi K2.5 technical report 78.9 BLINK (Table 4) Capabilities
Jan 14, 2026 STEP3-VL-10B technical report 66.8 BLINK, SeRe (Tables 1 and 3). PaCoRe 67.39 Capabilities
Jan 1, 2026 GLM-4.6V technical report 65.5 BLINK (Val), GLM-4.6V thinking (Table 2) Capabilities
Nov 28, 2025 LFM2-VL-1.6B technical report 44.5 BLINK, LFM2-VL-1.6B (Table 8) Capabilities
Nov 26, 2025 Qwen3-VL-235B-A22B technical report 67.1 BLINK (Table 2) Capabilities
Nov 6, 2025 Nemotron Nano 12B v2 VL technical report 56.7 BLINK (Val), reasoning on (Table 3). Reasoning off 57.6 Capabilities
Oct 31, 2025 LongCat-Flash-Omni technical report 63.1 BLINK (Table 5) Capabilities
Sep 25, 2025 gemini-robotics-er-1.5-preview technical report 57.8 BLINK (Table 19, thinking on). Thinking off 65.2 Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 61.4 BLINK (val) (Table 5) Capabilities
Aug 13, 2025 GLM-4.5V technical report 65.3 BLINK (Val), GLM-4.5V thinking (Table 2) Capabilities
Jul 1, 2025 GLM-4.1V-9B-Thinking technical report 65.1 BLINK (Val), GLM-4.1V-9B-Thinking thinking (Table 2) Capabilities
May 11, 2025 Seed1.5-VL technical report 72.1 BLINK, thinking (Table 6). Non-thinking 70.2 Capabilities
Mar 25, 2025 Gemma 3 4B technical report 38 BLINK, PT, 0-shot (Table 11) Capabilities
Mar 25, 2025 Gemma 3 27B technical report 39.6 BLINK, PT, 0-shot (Table 11) Capabilities
Mar 25, 2025 Gemma 3 12B technical report 35.9 BLINK, PT, 0-shot (Table 11) Capabilities
Mar 3, 2025 Phi-4-multimodal technical report 61.3 BLINK (test) (Table 1) Capabilities
Feb 19, 2025 Qwen2.5-VL-72B technical report 64.4 BLINK val (Table 3) Capabilities
Aug 30, 2024 Phi-3.5-vision-instruct technical report 57 BLINK (val), model picks the answer directly (Table 6) Capabilities

One logged change that moves scores

Changes by the maintainers, so a score from before a change may not compare with one from after it. Newest first, each with the maintainers' own source. The list may be incomplete. What we log.

  1. Grader changed September 27th, 2025

    The BLINK maintainers corrected the answer-matching prompt to mark the current response as the grading target. Runs using the earlier prompt can map free-form answers to choices differently. Source

More in Vision

All 636 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. Qwen 3.6 Plus 86%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MathVista In use Math reasoning in visual contexts (charts, geometry, puzzles) from 31 datasets. Answer accuracy. Seed2.1 Pro 90.7%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access Public tasks, plus a private test set.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (38 entries), launch posts (19 scores) and system cards (24 scores).
  • Pass Models scored 41 models we track, plus 27 other entries on its leaderboard.
  • Pass Newest score The newest is from October 2026, for Nemotron 3.5 Super VL.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Kimi K2.5 (#76).
  • Pass Headroom The best is 81.4%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on eval.ai: 11 rows for models we track and 27 other entries at the last read.
  • Pass Last read November 18th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check