CVBench
Seed1.8 88% leads since December 2025 Models answer 2,638 manually checked 2D and 3D vision questions built from ADE20K, COCO, and Omni3D. Scoring is accuracy.
- Released
- June 2024
- Kind
- Spatial reasoning and counting
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- All public
- Strongest model tested
-
DeepSeek V4.1 Flash #34 on the capability index - Last in a launch post
- September 2026, dropped after it saturated
- Last model added
- October 2026
15
Models scored in lab cards
6
Launch posts list it, from 5 labs
13
System cards report it, from 7 labs
Seed1.8 leads with 88%
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
- 88.0 Seed1.8
- 87.6 GLM-4.6V
- 87.3 GLM-4.5V
- 87.2
ERNIE 5.0 - 86.5
Cosmos3-Nano - 85.0 GLM-4.1V-9B-Thinking
- 84.9 Cosmos3-Edge
- 84.3
gemini-robotics-er-1.5-preview - 84.0 Nemotron 3 Nano Omni 30B-A3B
- 83.8 ERNIE-4.5-VL-28B-A3B-Thinking
- 83.5
STEP3-VL-10B - 78.3 Nemotron Nano 12B v2 VL
- 77.9 DeepSeek V4.1 Flash
- 77.9 DeepSeek-V4.1-Flash Base
- 69.6 Nemotron 3.5 Super VL
- 1 Seed1.8 88%
- 2 GLM-4.6V 87.6%
- 3 GLM-4.5V 87.3%
- 4 ERNIE 5.0 87.2%
- 5 Cosmos3-Nano 86.5%
- 6 GLM-4.1V-9B-Thinking 85%
- 7 Cosmos3-Edge 84.9%
- 8 gemini-robotics-er-1.5-preview 84.3%
- 9 Nemotron 3 Nano Omni 30B-A3B 84%
- 10 ERNIE-4.5-VL-28B-A3B-Thinking 83.8%
- 11 STEP3-VL-10B 83.5%
- 12 Nemotron Nano 12B v2 VL 78.3%
- 13 DeepSeek V4.1 Flash 77.9%
- 14 DeepSeek-V4.1-Flash Base 77.9%
- 15 Nemotron 3.5 Super VL 69.6%
Source: Labs’ model cards
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Seed1.8 | 88% | Dec 2025 | Lab model card | |
| 2 | GLM-4.6V | 87.6% | Jan 2026 | Lab model card | |
| 3 | GLM-4.5V | 87.3% | Aug 2025 | Lab model card | |
| 4 | ERNIE 5.0 | 87.2% | Feb 2026 | Lab model card | |
| 5 | Cosmos3-Nano | 86.5% | May 2026 | Lab model card | |
| 6 | GLM-4.1V-9B-Thinking | 85% | Jul 2025 | Lab model card | |
| 7 | Cosmos3-Edge | 84.9% | May 2026 | Lab model card | |
| 8 | gemini-robotics-er-1.5-preview | 84.3% | Sep 2025 | Lab model card | |
| 9 | Nemotron 3 Nano Omni 30B-A3B | 84% | Apr 2026 | Lab model card | |
| 10 | ERNIE-4.5-VL-28B-A3B-Thinking | 83.8% | Nov 2025 | Lab model card | |
| 11 | STEP3-VL-10B | 83.5% | Jan 2026 | Lab model card | |
| 12 | Nemotron Nano 12B v2 VL | 78.3% | Nov 2025 | Lab model card | |
| 13 | DeepSeek V4.1 Flash | 77.9% | Sep 2026 | Lab model card | |
| 14 | DeepSeek-V4.1-Flash Base | 77.9% | Sep 2026 | Lab model card | |
| 15 | Nemotron 3.5 Super VL | 69.6% | Oct 2026 | Lab model card |
3 models have held number one in labs’ own numbers since its release in June 2024
Lab claims count from the day the card came out, independent runs from the model’s release.
Z.ai ByteDance Seed
Source: Labs’ model cards
Z.ai’s own number for GLM-4.1V-9B-Thinking passed 85% in July 2025
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Reported by the lab in its model card
Best so far
Source: Labs’ model cards
Labs headline it less often now, last in September 2026
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Sep 10, 2026 | DeepSeek V4.1 Flash | 77.9 | CVBench (EM) |
| Apr 28, 2026 | Nemotron 3 Nano Omni 30B-A3B | 84 | CVBench2D |
| Feb 6, 2026 | ERNIE 5.0 | 87.2 | CV-Bench |
| Dec 18, 2025 | Seed1.8 | 88 | CV-Bench |
| Nov 11, 2025 | ERNIE-4.5-VL-28B-A3B-Thinking | 83.8 | CV-Bench |
| Aug 11, 2025 | GLM-4.5V | 87.3 | CV-Bench |
13 system cards report it, from 7 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Oct 15, 2026 | Nemotron 3.5 Super VL technical report | 69.6 | CV-Bench, SpatialClaw harness, with agentic vision training, Table 7 | Capabilities |
| Sep 10, 2026 | DeepSeek-V4.1-Flash Base model card | 77.9 | CVBench, EM, 4-shot | Capabilities |
| May 31, 2026 | Cosmos3-Nano technical report | 86.5 | CVBench, Cosmos3-Nano 8B column, Table 10 | Capabilities |
| May 31, 2026 | Cosmos3-Edge technical report | 84.9 | CVBench, Cosmos3-Edge 2B column, Table 10 | Capabilities |
| Apr 27, 2026 | Nemotron 3 Nano Omni 30B-A3B technical report | 84 | CV-Bench, reasoning on, 84.2 reasoning off (Table 7) | Capabilities |
| Mar 21, 2026 | Seed1.8 model card | 88 | CV-Bench (Table 2) | Capabilities |
| Feb 4, 2026 | ERNIE 5.0 technical report | 87.2 | CV-Bench (Table 3) | Capabilities |
| Jan 14, 2026 | STEP3-VL-10B technical report | 83.5 | CVBench, SeRe (Tables 1 and 3). PaCoRe 85.92 | Capabilities |
| Jan 1, 2026 | GLM-4.6V technical report | 87.6 | CV-Bench, GLM-4.6V thinking (Table 2) | Capabilities |
| Nov 6, 2025 | Nemotron Nano 12B v2 VL technical report | 78.3 | CV-Bench, reasoning on (Table 3). Reasoning off 81.0 | Capabilities |
| Sep 25, 2025 | gemini-robotics-er-1.5-preview technical report | 84.3 | CV-Bench (Table 19, thinking on). Thinking off 83.6 | Capabilities |
| Aug 13, 2025 | GLM-4.5V technical report | 87.3 | CV-Bench, GLM-4.5V thinking (Table 2) | Capabilities |
| Jul 1, 2025 | GLM-4.1V-9B-Thinking technical report | 85 | CV-Bench, GLM-4.1V-9B-Thinking thinking (Table 2) | Capabilities |
More in Vision
- MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
- MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
- CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
- MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy.
Qwen 3.8 Max 97.7% - MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%
- ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance.
Command A Vision 90.9%