Benchmark Almanac

TextVQA

MiniCPM-V-4.7-35B-A3B 85% leads since October 2026
In use

Models answer 45,336 questions about 28,408 images that require reading text in the image. Scoring is VQA accuracy against human answers.

Released
April 2019
Kind
Documents, charts and text in images
Level
Everyday
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Images → Text
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
In views
Vision
Strongest model tested
Gemma 3 27B #165 on the capability index
Last in a launch post
October 2026
Last model added
October 2026

In the authors’ words

Studies have shown that a dominant class of questions asked by visually impaired users on images of their surroundings involves reading text in the image. But today's VQA models can not read! Our paper takes a first step towards addressing this problem. First, we introduce a new "TextVQA" dataset to facilitate progress on this important problem. Existing datasets either have a small proportion of questions about text (e.g., the VQA dataset) or are too small (e.g., the VizWiz dataset). TextVQA contains 45,336 questions on 28,408 images that require reasoning about text to answer. Second, we introduce a novel model architecture that reads text in the image, reasons about it in the context of the image and the question, and predicts an answer which might be a deduction based on the text and the image or composed of the strings found in the image. Consequently, we call our approach Look, Read, Reason & Answer (LoRRA). We show that LoRRA outperforms existing state-of-the-art VQA models on our TextVQA dataset. We find that the gap between human performance and machine performance is significantly larger on TextVQA than on VQA 2.0, suggesting that TextVQA is well-suited to benchmark progress along directions complementary to VQA 2.0.

Abstract of Towards VQA Models That Can Read (arXiv:1904.08920, CC0)
18
Models scored in lab cards
11
Launch posts list it, from 8 labs
11
System cards report it, from 6 labs

MiniCPM-V-4.7-35B-A3B leads with 85%

Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.

  1. 1 MiniCPM-V-4.7-35B-A3B 85%
  2. 2 Command A Vision 84.8%
  3. 3 InternVL3.5-241B-A28B 84.5%
  4. 4 Qwen2.5-Omni-7B 84.4%
  5. 5 LFM2.5-VL-3B 84.3%
  6. 6 Qwen2.5-VL-72B 83.5%
  7. 7 Gemini 1.0 Ultra 82.3%
  8. 8 Seed1.5-VL 81.8%
  9. 9 Nemotron 3 Nano Omni 30B-A3B 81%
  10. 10 Gemini 1.5 Pro May 2024 78.7%
  11. 11 GPT-4 Mar 2023 78%
  12. 12 Nemotron Nano 12B v2 VL 76.1%
  13. 13 Phi-4-multimodal 75.6%
  14. 14 Gemini 1.5 Pro Feb 2024 73.5%
  15. 15 Phi-3.5-vision-instruct 72%
  16. 16 Gemma 3 27B 68.6%
  17. 17 Gemma 3 12B 67.7%
  18. 18 Gemma 3 4B 57.8%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 MiniCPM-V-4.7-35B-A3B 85% Oct 2026 Lab model card
2 Command A Vision 84.8% Jul 2025 Lab model card
3 InternVL3.5-241B-A28B 84.5% Aug 2025 Lab model card
4 Qwen2.5-Omni-7B 84.4% Mar 2025 Lab model card
5 LFM2.5-VL-3B 84.3% Aug 2026 Lab model card
6 Qwen2.5-VL-72B 83.5% Feb 2025 Lab model card
7 Gemini 1.0 Ultra 82.3% Dec 2023 Lab model card
8 Seed1.5-VL 81.8% May 2025 Lab model card
9 Nemotron 3 Nano Omni 30B-A3B 81% Apr 2026 Lab model card
10 Gemini 1.5 Pro May 2024 78.7% May 2024 Lab model card
11 GPT-4 Mar 2023 78% Mar 2023 Lab model card
12 Nemotron Nano 12B v2 VL 76.1% Nov 2025 Lab model card
13 Phi-4-multimodal 75.6% Feb 2025 Lab model card
14 Gemini 1.5 Pro Feb 2024 73.5% Feb 2024 Lab model card
15 Phi-3.5-vision-instruct 72% Aug 2024 Lab model card
16 Gemma 3 27B 68.6% Mar 2025 Lab model card
17 Gemma 3 12B 67.7% Mar 2025 Lab model card
18 Gemma 3 4B 57.8% Mar 2025 Lab model card

6 models have held number one in labs’ own numbers since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Google DeepMind Qwen Cohere OpenBMB (ModelBest)
Lab claims 6 leaders

Source: Labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 Near the ceiling (85%) MiniCPM-V-4.7-35B-A3B Command A Vision Gemini 1.0 Ultra GPT-4 0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 Near the ceiling (85%) MiniCPM-V-4.7-35B-A3B Gemini 1.0 Ultra

Source: Labs’ model cards

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

OpenAI
Google DeepMind
Microsoft
Qwen
ByteDance Seed
Cohere
Liquid AI
OpenBMB (ModelBest)
DateLaunch postScoreAs the post writes it
Oct 7, 2026 MiniCPM-V-4.7-35B-A3B 85 TextVQA_VAL
Aug 12, 2026 LFM2.5-VL-3B 84.3 TextVQA (val)
Jul 31, 2025 Command A Vision 84.8 TextVQA
May 12, 2025 Seed1.5-VL 81.8 TextVQA
Mar 27, 2025 Qwen2.5-Omni-7B 84.4 TextVQAval
Mar 12, 2025 Gemma 3 27B 68.6 TextVQA (val) (PT); IT = 65.1
Feb 26, 2025 Phi-4-multimodal 75.6 TextVQA (val)
May 14, 2024 Gemini 1.5 Pro 78.7 TextVQA (val), 0-shot
Feb 15, 2024 Gemini 1.5 Pro 73.5 TextVQA (val), 0-shot
Dec 6, 2023 Gemini 1.0 Ultra 82.3 TextVQA (val)
Mar 14, 2023 GPT-4 78 TextVQA VQA score (val) 0-shot

11 system cards report it, from 6 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Apr 27, 2026 Nemotron 3 Nano Omni 30B-A3B technical report 81 TextVQA (Val), reasoning on, 85.1 reasoning off (Table 7) Capabilities
Nov 6, 2025 Nemotron Nano 12B v2 VL technical report 76.1 TextVQA (Val), reasoning on (Table 3). Reasoning off 85.4 Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 84.5 TextVQA (val) (Table 4) Capabilities
May 11, 2025 Seed1.5-VL technical report 81.8 TextVQA, thinking (Table 6). Non-thinking 84.2 Capabilities
Mar 26, 2025 Qwen2.5-Omni-7B technical report 84.4 TextVQA val (Table 5) Capabilities
Mar 25, 2025 Gemma 3 4B technical report 57.8 TextVQA, IT, P&S applied (Table 16) Capabilities
Mar 25, 2025 Gemma 3 27B technical report 65.1 TextVQA, IT, P&S applied (Table 16) Capabilities
Mar 25, 2025 Gemma 3 12B technical report 67.7 TextVQA, IT, P&S applied (Table 16) Capabilities
Mar 3, 2025 Phi-4-multimodal technical report 75.6 TextVQA (test) (Table 1) Capabilities
Feb 19, 2025 Qwen2.5-VL-72B technical report 83.5 TextVQA val (Table 5) Capabilities
Aug 30, 2024 Phi-3.5-vision-instruct technical report 72 TextVQA (test) (Table 5) Capabilities

More in Vision

All 636 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. Qwen 3.6 Plus 86%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MathVista In use Math reasoning in visual contexts (charts, geometry, puzzles) from 31 datasets. Answer accuracy. Seed2.1 Pro 90.7%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the benchmark’s paper (4 entries), launch posts (11 scores) and system cards (11 scores).
  • Pass Models scored 18 models we track, plus 4 other entries on its leaderboard.
  • Pass Newest score The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165).
  • Pass Headroom The best is 85%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on arxiv.org: 4 entries at the last read, none a model we track.
  • Pass Last read November 18th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check