Benchmark Almanac

VizWiz

FT [ 6 ] 47.5% tops its leaderboard
Not in model cards

Models answer over 31,000 spoken questions that blind people asked about photos they took with phones. Some questions are unanswerable from the image.

Each question has 10 crowdsourced answers and uses VQA accuracy.

Released
February 2018
Kind
General visual questions
Level
Everyday
Scored
Checked automatically
Answers
Short answer
Tasks
Public, plus a private test set

In the authors’ words

The study of algorithms to automatically answer visual questions currently is motivated by visual question answering (VQA) datasets constructed in artificial VQA settings. We propose VizWiz, the first goal-oriented VQA dataset arising from a natural VQA setting. VizWiz consists of over 31,000 visual questions originating from blind people who each took a picture using a mobile phone and recorded a spoken question about it, together with 10 crowdsourced answers per visual question. VizWiz differs from the many existing VQA datasets because (1) images are captured by blind photographers and so are often poor quality, (2) questions are spoken and so are more conversational, and (3) often visual questions cannot be answered. Evaluation of modern algorithms for answering visual questions and deciding if a visual question is answerable reveals that VizWiz is a challenging dataset. We introduce this dataset to encourage a larger community to develop more generalized algorithms that can assist blind people.

Abstract of VizWiz Grand Challenge: Answering Visual Questions from Blind People (arXiv:1802.08218, CC0)
9
Entries on its own leaderboard, 0 models we track

FT [ 6 ] tops the leaderboard with 47.5%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

  1. 47.5 FT [ 6 ]
  2. 46.9 FT [ 24 ]
  3. 46.9 VizWiz [ 24 ]
  4. 46.9 VizWiz [ 6 ]
  5. 46.6 FT [ 18 ]
  6. 46.5 VizWiz [ 18 ]
  7. 14.5 Q+I+A [ 24 ]
  8. 13.7 Q+I [ 18 ]
  9. 13.4 Q+I+BUA [ 6 ]
  1. 1 FT [ 6 ] 47.5%
  2. 2 FT [ 24 ] 46.9%
  3. 3 VizWiz [ 24 ] 46.9%
  4. 4 VizWiz [ 6 ] 46.9%
  5. 5 FT [ 18 ] 46.6%
  6. 6 VizWiz [ 18 ] 46.5%
  7. 7 Q+I+A [ 24 ] 14.5%
  8. 8 Q+I [ 18 ] 13.7%
  9. 9 Q+I+BUA [ 6 ] 13.4%

Source: Official VizWiz leaderboard

#EntryScoreSettingDateSource
1 FT [ 6 ] 47.5% Official leaderboard
2 FT [ 24 ] 46.9% Official leaderboard
3 VizWiz [ 24 ] 46.9% Official leaderboard
4 VizWiz [ 6 ] 46.9% Official leaderboard
5 FT [ 18 ] 46.6% Official leaderboard
6 VizWiz [ 18 ] 46.5% Official leaderboard
7 Q+I+A [ 24 ] 14.5% Official leaderboard
8 Q+I [ 18 ] 13.7% Official leaderboard
9 Q+I+BUA [ 6 ] 13.4% Official leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Vision

All 281 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%