Benchmark Almanac

RealWorldQA

Qwen3.8 Flash 88.5% leads since August 2026
In use

Models answer a question about each of 700+ real-world photos, many taken from vehicles, that test basic spatial understanding. Answers are easy to verify.

Made by
xAI
Released
April 2024
Kind
Spatial reasoning and counting
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Strongest model tested
Qwen 3.8 Max #29 on the capability index
Last in a launch post
September 2026
Last model added
September 2026
38
Models scored in lab cards
29
Launch posts list it, from 7 labs
19
System cards report it, from 9 labs

Qwen3.8 Flash leads with 88.5% among models released by September 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in September 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

38 models. Scroll sideways for the rest.

  1. 88.5 Qwen3.8 Flash
  2. 88.0 Qwen 3.8 Max
  3. 87.7 Qwen3.8-Omni-Flash
  4. 86.9 Qwen3.7-Plus
  5. 86.7 Seed2.1 Pro
  6. 86.0 Seed2.0 Pro
  7. 85.9 Qwen 3.8 27B
  8. 85.4 Qwen 3.6 Plus
  9. 85.3 Qwen 3.6 35B-A3B
  10. 85.1 Qwen3.5-122B-A10B
  11. 84.1 Qwen3.5-35B-A3B
  12. 84.1 Qwen3.5-Omni-Plus
  13. 84.1 Qwen3.6 27B
  14. 83.9 Qwen3.5 397B-A17B
  15. 83.7 Qwen3.5-27B
  16. 81.3 Qwen3-VL-235B-A22B
  17. 80.3 Qwen3.5-9B
  18. 79.5 Qwen3.5-4B
  19. 78.4 Seed1.5-VL
  20. 77.8 Seed1.6-Thinking
  21. 75.7 Qwen2.5-VL-72B
  22. 75.2 InternVL3.5-241B-A28B
  23. 74.8 LongCat-Flash-Omni
  24. 74.5 Qwen3.5-2B
  25. 74.4 STEP3-VL-10B
  26. 73.3 Cosmos3-Edge
  27. 73.1 LFM2.5-VL-3B
  28. 72.2 Nemotron Nano 12B v2 VL
  29. 72.2 Cosmos3-Nano
  30. 70.4 Gemini 1.5 Pro
  31. 70.3 Qwen2.5-Omni-7B
  32. 70.0 Kimi-VL-A3B-Thinking-2506
  33. 65.8 LFM2-VL-1.6B
  34. 63.4 Qwen3.5-0.8B
  35. 62.2 North Micro Vision Instruct
  36. 53.9 Gemma 3 27B
  37. 52.2 Gemma 3 12B
  38. 45.5 Gemma 3 4B
  1. 1 Qwen3.8 Flash 88.5%
  2. 2 Qwen 3.8 Max 88%
  3. 3 Qwen3.8-Omni-Flash 87.7%
  4. 4 Qwen3.7-Plus 86.9%
  5. 5 Seed2.1 Pro 86.7%
  6. 6 Seed2.0 Pro 86%
  7. 7 Qwen 3.8 27B 85.9%
  8. 8 Qwen 3.6 Plus 85.4%
  9. 9 Qwen 3.6 35B-A3B 85.3%
  10. 10 Qwen3.5-122B-A10B 85.1%
  11. 11 Qwen3.5-35B-A3B 84.1%
  12. 12 Qwen3.5-Omni-Plus 84.1%
  13. 13 Qwen3.6 27B 84.1%
  14. 14 Qwen3.5 397B-A17B 83.9%
  15. 15 Qwen3.5-27B 83.7%
  16. 16 Qwen3-VL-235B-A22B 81.3%
  17. 17 Qwen3.5-9B 80.3%
  18. 18 Qwen3.5-4B 79.5%
  19. 19 Seed1.5-VL 78.4%
  20. 20 Seed1.6-Thinking 77.8%
  21. 21 Qwen2.5-VL-72B 75.7%
  22. 22 InternVL3.5-241B-A28B 75.2%
  23. 23 LongCat-Flash-Omni 74.8%
  24. 24 Qwen3.5-2B 74.5%
  25. 25 STEP3-VL-10B 74.4%
  26. 26 Cosmos3-Edge 73.3%
  27. 27 LFM2.5-VL-3B 73.1%
  28. 28 Nemotron Nano 12B v2 VL 72.2%
  29. 29 Cosmos3-Nano 72.2%
  30. 30 Gemini 1.5 Pro May 2024 70.4%
  31. 31 Qwen2.5-Omni-7B 70.3%
  32. 32 Kimi-VL-A3B-Thinking-2506 70%
  33. 33 LFM2-VL-1.6B 65.8%
  34. 34 Qwen3.5-0.8B 63.4%
  35. 35 North Micro Vision Instruct 62.2%
  36. 36 Gemma 3 27B 53.9%
  37. 37 Gemma 3 12B 52.2%
  38. 38 Gemma 3 4B 45.5%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 Qwen3.8 Flash 88.5% Aug 2026 Lab model card
2 Qwen 3.8 Max 88% Aug 2026 Lab model card
3 Qwen3.8-Omni-Flash 87.7% Sep 2026 Lab model card
4 Qwen3.7-Plus 86.9% Jun 2026 Lab model card
5 Seed2.1 Pro 86.7% Jun 2026 Lab model card
6 Seed2.0 Pro 86% Feb 2026 Lab model card
7 Qwen 3.8 27B 85.9% Aug 2026 Lab model card
8 Qwen 3.6 Plus 85.4% Apr 2026 Lab model card
9 Qwen 3.6 35B-A3B 85.3% Apr 2026 Lab model card
10 Qwen3.5-122B-A10B 85.1% Feb 2026 Lab model card
11 Qwen3.5-35B-A3B 84.1% Feb 2026 Lab model card
12 Qwen3.5-Omni-Plus 84.1% Mar 2026 Lab model card
13 Qwen3.6 27B 84.1% Apr 2026 Lab model card
14 Qwen3.5 397B-A17B 83.9% Feb 2026 Lab model card
15 Qwen3.5-27B 83.7% Feb 2026 Lab model card
16 Qwen3-VL-235B-A22B 81.3% Sep 2025 Lab model card
17 Qwen3.5-9B 80.3% Mar 2026 Lab model card
18 Qwen3.5-4B 79.5% Mar 2026 Lab model card
19 Seed1.5-VL 78.4% May 2025 Lab model card
20 Seed1.6-Thinking 77.8% Jun 2025 Lab model card
21 Qwen2.5-VL-72B 75.7% Feb 2025 Lab model card
22 InternVL3.5-241B-A28B 75.2% Aug 2025 Lab model card
23 LongCat-Flash-Omni 74.8% Oct 2025 Lab model card
24 Qwen3.5-2B 74.5% Mar 2026 Lab model card
25 STEP3-VL-10B 74.4% Jan 2026 Lab model card
26 Cosmos3-Edge 73.3% May 2026 Lab model card
27 LFM2.5-VL-3B 73.1% Aug 2026 Lab model card
28 Nemotron Nano 12B v2 VL 72.2% Nov 2025 Lab model card
29 Cosmos3-Nano 72.2% May 2026 Lab model card
30 Gemini 1.5 Pro May 2024 70.4% May 2024 Lab model card
31 Qwen2.5-Omni-7B 70.3% Mar 2025 Lab model card
32 Kimi-VL-A3B-Thinking-2506 70% Jun 2025 Lab model card
33 LFM2-VL-1.6B 65.8% Nov 2025 Lab model card
34 Qwen3.5-0.8B 63.4% Mar 2026 Lab model card
35 North Micro Vision Instruct 62.2% Aug 2026 Lab model card
36 Gemma 3 27B 53.9% Mar 2025 Lab model card
37 Gemma 3 12B 52.2% Mar 2025 Lab model card
38 Gemma 3 4B 45.5% Mar 2025 Lab model card

8 models have held number one in labs’ own numbers since its release in April 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind Qwen ByteDance Seed
Lab claims Passed 85% in Feb 2026, by Seed2.0 Pro
Qwen3.8 Flash 88.5%

Source: Labs’ model cards

ByteDance Seed’s own number for Seed2.0 Pro passed 85% in February 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen3.8 Flash Seed2.0 Pro Qwen3-VL-235B-A22B Qwen2.5-VL-72B Gemini 1.5 Pro 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen3.8 Flash Qwen3-VL-235B-A22B

Source: Labs’ model cards

3 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
Qwen
ByteDance Seed
Moonshot AI
Cohere
Liquid AI
Meituan
DateLaunch postScoreAs the post writes it
Sep 18, 2026 Qwen3.8-Omni-Flash 87.7 RealWorldQA
Aug 26, 2026 Qwen3.8 Flash 88.5 Real-world perception RealWorldQA
Aug 14, 2026 Qwen 3.8 27B 85.9 Real-world perception RealWorldQA
Aug 12, 2026 LFM2.5-VL-3B 73.1 RealWorldQA
Aug 12, 2026 North Micro Vision Instruct 62.2 RealWorldQA
Aug 3, 2026 Qwen 3.8 Max 88 RealWorldQA
Jun 1, 2026 Qwen3.7-Plus 86.9 RealWorldQA
Apr 22, 2026 Qwen3.6 27B 84.1 RealWorldQA
Apr 15, 2026 Qwen 3.6 35B-A3B 85.3 RealWorldQA
Apr 2, 2026 Qwen 3.6 Plus 85.4 RealWorldQA
Mar 30, 2026 Qwen3.5-Omni-Plus 84.1 RealWorldQA
Mar 2, 2026 Qwen3.5-0.8B 63.4 RealWorldQA 63.4/61.6 (with CI / without CI)
Mar 2, 2026 Qwen3.5-2B 74.5 RealWorldQA 74.5/71.2 (with CI / without CI)
Mar 2, 2026 Qwen3.5-4B 79.5 RealWorldQA
Mar 2, 2026 Qwen3.5-9B 80.3 RealWorldQA
Feb 24, 2026 Qwen3.5-35B-A3B 84.1 RealWorldQA
Feb 24, 2026 Qwen3.5-27B 83.7 RealWorldQA
Feb 24, 2026 Qwen3.5-122B-A10B 85.1 RealWorldQA
Feb 16, 2026 Qwen3.5 397B-A17B 83.9 RealWorldQA
Oct 31, 2025 LongCat-Flash-Omni 74.8 RealWorldQA
Sep 23, 2025 Qwen3-VL-235B-A22B 81.3 RealWorldQA
Aug 12, 2025 LFM2-VL-1.6B 65.2 RealWorldQA
Jul 31, 2025 Command A Vision RealWorldQA (chart only)
Jun 25, 2025 Seed1.6-Thinking 77.8 RealWorldQA
Jun 21, 2025 Kimi-VL-A3B-Thinking-2506 70 RealWorldQA (Acc)
May 12, 2025 Seed1.5-VL 78.4 RealWorldQA
Mar 27, 2025 Qwen2.5-Omni-7B 70.3 RealWorldQAavg
Mar 12, 2025 Gemma 3 27B 53.9 RealWorldQA (PT)
May 14, 2024 Gemini 1.5 Pro 70.4 Realworld QA (test), 0-shot

19 system cards report it, from 9 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Sep 22, 2026 Qwen3.8-Omni-Flash technical report 87.7 RealWorldQA (Table 3) Capabilities
Jun 23, 2026 Seed2.1 Pro model card 86.7 RealWorldQA (Table 9) Capabilities
May 31, 2026 Cosmos3-Nano technical report 72.2 RealWorldQA, Cosmos3-Nano 8B column, Table 10 Capabilities
May 31, 2026 Cosmos3-Edge technical report 73.3 RealWorldQA, Cosmos3-Edge 2B column, Table 10 Capabilities
Apr 17, 2026 Qwen3.5-Omni-Plus technical report 84.1 RealWorldQA (Table 6) Capabilities
Feb 14, 2026 Seed2.0 Pro model card 86 RealWorldQA (Table 8) Capabilities
Jan 14, 2026 STEP3-VL-10B technical report 74.4 RealWorldQA, SeRe (Tables 1 and 3). PaCoRe 75.56 Capabilities
Nov 28, 2025 LFM2-VL-1.6B technical report 65.8 RealWorldQA, LFM2-VL-1.6B (Table 8) Capabilities
Nov 26, 2025 Qwen3-VL-235B-A22B technical report 81.3 RealWorldQA (Table 2) Capabilities
Nov 6, 2025 Nemotron Nano 12B v2 VL technical report 72.2 RealWorldQA, reasoning on (Table 3). Reasoning off 76.2 Capabilities
Oct 31, 2025 LongCat-Flash-Omni technical report 74.8 RealWorldQA (Table 5) Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 75.2 RealWorldQA (Table 5) Capabilities
Jun 23, 2025 Kimi-VL-A3B-Thinking-2506 technical report 70 RealWorldQA (Acc) (Table 5) Capabilities
May 11, 2025 Seed1.5-VL technical report 78.4 RealWorldQA, thinking (Table 6). Non-thinking 77.0 Capabilities
Mar 26, 2025 Qwen2.5-Omni-7B technical report 70.3 RealWorldQA avg (Table 5) Capabilities
Mar 25, 2025 Gemma 3 4B technical report 45.5 RealWorldQA, PT, 4-shot (Table 11) Capabilities
Mar 25, 2025 Gemma 3 27B technical report 53.9 RealWorldQA, PT, 4-shot (Table 11) Capabilities
Mar 25, 2025 Gemma 3 12B technical report 52.2 RealWorldQA, PT, 4-shot (Table 11) Capabilities
Feb 19, 2025 Qwen2.5-VL-72B technical report 75.7 RealWorldQA avg (Table 3) Capabilities

More in Vision

All 281 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%