Benchmark Almanac

ODinW-13

Qwen 3.6 Plus 51.8% leads since April 2026
Fading

Models detect objects in 13 varied real-world datasets, such as aquarium and aerial images, from text category names. Scored by mean average precision.

A 13-dataset subset of the 35-task ODinW benchmark in ELEVATER.

Released
December 2021
Kind
Detection and captioning
Level
Everyday
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Strongest model tested
Qwen 3.6 Plus #74 on the capability index
Last in a launch post
June 2026, dropped before it saturated
Last model added
June 2026
13
Models scored in lab cards
12
Launch posts list it, from 1 lab
4
System cards report it, from 1 lab

Qwen 3.6 Plus leads with 51.8% among models released by June 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in June 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

13 models. Scroll sideways for the rest.

  1. 51.8 Qwen 3.6 Plus
  2. 51.1 Qwen3.7-Plus
  3. 50.8 Qwen 3.6 35B-A3B
  4. 49.5 Qwen3.5-Omni-Plus
  5. 47.0 Qwen3.5 397B-A17B
  6. 44.5 Qwen3.5-122B-A10B
  7. 43.2 Qwen3-VL-235B-A22B
  8. 43.1 Qwen2.5-VL-72B
  9. 42.6 Qwen3.5-35B-A3B
  10. 42.4 Qwen2.5-Omni-7B
  11. 41.1 Qwen3.5-27B
  12. 35.9 Qwen3.5-2B
  13. 31.6 Qwen3.5-0.8B
  1. 1 Qwen 3.6 Plus 51.8%
  2. 2 Qwen3.7-Plus 51.1%
  3. 3 Qwen 3.6 35B-A3B 50.8%
  4. 4 Qwen3.5-Omni-Plus 49.5%
  5. 5 Qwen3.5 397B-A17B 47%
  6. 6 Qwen3.5-122B-A10B 44.5%
  7. 7 Qwen3-VL-235B-A22B 43.2%
  8. 8 Qwen2.5-VL-72B 43.1%
  9. 9 Qwen3.5-35B-A3B 42.6%
  10. 10 Qwen2.5-Omni-7B 42.4%
  11. 11 Qwen3.5-27B 41.1%
  12. 12 Qwen3.5-2B 35.9%
  13. 13 Qwen3.5-0.8B 31.6%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 Qwen 3.6 Plus 51.8% Apr 2026 Lab model card
2 Qwen3.7-Plus 51.1% Jun 2026 Lab model card
3 Qwen 3.6 35B-A3B 50.8% Apr 2026 Lab model card
4 Qwen3.5-Omni-Plus 49.5% Mar 2026 Lab model card
5 Qwen3.5 397B-A17B 47% Feb 2026 Lab model card
6 Qwen3.5-122B-A10B 44.5% Feb 2026 Lab model card
7 Qwen3-VL-235B-A22B 43.2% Sep 2025 Lab model card
8 Qwen2.5-VL-72B 43.1% Feb 2025 Lab model card
9 Qwen3.5-35B-A3B 42.6% Feb 2026 Lab model card
10 Qwen2.5-Omni-7B 42.4% Mar 2025 Lab model card
11 Qwen3.5-27B 41.1% Feb 2026 Lab model card
12 Qwen3.5-2B 35.9% Mar 2026 Lab model card
13 Qwen3.5-0.8B 31.6% Mar 2026 Lab model card

5 models have held number one in labs’ own numbers since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Qwen
Lab claims Not tested since Jun 2026
Qwen 3.6 Plus 51.8%

Source: Labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Qwen 3.6 Plus Qwen3.5 397B-A17B Qwen3-VL-235B-A22B Qwen2.5-VL-72B 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Qwen 3.6 Plus Qwen3-VL-235B-A22B

Source: Labs’ model cards

Labs headline it less often now, last in June 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Qwen
DateLaunch postScoreAs the post writes it
Jun 1, 2026 Qwen3.7-Plus 51.1 ODinW13
Apr 15, 2026 Qwen 3.6 35B-A3B 50.8 ODInW13
Apr 2, 2026 Qwen 3.6 Plus 51.8 ODinW13
Mar 30, 2026 Qwen3.5-Omni-Plus 49.5 ODInW13
Mar 2, 2026 Qwen3.5-0.8B 31.6 ODInW13 31.6/33.2 (with CI / without CI)
Mar 2, 2026 Qwen3.5-2B 35.9 ODInW13 35.9/40.5 (with CI / without CI)
Feb 24, 2026 Qwen3.5-35B-A3B 42.6 ODInW13
Feb 24, 2026 Qwen3.5-27B 41.1 ODInW13
Feb 24, 2026 Qwen3.5-122B-A10B 44.5 ODInW13
Feb 16, 2026 Qwen3.5 397B-A17B 47 ODInW13
Sep 23, 2025 Qwen3-VL-235B-A22B 43.2 ODinW13
Mar 27, 2025 Qwen2.5-Omni-7B 42.4 ODinW

4 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Apr 17, 2026 Qwen3.5-Omni-Plus technical report 49.5 ODInW13 (Table 6) Capabilities
Nov 26, 2025 Qwen3-VL-235B-A22B technical report 43.2 ODinW-13 (Table 2) Capabilities
Mar 26, 2025 Qwen2.5-Omni-7B technical report 42.2 ODinW, mAP (Table 6 and text) Capabilities
Feb 19, 2025 Qwen2.5-VL-72B technical report 43.1 ODinW, mAP (Table 6) Capabilities

More in Vision

All 281 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%