Benchmark Almanac

SLAKE

Qwen 3.8 Max 90.8% leads since August 2026
Fading

Models answer English and Chinese questions about radiology images of the head, neck, chest, abdomen, and pelvis. Scoring is accuracy.

The authors say the released data was cleaned and may differ slightly from the paper.

Released
February 2021
Kind
Medicine
Level
University
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Strongest model tested
Qwen 3.8 Max #29 on the capability index
Last in a launch post
August 2026, dropped after it saturated
Last model added
August 2026

In the authors’ words

Medical visual question answering (Med-VQA) has tremendous potential in healthcare. However, the development of this technology is hindered by the lacking of publicly-available and high-quality labeled datasets for training and evaluation. In this paper, we present a large bilingual dataset, SLAKE, with comprehensive semantic labels annotated by experienced physicians and a new structural medical knowledge base for Med-VQA. Besides, SLAKE includes richer modalities and covers more human body parts than the currently available dataset. We show that SLAKE can be used to facilitate the development and evaluation of Med-VQA systems. The dataset can be downloaded from http://www.med-vqa.com/slake.

Abstract of SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering (arXiv:2102.09542, CC0)
10
Models scored in lab cards
10
Launch posts list it, from 1 lab
1
System card reports it, from 1 lab

Qwen 3.8 Max leads with 90.8% among models released by August 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in August 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 90.8 Qwen 3.8 Max
  2. 84.7 Qwen3.5-Omni-Plus
  3. 81.6 Qwen3.5-122B-A10B
  4. 80.0 Qwen3.5-27B
  5. 79.9 Qwen3.5 397B-A17B
  6. 79.0 Qwen3.5-9B
  7. 78.7 Qwen3.5-35B-A3B
  8. 76.1 Qwen3.5-4B
  9. 74.4 Qwen3.5-2B
  10. 62.6 Qwen3.5-0.8B
  1. 1 Qwen 3.8 Max 90.8%
  2. 2 Qwen3.5-Omni-Plus 84.7%
  3. 3 Qwen3.5-122B-A10B 81.6%
  4. 4 Qwen3.5-27B 80%
  5. 5 Qwen3.5 397B-A17B 79.9%
  6. 6 Qwen3.5-9B 79%
  7. 7 Qwen3.5-35B-A3B 78.7%
  8. 8 Qwen3.5-4B 76.1%
  9. 9 Qwen3.5-2B 74.4%
  10. 10 Qwen3.5-0.8B 62.6%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 Qwen 3.8 Max 90.8% Aug 2026 Lab model card
2 Qwen3.5-Omni-Plus 84.7% Mar 2026 Lab model card
3 Qwen3.5-122B-A10B 81.6% Feb 2026 Lab model card
4 Qwen3.5-27B 80% Feb 2026 Lab model card
5 Qwen3.5 397B-A17B 79.9% Feb 2026 Lab model card
6 Qwen3.5-9B 79% Mar 2026 Lab model card
7 Qwen3.5-35B-A3B 78.7% Feb 2026 Lab model card
8 Qwen3.5-4B 76.1% Mar 2026 Lab model card
9 Qwen3.5-2B 74.4% Mar 2026 Lab model card
10 Qwen3.5-0.8B 62.6% Mar 2026 Lab model card

4 models have held number one in labs’ own numbers since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Qwen
Lab claims Passed 85% in Aug 2026, by Qwen 3.8 Max
Qwen 3.8 Max 90.8%

Source: Labs’ model cards

Qwen’s own number for Qwen 3.8 Max passed 85% in August 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Qwen 3.8 Max Qwen3.5-Omni-Plus Qwen3.5 397B-A17B 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Qwen 3.8 Max Qwen3.5-122B-A10B

Source: Labs’ model cards

Labs headline it less often now, last in August 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Qwen
DateLaunch postScoreAs the post writes it
Aug 3, 2026 Qwen 3.8 Max 90.8 SLAKE
Mar 30, 2026 Qwen3.5-Omni-Plus 84.7 SLAKE
Mar 2, 2026 Qwen3.5-0.8B 62.6 SLAKE 62.6/59.5 (with CI / without CI)
Mar 2, 2026 Qwen3.5-2B 74.4 SLAKE 74.4/67.5 (with CI / without CI)
Mar 2, 2026 Qwen3.5-4B 76.1 SLAKE
Mar 2, 2026 Qwen3.5-9B 79 SLAKE
Feb 24, 2026 Qwen3.5-35B-A3B 78.7 SLAKE
Feb 24, 2026 Qwen3.5-27B 80 SLAKE
Feb 24, 2026 Qwen3.5-122B-A10B 81.6 SLAKE
Feb 16, 2026 Qwen3.5 397B-A17B 79.9 SLAKE

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Apr 17, 2026 Qwen3.5-Omni-Plus technical report 84.7 SLAKE (Table 6) Health

More in Law, finance and medicine

All 179 in Law, finance and medicine
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • FinSearchComp In use Agents search the web to fetch live financial data, look up history, and run multi-step investigations, across 635 expert-written questions. Scored by answer accuracy. Ling-3.1-flash 81.7%
  • KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Solar Pro 4 77.5%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Solar Pro 4 93.2%