Benchmark Almanac

K12Vista

Gemini2-thinking 57.4% tops its leaderboard Qwen2.5-VL-32B 55.4% leads the models we track
Not in model cards

Models solve Chinese primary and secondary school problems with text and images. The benchmark covers five subjects and includes multiple-choice, fill-in, and written answers.

Made by
Chong Li and colleagues
Released
June 2025
Kind
Other professions
Level
School
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Measures
Capability
Language
Chinese
In and out
Text, Images → Text
Interaction
Single turn
Items from
Exams and contests
Refresh
One fixed set
Run by
Anyone
Strongest model tested
o3-mini #124 on the capability index
Last model added
March 2025
Scores refreshed
November 20th, 2026
29
Entries in its paper, 26 models we track

Gemini2-thinking tops the leaderboard with 57.4% Qwen2.5-VL-32B leads with 55.4% among models released by March 2025

Best score per entry from the benchmark's paper, read on November 20th, 2026. A paper’s table doesn’t add newer models. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Qwen2.5-VL-32B, at 55.4%. This view shows only models we track, each at its best.

The newest model with a score here came out in March 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 Gemini2-thinking Step-by-step reasoning (Text + Image) 57.4%
  2. 2 Qwen2.5-VL-32B Direct inference (Text + Image) 55.4%
  3. 3 o3-mini Step-by-step reasoning (Text + Captions) 52.1%
  4. 4 Qwen2.5-VL-72B Direct inference (Text + Image) 51.4%
  5. 5 Gemini 2.0 Flash Feb 2025, Direct inference (Text + Image) 51.1%
  6. 6 QVQ-72B-Preview Direct inference (Text + Image) 49.5%
  7. 7 Deepseek-v3 (snapshot unspecified) Step-by-step reasoning (Text + Captions) 47.6%
  8. 8 InternVL2.5-78B-MPO Direct inference (Text + Image) 45.4%
  9. 9 o1-mini Direct inference (Text + Captions) 43.9%
  10. 10 InternVL2.5-38B-MPO Direct inference (Text + Image) 40.9%
  11. 11 InternVL2_5-78B Direct inference (Text + Image) 40.4%
  12. 12 Qwen2.5-VL-7B Direct inference (Text + Image) 39.8%
  13. 13 InternVL2.5-38B Direct inference (Text + Image) 36.5%
  14. 14 GPT-4o (snapshot unspecified) Direct inference (Text + Image) 35%
  15. 15 Qwen2-VL-72B-Instruct Direct inference (Text + Image) 34.5%
  16. 16 InternVL2.5-26B-MPO Direct inference (Text + Image) 33.6%
  17. 17 Qwen2.5-VL-3B Direct inference (Text + Image) 32%
  18. 18 LLaVA-OneVision 72B Direct inference (Text + Image) 31.6%
  19. 19 InternVL2.5-26B Direct inference (Text + Image) 30.6%
  20. 20 InternVL2.5-8B-MPO Direct inference (Text + Image) 29.9%
  21. 21 InternVL2.5-4B-MPO Direct inference (Text + Image) 29.3%
  22. 22 InternVL2-Llama3-76B Direct inference (Text + Image) 28.9%
  23. 23 InternVL2-40B Direct inference (Text + Image) 28.2%
  24. 24 InternVL2.5-8B Direct inference (Text + Image) 27.3%
  25. 25 InternVL2.5-4B Direct inference (Text + Image) 27.2%
  26. 26 MiniCPM-o-2_6 Direct inference (Text + Image) 26.9%
  27. 27 InternVL2-8B Direct inference (Text + Image) 25.7%
  28. 28 Qwen2-VL-7B-Instruct Direct inference (Text + Image) 25.7%
  29. 29 LLaVA-OneVision 7B Direct inference (Text + Image) 23.9%

Source: The K12Vista paper

  1. 1 Qwen2.5-VL-32B Direct inference (Text + Image) 55.4%
  2. 2 o3-mini Step-by-step reasoning (Text + Captions) 52.1%
  3. 3 Qwen2.5-VL-72B Direct inference (Text + Image) 51.4%
  4. 4 Gemini 2.0 Flash Feb 2025, Direct inference (Text + Image) 51.1%
  5. 5 QVQ-72B-Preview Direct inference (Text + Image) 49.5%
  6. 6 InternVL2.5-78B-MPO Direct inference (Text + Image) 45.4%
  7. 7 o1-mini Direct inference (Text + Captions) 43.9%
  8. 8 InternVL2.5-38B-MPO Direct inference (Text + Image) 40.9%
  9. 9 InternVL2_5-78B Direct inference (Text + Image) 40.4%
  10. 10 Qwen2.5-VL-7B Direct inference (Text + Image) 39.8%
  11. 11 InternVL2.5-38B Direct inference (Text + Image) 36.5%
  12. 12 Qwen2-VL-72B-Instruct Direct inference (Text + Image) 34.5%
  13. 13 InternVL2.5-26B-MPO Direct inference (Text + Image) 33.6%
  14. 14 Qwen2.5-VL-3B Direct inference (Text + Image) 32%
  15. 15 LLaVA-OneVision 72B Direct inference (Text + Image) 31.6%
  16. 16 InternVL2.5-26B Direct inference (Text + Image) 30.6%
  17. 17 InternVL2.5-8B-MPO Direct inference (Text + Image) 29.9%
  18. 18 InternVL2.5-4B-MPO Direct inference (Text + Image) 29.3%
  19. 19 InternVL2-Llama3-76B Direct inference (Text + Image) 28.9%
  20. 20 InternVL2-40B Direct inference (Text + Image) 28.2%
  21. 21 InternVL2.5-8B Direct inference (Text + Image) 27.3%
  22. 22 InternVL2.5-4B Direct inference (Text + Image) 27.2%
  23. 23 MiniCPM-o-2_6 Direct inference (Text + Image) 26.9%
  24. 24 InternVL2-8B Direct inference (Text + Image) 25.7%
  25. 25 Qwen2-VL-7B-Instruct Direct inference (Text + Image) 25.7%
  26. 26 LLaVA-OneVision 7B Direct inference (Text + Image) 23.9%

Source: The K12Vista paper

#EntryScoreSettingDateSource
1 Gemini2-thinking 57.4% Step-by-step reasoning (Text + Image) Official leaderboard
2 Qwen2.5-VL-32B 55.4% Direct inference (Text + Image) Mar 2025 Official leaderboard
3 o3-mini 52.1% Step-by-step reasoning (Text + Captions) Jan 2025 Official leaderboard
4 Qwen2.5-VL-72B 51.4% Direct inference (Text + Image) Jan 2025 Official leaderboard
5 Gemini 2.0 Flash Feb 2025 51.1% Direct inference (Text + Image) Feb 2025 Official leaderboard
6 QVQ-72B-Preview 49.5% Direct inference (Text + Image) Dec 2024 Official leaderboard
7 Deepseek-v3 (snapshot unspecified) 47.6% Step-by-step reasoning (Text + Captions) Official leaderboard
8 InternVL2.5-78B-MPO 45.4% Direct inference (Text + Image) Dec 2024 Official leaderboard
9 o1-mini 43.9% Direct inference (Text + Captions) Sep 2024 Official leaderboard
10 InternVL2.5-38B-MPO 40.9% Direct inference (Text + Image) Dec 2024 Official leaderboard
11 InternVL2_5-78B 40.4% Direct inference (Text + Image) Dec 2024 Official leaderboard
12 Qwen2.5-VL-7B 39.8% Direct inference (Text + Image) Jan 2025 Official leaderboard
13 InternVL2.5-38B 36.5% Direct inference (Text + Image) Dec 2024 Official leaderboard
14 GPT-4o (snapshot unspecified) 35% Direct inference (Text + Image) Official leaderboard
15 Qwen2-VL-72B-Instruct 34.5% Direct inference (Text + Image) Aug 2024 Official leaderboard
16 InternVL2.5-26B-MPO 33.6% Direct inference (Text + Image) Dec 2024 Official leaderboard
17 Qwen2.5-VL-3B 32% Direct inference (Text + Image) Jan 2025 Official leaderboard
18 LLaVA-OneVision 72B 31.6% Direct inference (Text + Image) Aug 2024 Official leaderboard
19 InternVL2.5-26B 30.6% Direct inference (Text + Image) Dec 2024 Official leaderboard
20 InternVL2.5-8B-MPO 29.9% Direct inference (Text + Image) Dec 2024 Official leaderboard
21 InternVL2.5-4B-MPO 29.3% Direct inference (Text + Image) Dec 2024 Official leaderboard
22 InternVL2-Llama3-76B 28.9% Direct inference (Text + Image) Jul 2024 Official leaderboard
23 InternVL2-40B 28.2% Direct inference (Text + Image) Jul 2024 Official leaderboard
24 InternVL2.5-8B 27.3% Direct inference (Text + Image) Dec 2024 Official leaderboard
25 InternVL2.5-4B 27.2% Direct inference (Text + Image) Dec 2024 Official leaderboard
26 MiniCPM-o-2_6 26.9% Direct inference (Text + Image) Jan 2025 Official leaderboard
27 InternVL2-8B 25.7% Direct inference (Text + Image) Jul 2024 Official leaderboard
28 Qwen2-VL-7B-Instruct 25.7% Direct inference (Text + Image) Aug 2024 Official leaderboard
29 LLaVA-OneVision 7B 23.9% Direct inference (Text + Image) Aug 2024 Official leaderboard
#ModelScoreSettingDateSource
1 Qwen2.5-VL-32B 55.4% Direct inference (Text + Image) Mar 2025 Official leaderboard
2 o3-mini 52.1% Step-by-step reasoning (Text + Captions) Jan 2025 Official leaderboard
3 Qwen2.5-VL-72B 51.4% Direct inference (Text + Image) Jan 2025 Official leaderboard
4 Gemini 2.0 Flash Feb 2025 51.1% Direct inference (Text + Image) Feb 2025 Official leaderboard
5 QVQ-72B-Preview 49.5% Direct inference (Text + Image) Dec 2024 Official leaderboard
6 InternVL2.5-78B-MPO 45.4% Direct inference (Text + Image) Dec 2024 Official leaderboard
7 o1-mini 43.9% Direct inference (Text + Captions) Sep 2024 Official leaderboard
8 InternVL2.5-38B-MPO 40.9% Direct inference (Text + Image) Dec 2024 Official leaderboard
9 InternVL2_5-78B 40.4% Direct inference (Text + Image) Dec 2024 Official leaderboard
10 Qwen2.5-VL-7B 39.8% Direct inference (Text + Image) Jan 2025 Official leaderboard
11 InternVL2.5-38B 36.5% Direct inference (Text + Image) Dec 2024 Official leaderboard
12 Qwen2-VL-72B-Instruct 34.5% Direct inference (Text + Image) Aug 2024 Official leaderboard
13 InternVL2.5-26B-MPO 33.6% Direct inference (Text + Image) Dec 2024 Official leaderboard
14 Qwen2.5-VL-3B 32% Direct inference (Text + Image) Jan 2025 Official leaderboard
15 LLaVA-OneVision 72B 31.6% Direct inference (Text + Image) Aug 2024 Official leaderboard
16 InternVL2.5-26B 30.6% Direct inference (Text + Image) Dec 2024 Official leaderboard
17 InternVL2.5-8B-MPO 29.9% Direct inference (Text + Image) Dec 2024 Official leaderboard
18 InternVL2.5-4B-MPO 29.3% Direct inference (Text + Image) Dec 2024 Official leaderboard
19 InternVL2-Llama3-76B 28.9% Direct inference (Text + Image) Jul 2024 Official leaderboard
20 InternVL2-40B 28.2% Direct inference (Text + Image) Jul 2024 Official leaderboard
21 InternVL2.5-8B 27.3% Direct inference (Text + Image) Dec 2024 Official leaderboard
22 InternVL2.5-4B 27.2% Direct inference (Text + Image) Dec 2024 Official leaderboard
23 MiniCPM-o-2_6 26.9% Direct inference (Text + Image) Jan 2025 Official leaderboard
24 InternVL2-8B 25.7% Direct inference (Text + Image) Jul 2024 Official leaderboard
25 Qwen2-VL-7B-Instruct 25.7% Direct inference (Text + Image) Aug 2024 Official leaderboard
26 LLaVA-OneVision 7B 23.9% Direct inference (Text + Image) Aug 2024 Official leaderboard

Rankings by part

We read 3 parts from the benchmark’s paper: Primary, Middle and High. Each ranks models on one part of the test. Qwen2.5-VL-32B leads every one of them. Arrows show how far a model moved from its overall rank among the models we track.

Primary

26 models we track have a score here.

  1. 1 Qwen2.5-VL-32B 59%
  2. 2 o3-mini 57.5%
  3. 3 Gemini 2.0 Flash Feb 2025 ↑1 56.7%
  4. 4 o1-mini ↑3 56.2%
  5. 5 Qwen2.5-VL-72B ↓2 55.3%
  6. 6 QVQ-72B-Preview ↓1 55%
  7. 7 InternVL2.5-78B-MPO ↓1 50.3%
  8. 8 InternVL2_5-78B ↑1 47.3%
  9. 9 InternVL2.5-38B-MPO ↓1 47%
  10. 10 InternVL2.5-38B ↑1 42.3%

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Qwen2.5-VL-32B
Step-by-ste…
55.4%
o3-mini
Direct infe…
52.1%
Qwen2.5-VL-72B
Step-by-ste…
51.4%
Gemini 2.0 Flash
Step-by-ste… Direct infe…
51.1%
QVQ-72B-Preview
Step-by-ste… Direct infe…
49.5%
InternVL2.5-78B-MPO
Step-by-ste…
45.4%
o1-mini
Step-by-ste…
43.9%
InternVL2.5-38B-MPO
Step-by-ste…
40.9%
InternVL2_5-78B
Step-by-ste…
40.4%
Qwen2.5-VL-7B
Step-by-ste… Direct infe…
39.8%
InternVL2.5-38B
Step-by-ste… Direct infe…
36.5%
Qwen2-VL-72B-Instruct
Step-by-ste… Direct infe…
34.5%

Source: The K12Vista paper

ModelEffortHarnessScoreCost per taskOutput tokens
Qwen2.5-VL-32B Direct inference (Text + Image) 55.4%
Qwen2.5-VL-32B Step-by-step reasoning (Text + Image) 53.4%
o3-mini Step-by-step reasoning (Text + Captions) 52.1%
Qwen2.5-VL-72B Direct inference (Text + Image) 51.4%
o3-mini Direct inference (Text + Captions) 51.1%
Gemini 2.0 Flash Feb 2025 Direct inference (Text + Image) 51.1%
Qwen2.5-VL-72B Step-by-step reasoning (Text + Image) 49.9%
QVQ-72B-Preview Direct inference (Text + Image) 49.5%
Gemini 2.0 Flash Feb 2025 Step-by-step reasoning (Text + Image) 47.3%
QVQ-72B-Preview Step-by-step reasoning (Text + Image) 46.3%
InternVL2.5-78B-MPO Direct inference (Text + Image) 45.4%
o1-mini Direct inference (Text + Captions) 43.9%
o1-mini Step-by-step reasoning (Text + Captions) 43.5%
InternVL2.5-78B-MPO Step-by-step reasoning (Text + Image) 42.8%
InternVL2.5-38B-MPO Direct inference (Text + Image) 40.9%
InternVL2_5-78B Direct inference (Text + Image) 40.4%
Qwen2.5-VL-7B Direct inference (Text + Image) 39.8%
InternVL2.5-38B-MPO Step-by-step reasoning (Text + Image) 38.5%
InternVL2_5-78B Step-by-step reasoning (Text + Image) 38.5%
InternVL2.5-38B Direct inference (Text + Image) 36.5%
Qwen2-VL-72B-Instruct Direct inference (Text + Image) 34.5%
InternVL2.5-26B-MPO Direct inference (Text + Image) 33.6%
InternVL2.5-38B Step-by-step reasoning (Text + Image) 33.4%
Qwen2.5-VL-3B Direct inference (Text + Image) 32%
LLaVA-OneVision 72B Direct inference (Text + Image) 31.6%
InternVL2.5-26B Direct inference (Text + Image) 30.6%
InternVL2.5-8B-MPO Direct inference (Text + Image) 29.9%
InternVL2.5-26B-MPO Step-by-step reasoning (Text + Image) 29.9%
InternVL2.5-4B-MPO Direct inference (Text + Image) 29.3%
InternVL2-Llama3-76B Direct inference (Text + Image) 28.9%
InternVL2-40B Direct inference (Text + Image) 28.2%
InternVL2.5-8B Direct inference (Text + Image) 27.3%
InternVL2.5-4B Direct inference (Text + Image) 27.2%
Qwen2.5-VL-7B Step-by-step reasoning (Text + Image) 27.2%
LLaVA-OneVision 72B Step-by-step reasoning (Text + Image) 27.1%
InternVL2.5-8B-MPO Step-by-step reasoning (Text + Image) 26.9%
MiniCPM-o-2_6 Direct inference (Text + Image) 26.9%
InternVL2-8B Direct inference (Text + Image) 25.7%
Qwen2-VL-72B-Instruct Step-by-step reasoning (Text + Image) 25.7%
Qwen2-VL-7B-Instruct Direct inference (Text + Image) 25.7%
InternVL2.5-4B-MPO Step-by-step reasoning (Text + Image) 25.1%
InternVL2.5-26B Step-by-step reasoning (Text + Image) 24.7%
LLaVA-OneVision 7B Direct inference (Text + Image) 23.9%
InternVL2.5-8B Step-by-step reasoning (Text + Image) 22.7%
InternVL2-Llama3-76B Step-by-step reasoning (Text + Image) 20.9%
InternVL2.5-4B Step-by-step reasoning (Text + Image) 20.7%
MiniCPM-o-2_6 Step-by-step reasoning (Text + Image) 19.1%
Qwen2.5-VL-3B Step-by-step reasoning (Text + Image) 16.4%
InternVL2-40B Step-by-step reasoning (Text + Image) 13.9%
InternVL2-8B Step-by-step reasoning (Text + Image) 13%
Qwen2-VL-7B-Instruct Step-by-step reasoning (Text + Image) 12.6%
LLaVA-OneVision 7B Step-by-step reasoning (Text + Image) 11.9%

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Qwen2.5-VL-32B
Step-by-ste…
55.4%
o3-mini
Direct infe…
52.1%
Qwen2.5-VL-72B
Step-by-ste…
51.4%
Gemini 2.0 Flash
Step-by-ste… Direct infe…
51.1%
QVQ-72B-Preview
Step-by-ste… Direct infe…
49.5%
InternVL2.5-78B-MPO
Step-by-ste…
45.4%
o1-mini
Step-by-ste…
43.9%
InternVL2.5-38B-MPO
Step-by-ste…
40.9%
InternVL2_5-78B
Step-by-ste…
40.4%
Qwen2.5-VL-7B
Step-by-ste… Direct infe…
39.8%
InternVL2.5-38B
Step-by-ste… Direct infe…
36.5%
Qwen2-VL-72B-Instruct
Step-by-ste… Direct infe…
34.5%

Source: The K12Vista paper

ModelEffortHarnessScoreCost per taskOutput tokens
Qwen2.5-VL-32B Direct inference (Text + Image) 55.4%
Qwen2.5-VL-32B Step-by-step reasoning (Text + Image) 53.4%
o3-mini Step-by-step reasoning (Text + Captions) 52.1%
Qwen2.5-VL-72B Direct inference (Text + Image) 51.4%
o3-mini Direct inference (Text + Captions) 51.1%
Gemini 2.0 Flash Feb 2025 Direct inference (Text + Image) 51.1%
Qwen2.5-VL-72B Step-by-step reasoning (Text + Image) 49.9%
QVQ-72B-Preview Direct inference (Text + Image) 49.5%
Gemini 2.0 Flash Feb 2025 Step-by-step reasoning (Text + Image) 47.3%
QVQ-72B-Preview Step-by-step reasoning (Text + Image) 46.3%
InternVL2.5-78B-MPO Direct inference (Text + Image) 45.4%
o1-mini Direct inference (Text + Captions) 43.9%
o1-mini Step-by-step reasoning (Text + Captions) 43.5%
InternVL2.5-78B-MPO Step-by-step reasoning (Text + Image) 42.8%
InternVL2.5-38B-MPO Direct inference (Text + Image) 40.9%
InternVL2_5-78B Direct inference (Text + Image) 40.4%
Qwen2.5-VL-7B Direct inference (Text + Image) 39.8%
InternVL2.5-38B-MPO Step-by-step reasoning (Text + Image) 38.5%
InternVL2_5-78B Step-by-step reasoning (Text + Image) 38.5%
InternVL2.5-38B Direct inference (Text + Image) 36.5%
Qwen2-VL-72B-Instruct Direct inference (Text + Image) 34.5%
InternVL2.5-26B-MPO Direct inference (Text + Image) 33.6%
InternVL2.5-38B Step-by-step reasoning (Text + Image) 33.4%
Qwen2.5-VL-3B Direct inference (Text + Image) 32%
LLaVA-OneVision 72B Direct inference (Text + Image) 31.6%
InternVL2.5-26B Direct inference (Text + Image) 30.6%
InternVL2.5-8B-MPO Direct inference (Text + Image) 29.9%
InternVL2.5-26B-MPO Step-by-step reasoning (Text + Image) 29.9%
InternVL2.5-4B-MPO Direct inference (Text + Image) 29.3%
InternVL2-Llama3-76B Direct inference (Text + Image) 28.9%
InternVL2-40B Direct inference (Text + Image) 28.2%
InternVL2.5-8B Direct inference (Text + Image) 27.3%
InternVL2.5-4B Direct inference (Text + Image) 27.2%
Qwen2.5-VL-7B Step-by-step reasoning (Text + Image) 27.2%
LLaVA-OneVision 72B Step-by-step reasoning (Text + Image) 27.1%
InternVL2.5-8B-MPO Step-by-step reasoning (Text + Image) 26.9%
MiniCPM-o-2_6 Direct inference (Text + Image) 26.9%
InternVL2-8B Direct inference (Text + Image) 25.7%
Qwen2-VL-72B-Instruct Step-by-step reasoning (Text + Image) 25.7%
Qwen2-VL-7B-Instruct Direct inference (Text + Image) 25.7%
InternVL2.5-4B-MPO Step-by-step reasoning (Text + Image) 25.1%
InternVL2.5-26B Step-by-step reasoning (Text + Image) 24.7%
LLaVA-OneVision 7B Direct inference (Text + Image) 23.9%
InternVL2.5-8B Step-by-step reasoning (Text + Image) 22.7%
InternVL2-Llama3-76B Step-by-step reasoning (Text + Image) 20.9%
InternVL2.5-4B Step-by-step reasoning (Text + Image) 20.7%
MiniCPM-o-2_6 Step-by-step reasoning (Text + Image) 19.1%
Qwen2.5-VL-3B Step-by-step reasoning (Text + Image) 16.4%
InternVL2-40B Step-by-step reasoning (Text + Image) 13.9%
InternVL2-8B Step-by-step reasoning (Text + Image) 13%
Qwen2-VL-7B-Instruct Step-by-step reasoning (Text + Image) 12.6%
LLaVA-OneVision 7B Step-by-step reasoning (Text + Image) 11.9%

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Qwen
Benchmark paper Not tested since Jun 2025

Source: The K12Vista paper

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
In the benchmark’s paper Best so far
0% 20% 40% 60% 80% 100% 2025 Near the ceiling (85%) Qwen2.5-VL-32B Qwen2.5-VL-72B InternVL2.5-78B-MPO o1-mini Qwen2-VL-72B-Instruct InternVL2-Llama3-76B 0% 20% 40% 60% 80% 100% 2025 Near the ceiling (85%) Qwen2.5-VL-32B InternVL2.5-78B-MPO

Source: Official K12Vista leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Law, finance and medicine

All 637 in Law, finance and medicine
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • FinSearchComp In use Agents search the web to fetch live financial data, look up history, and run multi-step investigations, across 635 expert-written questions. Scored by answer accuracy. Ling-3.1-flash 81.7%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. o1-preview 92.7%
  • Harvey LAB-AA v1.1 In use Artificial Analysis's run of Harvey's Legal Agent Benchmark on Harvey's newer private set of 120 tasks across 24 practice areas. A deliverable with a material hallucination scores zero, and the headline is the share of judges who find every rubric criterion passed. GPT-6.2 Astra 15%

Data audit

9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 20 months ago, in March 2025, for Qwen2.5-VL-32B. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
  • Pass Independent runs Scores from the benchmark’s paper (29 entries).
  • Pass Models scored 26 models we track, plus 3 other entries on its leaderboard.
  • Warning Newest score The newest came 20 months ago, in March 2025, for Qwen2.5-VL-32B.
  • Note Frontier models None of the top 10 has a score. The strongest tested is o3-mini (#124).
  • Pass Headroom The best is 55.4%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on arxiv.org: 52 rows for models we track and 3 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check