Benchmark Almanac

Open LMM Reasoning Leaderboard

Doubao-1.5-pro 61.6 leads since January 2025
Not in model cards

Models solve multimodal reasoning tasks. OpenCompass ranks the models using the component results.

The leaderboard aggregates results from component evaluations.

Made by
OpenCompass
Released
December 2024
Kind
Composite indices
Level
University
Scored
Settled by events
Answers
Short answer
Tasks
All public
Measures
Capability
In and out
Text, Images → Text
Interaction
Single turn
Items from
Another benchmark
Refresh
One fixed set
Run by
Anyone
In views
Vision
Strongest model tested
Claude 3.7 Sonnet #119 on the capability index
Last model added
April 2025
Scores refreshed
November 20th, 2026
82
Entries on its own leaderboard, 61 models we track

Doubao-1.5-pro leads with 61.6 among models released by April 2025 Doubao-1.5-pro leads with 61.6 among models released by April 2025

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

The newest model with a score here came out in April 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

82 entries. Scroll sideways for the rest.

  1. 61.6 Doubao-1.5-pro
  2. 56.6 Gemini 2.0 Pro
  3. 54.8 ChatGPT-4o-latest
  4. 54.1 GPT-4.1
  5. 51.0 InternVL3-78B
  6. 50.6 Gemini 2.0 Flash
  7. 50.4 Claude 3.7 Sonnet
  8. 50.3 InternVL3-38B
  9. 50.2 Qwen2.5-VL-72B
  10. 50.0 Gemini 1.5 Pro
  11. 49.1 GLM-4v-Plus-20250111
  12. 47.9 Claude 3.5 Sonnet
  13. 47.9 Ovis2-34B
  14. 46.9 QVQ-72B-Preview
  15. 46.0 InternVL3-14B
  16. 46.0 GPT-4.1 mini
  17. 45.4 Step-1o
  18. 44.7 Ovis2-16B
  19. 44.3 InternVL2.5-78B-MPO
  20. 44.2 GPT-4o (1120, detail-high)
  21. 43.6 Taichu-VLR-7B
  22. 43.2 InternVL2.5-38B-MPO
  23. 42.5 VLAA-Thinker-Qwen2.5VL-7B
  24. 41.9 InternVL2.5-38B
  25. 41.7 InternVL2_5-78B
  26. 41.4 SenseNova
  27. 41.4 InternVL3-8B
  28. 40.8 InternVL3-9B
  29. 40.6 Gemma 3 27B
  30. 40.1 Qwen2.5-VL-7B
  31. 38.7 Qwen2-VL-72B
  32. 38.0 moonshot-v1-8k
  33. 37.8 Ovis2-8B
  34. 37.3 InternVL2.5-8B-BoN-8
  35. 37.0 Ovis1.6-Gemma2-27B
  36. 37.0 Grok 2 Vision (1212)
  37. 36.6 MUG-U-7B
  38. 36.0 SAIL-VL-1.6-8B
  39. 35.8 Kimi-VL-A3B-Instruct
  40. 35.4 VLAA-Thinker-Qwen2.5VL-3B
  41. 34.8 SAIL-VL-1.5-8B
  42. 34.7 LLaVA-OneVision-72B (SI)
  43. 34.4 InternVL2-Llama3-76B
  44. 34.2 GPT-4.1 nano
  45. 33.9 Gemma 3 12B
  46. 33.7 NVLM-D-72B
  47. 33.6 Taichu-VLR-3B
  48. 33.6 MiniCPM-o-2_6
  49. 33.4 VLM-R1-3B-Math-0305
  50. 33.3 Ovis2-4B
  51. 33.0 Kimi-VL-A3B-Thinking
  52. 32.9 URSA-8B
  53. 32.9 InternVL2.5-26B
  54. 31.8 Qwen2.5-VL-3B
  55. 31.4 POINTS1.5-Qwen2.5-7B
  56. 29.8 Yi-Vision
  57. 29.5 VITA-1.5
  58. 29.1 InternVL3-2B
  59. 28.8 Qwen2-VL-7B-Instruct
  60. 27.9 SAIL-VL-1.5-2B
  61. 27.6 Ovis2-2B
  62. 27.1 Gemma 3 4B
  63. 27.0 LLaVA-CoT
  64. 26.9 InternVL2-8B
  65. 26.6 LLaVA-OneVision 7B (SI)
  66. 26.4 InternVL2-26B
  67. 25.8 InternLM-XComposer2.5-7B
  68. 25.2 SAIL-VL-2B
  69. 24.6 InternVL-Chat-V1-5
  70. 24.3 Aquila-VL-2B
  71. 23.0 Ovis2-1B
  72. 22.5 GLM-4V-9B
  73. 21.7 Llama 3.2 11B
  74. 21.7 InternVL2.5-2B-MPO
  75. 21.3 InternVL3-1B
  76. 21.2 InternVL2.5-2B
  77. 20.5 Qwen2-VL-2B-Instruct
  78. 19.1 Janus-Pro-7B
  79. 18.5 VILA1.5-13B
  80. 18.4 SmolVLM-Instruct
  81. 13.3 LLaVA-v1.5-13B
  82. 12.1 llava-v1.5-7b
  1. 1 Doubao-1.5-pro 61.6
  2. 2 Gemini 2.0 Pro 56.6
  3. 3 ChatGPT-4o-latest 54.8
  4. 4 GPT-4.1 54.1
  5. 5 InternVL3-78B 51.0
  6. 6 Gemini 2.0 Flash Feb 2025 50.6
  7. 7 Claude 3.7 Sonnet 50.4
  8. 8 InternVL3-38B 50.3
  9. 9 Qwen2.5-VL-72B 50.2
  10. 10 Gemini 1.5 Pro Sep 2024 50.0
  11. 11 GLM-4v-Plus-20250111 49.1
  12. 12 Claude 3.5 Sonnet 47.9
  13. 13 Ovis2-34B 47.9
  14. 14 QVQ-72B-Preview 46.9
  15. 15 InternVL3-14B 46.0
  16. 16 GPT-4.1 mini 46.0
  17. 17 Step-1o 45.4
  18. 18 Ovis2-16B 44.7
  19. 19 InternVL2.5-78B-MPO 44.3
  20. 20 GPT-4o (1120, detail-high) 44.2
  21. 21 Taichu-VLR-7B 43.6
  22. 22 InternVL2.5-38B-MPO 43.2
  23. 23 VLAA-Thinker-Qwen2.5VL-7B 42.5
  24. 24 InternVL2.5-38B 41.9
  25. 25 InternVL2_5-78B 41.7
  26. 26 SenseNova 41.4
  27. 27 InternVL3-8B 41.4
  28. 28 InternVL3-9B 40.8
  29. 29 Gemma 3 27B 40.6
  30. 30 Qwen2.5-VL-7B 40.1
  31. 31 Qwen2-VL-72B 38.7
  32. 32 moonshot-v1-8k 38.0
  33. 33 Ovis2-8B 37.8
  34. 34 InternVL2.5-8B-BoN-8 37.3
  35. 35 Ovis1.6-Gemma2-27B 37.0
  36. 36 Grok 2 Vision (1212) 37.0
  37. 37 MUG-U-7B 36.6
  38. 38 SAIL-VL-1.6-8B 36.0
  39. 39 Kimi-VL-A3B-Instruct 35.8
  40. 40 VLAA-Thinker-Qwen2.5VL-3B 35.4
  41. 41 SAIL-VL-1.5-8B 34.8
  42. 42 LLaVA-OneVision-72B (SI) 34.7
  43. 43 InternVL2-Llama3-76B 34.4
  44. 44 GPT-4.1 nano 34.2
  45. 45 Gemma 3 12B 33.9
  46. 46 NVLM-D-72B 33.7
  47. 47 Taichu-VLR-3B 33.6
  48. 48 MiniCPM-o-2_6 33.6
  49. 49 VLM-R1-3B-Math-0305 33.4
  50. 50 Ovis2-4B 33.3
  51. 51 Kimi-VL-A3B-Thinking 33.0
  52. 52 URSA-8B 32.9
  53. 53 InternVL2.5-26B 32.9
  54. 54 Qwen2.5-VL-3B 31.8
  55. 55 POINTS1.5-Qwen2.5-7B 31.4
  56. 56 Yi-Vision 29.8
  57. 57 VITA-1.5 29.5
  58. 58 InternVL3-2B 29.1
  59. 59 Qwen2-VL-7B-Instruct 28.8
  60. 60 SAIL-VL-1.5-2B 27.9
  61. 61 Ovis2-2B 27.6
  62. 62 Gemma 3 4B 27.1
  63. 63 LLaVA-CoT 27.0
  64. 64 InternVL2-8B 26.9
  65. 65 LLaVA-OneVision 7B (SI) 26.6
  66. 66 InternVL2-26B 26.4
  67. 67 InternLM-XComposer2.5-7B 25.8
  68. 68 SAIL-VL-2B 25.2
  69. 69 InternVL-Chat-V1-5 24.6
  70. 70 Aquila-VL-2B 24.3
  71. 71 Ovis2-1B 23.0
  72. 72 GLM-4V-9B 22.5
  73. 73 Llama 3.2 11B 21.7
  74. 74 InternVL2.5-2B-MPO 21.7
  75. 75 InternVL3-1B 21.3
  76. 76 InternVL2.5-2B 21.2
  77. 77 Qwen2-VL-2B-Instruct 20.5
  78. 78 Janus-Pro-7B 19.1
  79. 79 VILA1.5-13B 18.5
  80. 80 SmolVLM-Instruct 18.4
  81. 81 LLaVA-v1.5-13B 13.3
  82. 82 llava-v1.5-7b 12.1

Source: Official Open LMM Reasoning Leaderboard leaderboard

Source: Official Open LMM Reasoning Leaderboard leaderboard

#EntryScoreSettingDateLinkSource
1 Doubao-1.5-pro 61.6 Jan 2025 team.doubao.com Official leaderboard
2 Gemini 2.0 Pro 56.6 Feb 2025 deepmind.google Official leaderboard
3 ChatGPT-4o-latest 54.8 openai.com Official leaderboard
4 GPT-4.1 54.1 Apr 2025 openai.com Official leaderboard
5 InternVL3-78B 51.0 Apr 2025 huggingface.co Official leaderboard
6 Gemini 2.0 Flash Feb 2025 50.6 Feb 2025 deepmind.google Official leaderboard
7 Claude 3.7 Sonnet 50.4 Feb 2025 docs.anthropic.com Official leaderboard
8 InternVL3-38B 50.3 Apr 2025 huggingface.co Official leaderboard
9 Qwen2.5-VL-72B 50.2 Jan 2025 huggingface.co Official leaderboard
10 Gemini 1.5 Pro Sep 2024 50.0 Sep 2024 deepmind.google Official leaderboard
11 GLM-4v-Plus-20250111 49.1 open.bigmodel.cn Official leaderboard
12 Claude 3.5 Sonnet 47.9 Jun 2024 docs.anthropic.com Official leaderboard
13 Ovis2-34B 47.9 Feb 2025 huggingface.co Official leaderboard
14 QVQ-72B-Preview 46.9 Dec 2024 huggingface.co Official leaderboard
15 InternVL3-14B 46.0 Apr 2025 huggingface.co Official leaderboard
16 GPT-4.1 mini 46.0 Apr 2025 openai.com Official leaderboard
17 Step-1o 45.4 platform.stepfun.com Official leaderboard
18 Ovis2-16B 44.7 Feb 2025 huggingface.co Official leaderboard
19 InternVL2.5-78B-MPO 44.3 Dec 2024 huggingface.co Official leaderboard
20 GPT-4o (1120, detail-high) 44.2 openai.com Official leaderboard
21 Taichu-VLR-7B 7B 43.6 platform.wair.ac.cn Official leaderboard
22 InternVL2.5-38B-MPO 43.2 Dec 2024 huggingface.co Official leaderboard
23 VLAA-Thinker-Qwen2.5VL-7B 8.3B 42.5 huggingface.co Official leaderboard
24 InternVL2.5-38B 41.9 Dec 2024 huggingface.co Official leaderboard
25 InternVL2_5-78B 41.7 Dec 2024 huggingface.co Official leaderboard
26 SenseNova 41.4 platform.sensenova.cn Official leaderboard
27 InternVL3-8B 41.4 Apr 2025 huggingface.co Official leaderboard
28 InternVL3-9B 40.8 Apr 2025 huggingface.co Official leaderboard
29 Gemma 3 27B 40.6 Mar 2025 huggingface.co Official leaderboard
30 Qwen2.5-VL-7B 40.1 Jan 2025 huggingface.co Official leaderboard
31 Qwen2-VL-72B 73.4B 38.7 huggingface.co Official leaderboard
32 moonshot-v1-8k 38.0 platform.moonshot.cn Official leaderboard
33 Ovis2-8B 37.8 Feb 2025 huggingface.co Official leaderboard
34 InternVL2.5-8B-BoN-8 8B 37.3 huggingface.co Official leaderboard
35 Ovis1.6-Gemma2-27B 37.0 Nov 2024 huggingface.co Official leaderboard
36 Grok 2 Vision (1212) 37.0 Dec 2024 docs.x.ai Official leaderboard
37 MUG-U-7B 7B 36.6 github.com Official leaderboard
38 SAIL-VL-1.6-8B 36.0 Apr 2025 huggingface.co Official leaderboard
39 Kimi-VL-A3B-Instruct 35.8 Apr 2025 huggingface.co Official leaderboard
40 VLAA-Thinker-Qwen2.5VL-3B 3.8B 35.4 huggingface.co Official leaderboard
41 SAIL-VL-1.5-8B 8B 34.8 huggingface.co Official leaderboard
42 LLaVA-OneVision-72B (SI) 73B 34.7 huggingface.co Official leaderboard
43 InternVL2-Llama3-76B 34.4 Jul 2024 huggingface.co Official leaderboard
44 GPT-4.1 nano 34.2 Apr 2025 openai.com Official leaderboard
45 Gemma 3 12B 33.9 Mar 2025 huggingface.co Official leaderboard
46 NVLM-D-72B 33.7 Sep 2024 huggingface.co Official leaderboard
47 Taichu-VLR-3B 3B 33.6 platform.wair.ac.cn Official leaderboard
48 MiniCPM-o-2_6 33.6 Jan 2025 huggingface.co Official leaderboard
49 VLM-R1-3B-Math-0305 3.8B 33.4 huggingface.co Official leaderboard
50 Ovis2-4B 33.3 Feb 2025 huggingface.co Official leaderboard
51 Kimi-VL-A3B-Thinking 33.0 Apr 2025 huggingface.co Official leaderboard
52 URSA-8B 8B 32.9 huggingface.co Official leaderboard
53 InternVL2.5-26B 32.9 Dec 2024 huggingface.co Official leaderboard
54 Qwen2.5-VL-3B 31.8 Jan 2025 huggingface.co Official leaderboard
55 POINTS1.5-Qwen2.5-7B 8.3B 31.4 huggingface.co Official leaderboard
56 Yi-Vision 29.8 platform.lingyiwanwu.com Official leaderboard
57 VITA-1.5 29.5 Dec 2024 huggingface.co Official leaderboard
58 InternVL3-2B 29.1 Apr 2025 huggingface.co Official leaderboard
59 Qwen2-VL-7B-Instruct 28.8 Aug 2024 huggingface.co Official leaderboard
60 SAIL-VL-1.5-2B 2.5B 27.9 huggingface.co Official leaderboard
61 Ovis2-2B 27.6 Feb 2025 huggingface.co Official leaderboard
62 Gemma 3 4B 27.1 Mar 2025 huggingface.co Official leaderboard
63 LLaVA-CoT 11B 27.0 huggingface.co Official leaderboard
64 InternVL2-8B 26.9 Jul 2024 huggingface.co Official leaderboard
65 LLaVA-OneVision 7B (SI) 26.6 Aug 2024 huggingface.co Official leaderboard
66 InternVL2-26B 26.4 Jul 2024 huggingface.co Official leaderboard
67 InternLM-XComposer2.5-7B 25.8 Jul 2024 huggingface.co Official leaderboard
68 SAIL-VL-2B 25.2 Dec 2024 huggingface.co Official leaderboard
69 InternVL-Chat-V1-5 24.6 Apr 2024 github.com Official leaderboard
70 Aquila-VL-2B 24.3 Oct 2024 huggingface.co Official leaderboard
71 Ovis2-1B 23.0 Feb 2025 huggingface.co Official leaderboard
72 GLM-4V-9B 22.5 Jun 2024 huggingface.co Official leaderboard
73 Llama 3.2 11B 21.7 Sep 2024 huggingface.co Official leaderboard
74 InternVL2.5-2B-MPO 21.7 Dec 2024 huggingface.co Official leaderboard
75 InternVL3-1B 21.3 Apr 2025 huggingface.co Official leaderboard
76 InternVL2.5-2B 21.2 Dec 2024 huggingface.co Official leaderboard
77 Qwen2-VL-2B-Instruct 20.5 Aug 2024 huggingface.co Official leaderboard
78 Janus-Pro-7B 19.1 Jan 2025 huggingface.co Official leaderboard
79 VILA1.5-13B 18.5 Apr 2024 huggingface.co Official leaderboard
80 SmolVLM-Instruct 18.4 Nov 2024 huggingface.co Official leaderboard
81 LLaVA-v1.5-13B 13.3 Oct 2023 arxiv.org Official leaderboard
82 llava-v1.5-7b 12.1 Oct 2023 arxiv.org Official leaderboard
#ModelScoreSettingDateLinkSource
1 Doubao-1.5-pro 61.6 Jan 2025 team.doubao.com Official leaderboard
2 Gemini 2.0 Pro 56.6 Feb 2025 deepmind.google Official leaderboard
3 GPT-4.1 54.1 Apr 2025 openai.com Official leaderboard
4 InternVL3-78B 51.0 Apr 2025 huggingface.co Official leaderboard
5 Gemini 2.0 Flash Feb 2025 50.6 Feb 2025 deepmind.google Official leaderboard
6 Claude 3.7 Sonnet 50.4 Feb 2025 docs.anthropic.com Official leaderboard
7 InternVL3-38B 50.3 Apr 2025 huggingface.co Official leaderboard
8 Qwen2.5-VL-72B 50.2 Jan 2025 huggingface.co Official leaderboard
9 Gemini 1.5 Pro Sep 2024 50.0 Sep 2024 deepmind.google Official leaderboard
10 Claude 3.5 Sonnet 47.9 Jun 2024 docs.anthropic.com Official leaderboard
11 Ovis2-34B 47.9 Feb 2025 huggingface.co Official leaderboard
12 QVQ-72B-Preview 46.9 Dec 2024 huggingface.co Official leaderboard
13 InternVL3-14B 46.0 Apr 2025 huggingface.co Official leaderboard
14 GPT-4.1 mini 46.0 Apr 2025 openai.com Official leaderboard
15 Ovis2-16B 44.7 Feb 2025 huggingface.co Official leaderboard
16 InternVL2.5-78B-MPO 44.3 Dec 2024 huggingface.co Official leaderboard
17 InternVL2.5-38B-MPO 43.2 Dec 2024 huggingface.co Official leaderboard
18 InternVL2.5-38B 41.9 Dec 2024 huggingface.co Official leaderboard
19 InternVL2_5-78B 41.7 Dec 2024 huggingface.co Official leaderboard
20 InternVL3-8B 41.4 Apr 2025 huggingface.co Official leaderboard
21 InternVL3-9B 40.8 Apr 2025 huggingface.co Official leaderboard
22 Gemma 3 27B 40.6 Mar 2025 huggingface.co Official leaderboard
23 Qwen2.5-VL-7B 40.1 Jan 2025 huggingface.co Official leaderboard
24 Ovis2-8B 37.8 Feb 2025 huggingface.co Official leaderboard
25 Ovis1.6-Gemma2-27B 37.0 Nov 2024 huggingface.co Official leaderboard
26 Grok 2 Vision (1212) 37.0 Dec 2024 docs.x.ai Official leaderboard
27 SAIL-VL-1.6-8B 36.0 Apr 2025 huggingface.co Official leaderboard
28 Kimi-VL-A3B-Instruct 35.8 Apr 2025 huggingface.co Official leaderboard
29 InternVL2-Llama3-76B 34.4 Jul 2024 huggingface.co Official leaderboard
30 GPT-4.1 nano 34.2 Apr 2025 openai.com Official leaderboard
31 Gemma 3 12B 33.9 Mar 2025 huggingface.co Official leaderboard
32 NVLM-D-72B 33.7 Sep 2024 huggingface.co Official leaderboard
33 MiniCPM-o-2_6 33.6 Jan 2025 huggingface.co Official leaderboard
34 Ovis2-4B 33.3 Feb 2025 huggingface.co Official leaderboard
35 Kimi-VL-A3B-Thinking 33.0 Apr 2025 huggingface.co Official leaderboard
36 InternVL2.5-26B 32.9 Dec 2024 huggingface.co Official leaderboard
37 Qwen2.5-VL-3B 31.8 Jan 2025 huggingface.co Official leaderboard
38 VITA-1.5 29.5 Dec 2024 huggingface.co Official leaderboard
39 InternVL3-2B 29.1 Apr 2025 huggingface.co Official leaderboard
40 Qwen2-VL-7B-Instruct 28.8 Aug 2024 huggingface.co Official leaderboard
41 Ovis2-2B 27.6 Feb 2025 huggingface.co Official leaderboard
42 Gemma 3 4B 27.1 Mar 2025 huggingface.co Official leaderboard
43 InternVL2-8B 26.9 Jul 2024 huggingface.co Official leaderboard
44 LLaVA-OneVision 7B (SI) 26.6 Aug 2024 huggingface.co Official leaderboard
45 InternVL2-26B 26.4 Jul 2024 huggingface.co Official leaderboard
46 InternLM-XComposer2.5-7B 25.8 Jul 2024 huggingface.co Official leaderboard
47 SAIL-VL-2B 25.2 Dec 2024 huggingface.co Official leaderboard
48 InternVL-Chat-V1-5 24.6 Apr 2024 github.com Official leaderboard
49 Aquila-VL-2B 24.3 Oct 2024 huggingface.co Official leaderboard
50 Ovis2-1B 23.0 Feb 2025 huggingface.co Official leaderboard
51 GLM-4V-9B 22.5 Jun 2024 huggingface.co Official leaderboard
52 Llama 3.2 11B 21.7 Sep 2024 huggingface.co Official leaderboard
53 InternVL2.5-2B-MPO 21.7 Dec 2024 huggingface.co Official leaderboard
54 InternVL3-1B 21.3 Apr 2025 huggingface.co Official leaderboard
55 InternVL2.5-2B 21.2 Dec 2024 huggingface.co Official leaderboard
56 Qwen2-VL-2B-Instruct 20.5 Aug 2024 huggingface.co Official leaderboard
57 Janus-Pro-7B 19.1 Jan 2025 huggingface.co Official leaderboard
58 VILA1.5-13B 18.5 Apr 2024 huggingface.co Official leaderboard
59 SmolVLM-Instruct 18.4 Nov 2024 huggingface.co Official leaderboard
60 LLaVA-v1.5-13B 13.3 Oct 2023 arxiv.org Official leaderboard
61 llava-v1.5-7b 12.1 Oct 2023 arxiv.org Official leaderboard

Rankings by part

We read 6 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as MathVista, MathVision and MathVerse. Doubao-1.5-pro leads 2 of them, and 3 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.

MathVista

61 models we track have a score here.

  1. 1 InternVL3-78B ↑3 79.0
  2. 2 Doubao-1.5-pro ↓1 78.6
  3. 3 InternVL2.5-78B-MPO ↑13 76.6
  4. 4 InternVL3-38B ↑3 76.3
  5. 5 Ovis2-34B ↑6 76.1
  6. 6 InternVL3-14B ↑7 74.4
  7. 7 Qwen2.5-VL-72B ↑1 74.2
  8. 8 SAIL-VL-1.6-8B ↑19 74.2
  9. 9 Ovis2-16B ↑6 73.7
  10. 10 InternVL2.5-38B-MPO ↑7 73.6

2 models have held number one on the official leaderboard since its release in December 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind ByteDance Seed
Own leaderboard Not tested since Apr 2025

Source: Official Open LMM Reasoning Leaderboard leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
20.0 40.0 60.0 2024 2025 20.0 40.0 60.0 2024 2025

Source: Official Open LMM Reasoning Leaderboard leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Other

All 69 in Other
  • Artificial Analysis Intelligence Index v5 In use Version 5 of Artificial Analysis's index: nine evaluations in four equal categories, most with private test sets, run three times each in its own agent harness. Claude Fable 5.2 64.6
  • Personal Superintelligence Index (Meta) In use Meta's composite of seven personal and knowledge-work evals: Agents' Last Exam, AutomationBench-AA, FocusBench, JobBench, GDP.pdf, VBT and DRACO.
  • Vals Science Index In use Agents are scored on the equal average of BioMysteryBench, MysteryMechanism and Terminal-Bench-Science. GPT-6.2 Astra 68.4%
  • Decision Index Public v0.3 In use Decision models return probabilities over labeled options in public tasks covering knowledge, language, retrieval, tools, and arts. A chance-corrected weighted aggregate combines the task scores. Granite Switch 4.1 30B 51.3
  • SAGE v1.1 (Vals) In use Models grade handwritten undergraduate math and computer science work against rubrics. Scored by class-balanced accuracy of each rubric check against expert grading. Claude Opus 5.5 91.3%
  • MTEB Fading Massive Text Embedding Benchmark: retrieval, clustering, classification across languages and modalities. Qwen3-Embedding-8B 72.2%

Data audit

8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 19 months ago, in April 2025, for SAIL-VL-1.6-8B. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (82 entries).
  • Pass Models scored 61 models we track, plus 21 other entries on its leaderboard.
  • Warning Newest score The newest came 19 months ago, in April 2025, for SAIL-VL-1.6-8B.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119).
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on huggingface.co: 61 rows for models we track and 21 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check