Benchmark Almanac

We-Math

Qwen 3.6 Plus 89% leads since April 2026
Seed1.5-VL 77.5% leads since May 2025
Qwen 3.6 Plus 89% leads since April 2026
Fading

Models solve visual math problems decomposed into sub-problems by knowledge concept. Scoring compares sub-problem and full-problem results to diagnose knowledge and generalization gaps.

A successor, We-Math 2.0, followed in August 2025.

Released
July 2024
Kind
Visual math and diagrams
Level
School
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Strongest model tested
Qwen 3.6 Plus #74 on the capability index
Last in a launch post
May 2026, dropped after it saturated
Last model added
June 2026
Scores refreshed
November 12th, 2026

In the authors’ words

Visual mathematical reasoning, as a fundamental visual reasoning ability, has received widespread attention from the Large Multimodal Models (LMMs) community. Existing benchmarks, such as MathVista and MathVerse, focus more on the result-oriented performance but neglect the underlying principles in knowledge acquisition and generalization. Inspired by human-like mathematical reasoning, we introduce WE-MATH, the first benchmark specifically designed to explore the problem-solving principles beyond end-to-end performance. We meticulously collect and categorize 6.5K visual math problems, spanning 67 hierarchical knowledge concepts and five layers of knowledge granularity. We decompose composite problems into sub-problems according to the required knowledge concepts and introduce a novel four-dimensional metric, namely Insufficient Knowledge (IK), Inadequate Generalization (IG), Complete Mastery (CM), and Rote Memorization (RM), to hierarchically assess inherent issues in LMMs' reasoning process. With WE-MATH, we conduct a thorough evaluation of existing LMMs in visual mathematical reasoning and reveal a negative correlation between solving steps and problem-specific performance. We confirm the IK issue of LMMs can be effectively improved via knowledge augmentation strategies. More notably, the primary challenge of GPT-4o has significantly transitioned from IK to IG, establishing it as the first LMM advancing towards the knowledge generalization stage. In contrast, other LMMs exhibit a marked inclination towards Rote Memorization - they correctly solve composite problems involving multiple knowledge concepts yet fail to answer sub-problems. We anticipate that WE-MATH will open new pathways for advancements in visual mathematical reasoning for LMMs. The WE-MATH data and evaluation code are available at https://github.com/We-Math/We-Math.

Abstract of We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning? (arXiv:2407.01284, CC0)
13
Models scored in lab cards
10
Launch posts list it, from 5 labs
9
System cards report it, from 7 labs

Qwen 3.6 Plus leads with 89% among models released by May 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Seed1.5-VL leads with 77.5% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Qwen 3.6 Plus leads with 89%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

The newest model with a score here came out in May 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

13 models. Scroll sideways for the rest.

  1. 89.0 Qwen 3.6 Plus
  2. 87.9 Qwen3.5 397B-A17B
  3. 79.1 EXAONE 4.5 33B
  4. 75.5 Keye-VL-2.0-30B-A3B
  5. 75.4 Qwen3.5-4B
  6. 75.2 Qwen3.5-9B
  7. 74.8 Qwen3-VL-235B-A22B
  8. 73.0 STEP3-VL-10B
  9. 69.8 GLM-4.6V
  10. 68.8 GLM-4.5V
  11. 63.8 GLM-4.1V-9B-Thinking
  12. 62.3 InternVL3.5-241B-A28B
  13. 39.3 Nemotron Nano 12B v2 VL
  1. 1 Qwen 3.6 Plus 89%
  2. 2 Qwen3.5 397B-A17B 87.9%
  3. 3 EXAONE 4.5 33B 79.1%
  4. 4 Keye-VL-2.0-30B-A3B 75.5%
  5. 5 Qwen3.5-4B 75.4%
  6. 6 Qwen3.5-9B 75.2%
  7. 7 Qwen3-VL-235B-A22B 74.8%
  8. 8 STEP3-VL-10B 73%
  9. 9 GLM-4.6V 69.8%
  10. 10 GLM-4.5V 68.8%
  11. 11 GLM-4.1V-9B-Thinking 63.8%
  12. 12 InternVL3.5-241B-A28B 62.3%
  13. 13 Nemotron Nano 12B v2 VL 39.3%

Source: Labs’ model cards

21 models. Scroll sideways for the rest.

  1. 77.5 Seed1.5-VL
  2. 65.7 Doubao-1.5-pro
  3. 55.5 GPT-4.1
  4. 49.1 Qwen2.5-VL-72B
  5. 46.1 InternVL3-78B
  6. 39.5 InternVL3-8B
  7. 36.9 InternVL2-Llama3-76B
  8. 36.2 Qwen2.5-VL-7B
  9. 31.1 GPT-4V
  10. 28.7 LLaVA-OneVision 72B
  11. 26.6 InternVL2-8B
  12. 25.6 Qwen2-VL-7B-Instruct
  13. 23.5 InternVL2.5-8B
  14. 23.1 LLaVA-OneVision 7B
  15. 14.9 InternVL-Chat-V1-5
  16. 14.9 GLM-4V-9B
  17. 12.7 InternLM-XComposer2-VL-7B
  18. 11.5 LongVA-7B
  19. 10.5 Qwen-VL-Max
  20. 9.52 MiniCPM-Llama3-V 2.5
  21. 5.90 DeepSeek-VL-1.3B-Chat
  1. 1 Seed1.5-VL 77.5%
  2. 2 Doubao-1.5-pro 65.7%
  3. 3 GPT-4.1 55.5%
  4. 4 Qwen2.5-VL-72B 49.1%
  5. 5 InternVL3-78B 46.1%
  6. 6 InternVL3-8B 39.5%
  7. 7 InternVL2-Llama3-76B 36.9%
  8. 8 Qwen2.5-VL-7B 36.2%
  9. 9 GPT-4V 31.1%
  10. 10 LLaVA-OneVision 72B 28.7%
  11. 11 InternVL2-8B 26.6%
  12. 12 Qwen2-VL-7B-Instruct 25.6%
  13. 13 InternVL2.5-8B 23.5%
  14. 14 LLaVA-OneVision 7B 23.1%
  15. 15 InternVL-Chat-V1-5 14.9%
  16. 16 GLM-4V-9B 14.9%
  17. 17 InternLM-XComposer2-VL-7B 12.7%
  18. 18 LongVA-7B 11.5%
  19. 19 Qwen-VL-Max 10.5%
  20. 20 MiniCPM-Llama3-V 2.5 9.52%
  21. 21 DeepSeek-VL-1.3B-Chat 5.90%

Source: Official We-Math leaderboard

34 models. Scroll sideways for the rest.

  1. 89.0 Qwen 3.6 Plus
  2. 87.9 Qwen3.5 397B-A17B
  3. 79.1 EXAONE 4.5 33B
  4. 77.5 Seed1.5-VL
  5. 75.5 Keye-VL-2.0-30B-A3B
  6. 75.4 Qwen3.5-4B
  7. 75.2 Qwen3.5-9B
  8. 74.8 Qwen3-VL-235B-A22B
  9. 73.0 STEP3-VL-10B
  10. 69.8 GLM-4.6V
  11. 68.8 GLM-4.5V
  12. 65.7 Doubao-1.5-pro
  13. 63.8 GLM-4.1V-9B-Thinking
  14. 62.3 InternVL3.5-241B-A28B
  15. 55.5 GPT-4.1
  16. 49.1 Qwen2.5-VL-72B
  17. 46.1 InternVL3-78B
  18. 39.5 InternVL3-8B
  19. 39.3 Nemotron Nano 12B v2 VL
  20. 36.9 InternVL2-Llama3-76B
  21. 36.2 Qwen2.5-VL-7B
  22. 31.1 GPT-4V
  23. 28.7 LLaVA-OneVision 72B
  24. 26.6 InternVL2-8B
  25. 25.6 Qwen2-VL-7B-Instruct
  26. 23.5 InternVL2.5-8B
  27. 23.1 LLaVA-OneVision 7B
  28. 14.9 InternVL-Chat-V1-5
  29. 14.9 GLM-4V-9B
  30. 12.7 InternLM-XComposer2-VL-7B
  31. 11.5 LongVA-7B
  32. 10.5 Qwen-VL-Max
  33. 9.52 MiniCPM-Llama3-V 2.5
  34. 5.90 DeepSeek-VL-1.3B-Chat
  1. 1 Qwen 3.6 Plus 89%
  2. 2 Qwen3.5 397B-A17B 87.9%
  3. 3 EXAONE 4.5 33B 79.1%
  4. 4 Seed1.5-VL 77.5%
  5. 5 Keye-VL-2.0-30B-A3B 75.5%
  6. 6 Qwen3.5-4B 75.4%
  7. 7 Qwen3.5-9B 75.2%
  8. 8 Qwen3-VL-235B-A22B 74.8%
  9. 9 STEP3-VL-10B 73%
  10. 10 GLM-4.6V 69.8%
  11. 11 GLM-4.5V 68.8%
  12. 12 Doubao-1.5-pro 65.7%
  13. 13 GLM-4.1V-9B-Thinking 63.8%
  14. 14 InternVL3.5-241B-A28B 62.3%
  15. 15 GPT-4.1 55.5%
  16. 16 Qwen2.5-VL-72B 49.1%
  17. 17 InternVL3-78B 46.1%
  18. 18 InternVL3-8B 39.5%
  19. 19 Nemotron Nano 12B v2 VL 39.3%
  20. 20 InternVL2-Llama3-76B 36.9%
  21. 21 Qwen2.5-VL-7B 36.2%
  22. 22 GPT-4V 31.1%
  23. 23 LLaVA-OneVision 72B 28.7%
  24. 24 InternVL2-8B 26.6%
  25. 25 Qwen2-VL-7B-Instruct 25.6%
  26. 26 InternVL2.5-8B 23.5%
  27. 27 LLaVA-OneVision 7B 23.1%
  28. 28 InternVL-Chat-V1-5 14.9%
  29. 29 GLM-4V-9B 14.9%
  30. 30 InternLM-XComposer2-VL-7B 12.7%
  31. 31 LongVA-7B 11.5%
  32. 32 Qwen-VL-Max 10.5%
  33. 33 MiniCPM-Llama3-V 2.5 9.52%
  34. 34 DeepSeek-VL-1.3B-Chat 5.90%

Sources: Labs’ model cards and official We-Math leaderboard

#ModelScoreSettingDateSource
1 Qwen 3.6 Plus 89% Apr 2026 Lab model card
2 Qwen3.5 397B-A17B 87.9% Feb 2026 Lab model card
3 EXAONE 4.5 33B 79.1% Apr 2026 Lab model card
4 Keye-VL-2.0-30B-A3B 75.5% May 2026 Lab model card
5 Qwen3.5-4B 75.4% Mar 2026 Lab model card
6 Qwen3.5-9B 75.2% Mar 2026 Lab model card
7 Qwen3-VL-235B-A22B 74.8% Nov 2025 Lab model card
8 STEP3-VL-10B 73% Jan 2026 Lab model card
9 GLM-4.6V 69.8% Dec 2025 Lab model card
10 GLM-4.5V 68.8% Aug 2025 Lab model card
11 GLM-4.1V-9B-Thinking 63.8% Jul 2025 Lab model card
12 InternVL3.5-241B-A28B 62.3% Aug 2025 Lab model card
13 Nemotron Nano 12B v2 VL 39.3% Nov 2025 Lab model card
#ModelScoreSettingDateSource
1 Seed1.5-VL 77.5% May 2025 Official leaderboard
2 Doubao-1.5-pro 65.7% Jan 2025 Official leaderboard
3 GPT-4.1 55.5% Apr 2025 Official leaderboard
4 Qwen2.5-VL-72B 49.1% Jan 2025 Official leaderboard
5 InternVL3-78B 46.1% Apr 2025 Official leaderboard
6 InternVL3-8B 39.5% Apr 2025 Official leaderboard
7 InternVL2-Llama3-76B 36.9% Jul 2024 Official leaderboard
8 Qwen2.5-VL-7B 36.2% Jan 2025 Official leaderboard
9 GPT-4V 31.1% Nov 2023 Official leaderboard
10 LLaVA-OneVision 72B 28.7% Aug 2024 Official leaderboard
11 InternVL2-8B 26.6% Jul 2024 Official leaderboard
12 Qwen2-VL-7B-Instruct 25.6% Aug 2024 Official leaderboard
13 InternVL2.5-8B 23.5% Dec 2024 Official leaderboard
14 LLaVA-OneVision 7B 23.1% Aug 2024 Official leaderboard
15 InternVL-Chat-V1-5 14.9% Apr 2024 Official leaderboard
16 GLM-4V-9B 14.9% Jun 2024 Official leaderboard
17 InternLM-XComposer2-VL-7B 12.7% Jan 2024 Official leaderboard
18 LongVA-7B 11.5% Jun 2024 Official leaderboard
19 Qwen-VL-Max 10.5% Jan 2024 Official leaderboard
20 MiniCPM-Llama3-V 2.5 9.52% May 2024 Official leaderboard
21 DeepSeek-VL-1.3B-Chat 5.90% Mar 2024 Official leaderboard
#ModelScoreSettingDateSource
1 Qwen 3.6 Plus 89% Apr 2026 Lab model card
2 Qwen3.5 397B-A17B 87.9% Feb 2026 Lab model card
3 EXAONE 4.5 33B 79.1% Apr 2026 Lab model card
4 Seed1.5-VL 77.5% May 2025 Official leaderboard
5 Keye-VL-2.0-30B-A3B 75.5% May 2026 Lab model card
6 Qwen3.5-4B 75.4% Mar 2026 Lab model card
7 Qwen3.5-9B 75.2% Mar 2026 Lab model card
8 Qwen3-VL-235B-A22B 74.8% Nov 2025 Lab model card
9 STEP3-VL-10B 73% Jan 2026 Lab model card
10 GLM-4.6V 69.8% Dec 2025 Lab model card
11 GLM-4.5V 68.8% Aug 2025 Lab model card
12 Doubao-1.5-pro 65.7% Jan 2025 Official leaderboard
13 GLM-4.1V-9B-Thinking 63.8% Jul 2025 Lab model card
14 InternVL3.5-241B-A28B 62.3% Aug 2025 Lab model card
15 GPT-4.1 55.5% Apr 2025 Official leaderboard
16 Qwen2.5-VL-72B 49.1% Jan 2025 Official leaderboard
17 InternVL3-78B 46.1% Apr 2025 Official leaderboard
18 InternVL3-8B 39.5% Apr 2025 Official leaderboard
19 Nemotron Nano 12B v2 VL 39.3% Nov 2025 Lab model card
20 InternVL2-Llama3-76B 36.9% Jul 2024 Official leaderboard
21 Qwen2.5-VL-7B 36.2% Jan 2025 Official leaderboard
22 GPT-4V 31.1% Nov 2023 Official leaderboard
23 LLaVA-OneVision 72B 28.7% Aug 2024 Official leaderboard
24 InternVL2-8B 26.6% Jul 2024 Official leaderboard
25 Qwen2-VL-7B-Instruct 25.6% Aug 2024 Official leaderboard
26 InternVL2.5-8B 23.5% Dec 2024 Official leaderboard
27 LLaVA-OneVision 7B 23.1% Aug 2024 Official leaderboard
28 InternVL-Chat-V1-5 14.9% Apr 2024 Official leaderboard
29 GLM-4V-9B 14.9% Jun 2024 Official leaderboard
30 InternLM-XComposer2-VL-7B 12.7% Jan 2024 Official leaderboard
31 LongVA-7B 11.5% Jun 2024 Official leaderboard
32 Qwen-VL-Max 10.5% Jan 2024 Official leaderboard
33 MiniCPM-Llama3-V 2.5 9.52% May 2024 Official leaderboard
34 DeepSeek-VL-1.3B-Chat 5.90% Mar 2024 Official leaderboard

The benchmark's own leaderboard has 33 entries, 21 of them models we track. Switch the scores above to rank them together.

5 models have held number one in labs’ own numbers since its release in July 2024 4 models have held number one across all independent runs since its release in July 2024 6 models have held number one with claims included since its release in July 2024

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

OpenAI Shanghai AI Lab ByteDance Seed Z.ai Qwen
Own leaderboard Not tested since May 2025
Seed1.5-VL 77.5%
Lab claims Passed 85% in Feb 2026, by Qwen3.5 397B-A17B
Qwen 3.6 Plus 89%

Sources: Official We-Math leaderboard and labs’ model cards

Qwen’s own number for Qwen3.5 397B-A17B passed 85% in February 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen 3.6 Plus Seed1.5-VL Doubao-1.5-pro InternVL2-Llama3-76B GPT-4V 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen 3.6 Plus Seed1.5-VL

Sources: Official We-Math leaderboard and labs’ model cards

Labs headline it less often now, last in May 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Z.ai
Shanghai AI Lab
Qwen
LG AI Research
Kuaishou
DateLaunch postScoreAs the post writes it
May 25, 2026 Keye-VL-2.0-30B-A3B 75.5 WeMath
Apr 9, 2026 EXAONE 4.5 33B 79.1 WeMath
Apr 2, 2026 Qwen 3.6 Plus 89 We-Math
Mar 2, 2026 Qwen3.5-4B 75.4 We-Math
Mar 2, 2026 Qwen3.5-9B 75.2 We-Math
Feb 16, 2026 Qwen3.5 397B-A17B 87.9 We-Math
Dec 8, 2025 GLM-4.6V 69.8 WeMath
Aug 25, 2025 InternVL3.5-241B-A28B 62.3 WeMath
Aug 11, 2025 GLM-4.5V 68.8 WeMath
Jul 1, 2025 GLM-4.1V-9B-Thinking 63.8 WeMath

9 system cards report it, from 7 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Jun 9, 2026 Keye-VL-2.0-30B-A3B technical report 75.5 WeMath (Figure 5, printed value) Capabilities
Apr 9, 2026 EXAONE 4.5 33B technical report 79.1 We-Math (Table 2, Reasoning mode) Capabilities
Jan 14, 2026 STEP3-VL-10B technical report 73 We-Math, SeRe (Tables 1 and 3). PaCoRe 73.90 Capabilities
Jan 1, 2026 GLM-4.6V technical report 65.9 WeMath, GLM-4.6V thinking (Table 2). The launch image prints 69.8 Capabilities
Nov 26, 2025 Qwen3-VL-235B-A22B technical report 74.8 We-Math (Table 2) Capabilities
Nov 6, 2025 Nemotron Nano 12B v2 VL technical report 39.3 WeMath, reasoning on (Table 3). Reasoning off 31.5 Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 62.3 WeMath (Table 2, Table 3). With parallel thinking 64.4 Capabilities
Aug 13, 2025 GLM-4.5V technical report 68.8 WeMath, GLM-4.5V thinking (Table 2) Capabilities
Jul 1, 2025 GLM-4.1V-9B-Thinking technical report 63.8 WeMath, GLM-4.1V-9B-Thinking thinking (Table 2) Capabilities

More in Vision

All 281 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%