MathVerse
The official leaderboard hasn’t scored a model released after LLaVA-v1.6-34B (January 2024). Labs report their own numbers for 12 newer models, such as Seed2.1 Pro at 89.7%.
Models solve diagram-based math problems shown in six versions that move information between the text and the image. It tests whether models read the diagram.
Labs usually report the testmini split.
- Made by
- CUHK MMLab and Shanghai AI Laboratory
- Released
- March 2024
- Kind
- Visual math and diagrams
- Level
- School
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- All public
- Measures
- Capability
- Subject
- Mathematics
- Language
- English
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
- In views
- Vision
- Last in a launch post
- October 2026, dropped after it saturated
- Last model added
- October 2026
- Scores refreshed
- November 19th, 2026
In the authors’ words
The remarkable progress of Multi-modal Large Language Models (MLLMs) has garnered unparalleled attention, due to their superior performance in visual contexts. However, their capabilities in visual math problem-solving remain insufficiently evaluated and understood. We investigate current benchmarks to incorporate excessive visual content within textual questions, which potentially assist MLLMs in deducing answers without truly interpreting the input diagrams. To this end, we introduce MathVerse, an all-around visual math benchmark designed for an equitable and in-depth evaluation of MLLMs. We meticulously collect 2,612 high-quality, multi-subject math problems with diagrams from publicly available sources. Each problem is then transformed by human annotators into six distinct versions, each offering varying degrees of information content in multi-modality, contributing to 15K test samples in total. This approach allows MathVerse to comprehensively assess whether and how much MLLMs can truly understand the visual diagrams for mathematical reasoning. In addition, we propose a Chain-of-Thought (CoT) evaluation strategy for a fine-grained assessment of the output answers. Rather than naively judging True or False, we employ GPT-4(V) to adaptively extract crucial reasoning steps, and then score each step with detailed error analysis, which can reveal the intermediate CoT reasoning quality by MLLMs. We hope the MathVerse benchmark may provide unique insights to guide the future development of MLLMs. Project page: https://mathverse-cuhk.github.io
Seed2.1 Pro leads with 89.7%
Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.
GPT-4V leads with 39.4% across all independent runs
Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.
Seed2.1 Pro leads with 89.7%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
- 1
Seed2.1 Pro 89.7%
- 2
MiniCPM-V-4.7-35B-A3B 86.5%
- 3
ERNIE 5.0 85.1%
- 4
Qwen3-VL-235B-A22B 85%
- 5
ERNIE-4.5-VL-28B-A3B-Thinking 81%
- 6
STEP3-VL-10B 75.7%
- 7 GLM-4.6V 75.4%
- 8 GLM-4.5V 72.1%
- 9
InternVL3.5-241B-A28B 68.5%
- 10 GLM-4.1V-9B-Thinking 68.4%
- 11
Nemotron Nano 12B v2 VL 58.2%
- 12
Qwen2.5-VL-72B 57.6%
- 13
Phi-4-Reasoning-Vision-15B 44.9%
Source: Labs’ model cards
- 1 GPT-4V 39.4%
- 2 Qwen-VL-Max 25.3%
- 3
LLaVA-v1.6-34B 23.8%
- 4
Gemini 1.0 Pro 23.5%
- 5
InternLM-XComposer2-VL-7B 16.5%
- 6 llava-v1.6-vicuna-13b 15.6%
- 7
ShareGPT4V-13B 13.1%
- 8
Qwen-VL-Plus 11.8%
- 9
LLaVA-v1.5-13B 7.60%
Source: Official MathVerse leaderboard
- 89.7
Seed2.1 Pro
- 86.5
MiniCPM-V-4.7-35B-A3B
- 85.1
ERNIE 5.0
- 85.0
Qwen3-VL-235B-A22B
- 81.0
ERNIE-4.5-VL-28B-A3B-Thinking
- 75.7
STEP3-VL-10B
- 75.4 GLM-4.6V
- 72.1 GLM-4.5V
- 68.5
InternVL3.5-241B-A28B
- 68.4 GLM-4.1V-9B-Thinking
- 58.2
Nemotron Nano 12B v2 VL
- 57.6
Qwen2.5-VL-72B
- 44.9
Phi-4-Reasoning-Vision-15B
- 39.4 GPT-4V
- 25.3 Qwen-VL-Max
- 23.8
LLaVA-v1.6-34B
- 23.5
Gemini 1.0 Pro
- 16.5
InternLM-XComposer2-VL-7B
- 15.6 llava-v1.6-vicuna-13b
- 13.1
ShareGPT4V-13B
- 11.8
Qwen-VL-Plus
- 7.60
LLaVA-v1.5-13B
- 1
Seed2.1 Pro 89.7%
- 2
MiniCPM-V-4.7-35B-A3B 86.5%
- 3
ERNIE 5.0 85.1%
- 4
Qwen3-VL-235B-A22B 85%
- 5
ERNIE-4.5-VL-28B-A3B-Thinking 81%
- 6
STEP3-VL-10B 75.7%
- 7 GLM-4.6V 75.4%
- 8 GLM-4.5V 72.1%
- 9
InternVL3.5-241B-A28B 68.5%
- 10 GLM-4.1V-9B-Thinking 68.4%
- 11
Nemotron Nano 12B v2 VL 58.2%
- 12
Qwen2.5-VL-72B 57.6%
- 13
Phi-4-Reasoning-Vision-15B 44.9%
- 14 GPT-4V 39.4%
- 15 Qwen-VL-Max 25.3%
- 16
LLaVA-v1.6-34B 23.8%
- 17
Gemini 1.0 Pro 23.5%
- 18
InternLM-XComposer2-VL-7B 16.5%
- 19 llava-v1.6-vicuna-13b 15.6%
- 20
ShareGPT4V-13B 13.1%
- 21
Qwen-VL-Plus 11.8%
- 22
LLaVA-v1.5-13B 7.60%
Sources: Labs’ model cards and official MathVerse leaderboard
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 89.7% | Jun 2026 | Lab model card | |
| 2 | | 86.5% | Oct 2026 | Lab model card | |
| 3 | | 85.1% | Feb 2026 | Lab model card | |
| 4 | | 85% | Sep 2025 | Lab model card | |
| 5 | | 81% | Nov 2025 | Lab model card | |
| 6 | | 75.7% | Jan 2026 | Lab model card | |
| 7 | GLM-4.6V | 75.4% | Jan 2026 | Lab model card | |
| 8 | GLM-4.5V | 72.1% | Aug 2025 | Lab model card | |
| 9 | | 68.5% | Aug 2025 | Lab model card | |
| 10 | GLM-4.1V-9B-Thinking | 68.4% | Jul 2025 | Lab model card | |
| 11 | | 58.2% | Nov 2025 | Lab model card | |
| 12 | | 57.6% | Feb 2025 | Lab model card | |
| 13 | | 44.9% | Mar 2026 | Lab model card |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-4V | 39.4% | Nov 2023 | Official leaderboard | |
| 2 | Qwen-VL-Max | 25.3% | Jan 2024 | Official leaderboard | |
| 3 | 23.8% | Jan 2024 | Official leaderboard | ||
| 4 | 23.5% | Dec 2023 | Official leaderboard | ||
| 5 | 16.5% | Jan 2024 | Official leaderboard | ||
| 6 | llava-v1.6-vicuna-13b | 15.6% | Jan 2024 | Official leaderboard | |
| 7 | 13.1% | Dec 2023 | Official leaderboard | ||
| 8 | 11.8% | Jan 2024 | Official leaderboard | ||
| 9 | 7.60% | Oct 2023 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | 89.7% | Jun 2026 | Lab model card | ||
| 2 | 86.5% | Oct 2026 | Lab model card | ||
| 3 | 85.1% | Feb 2026 | Lab model card | ||
| 4 | 85% | Sep 2025 | Lab model card | ||
| 5 | 81% | Nov 2025 | Lab model card | ||
| 6 | 75.7% | Jan 2026 | Lab model card | ||
| 7 | GLM-4.6V | 75.4% | Jan 2026 | Lab model card | |
| 8 | GLM-4.5V | 72.1% | Aug 2025 | Lab model card | |
| 9 | 68.5% | Aug 2025 | Lab model card | ||
| 10 | GLM-4.1V-9B-Thinking | 68.4% | Jul 2025 | Lab model card | |
| 11 | 58.2% | Nov 2025 | Lab model card | ||
| 12 | 57.6% | Feb 2025 | Lab model card | ||
| 13 | 44.9% | Mar 2026 | Lab model card | ||
| 14 | GPT-4V | 39.4% | Nov 2023 | Official leaderboard | |
| 15 | Qwen-VL-Max | 25.3% | Jan 2024 | Official leaderboard | |
| 16 | 23.8% | Jan 2024 | Official leaderboard | ||
| 17 | 23.5% | Dec 2023 | Official leaderboard | ||
| 18 | 16.5% | Jan 2024 | Official leaderboard | ||
| 19 | llava-v1.6-vicuna-13b | 15.6% | Jan 2024 | Official leaderboard | |
| 20 | 13.1% | Dec 2023 | Official leaderboard | ||
| 21 | 11.8% | Jan 2024 | Official leaderboard | ||
| 22 | 7.60% | Oct 2023 | Official leaderboard |
The benchmark's own leaderboard has 19 entries, 9 of them models we track. Switch the scores above to rank them together.
6 models have held number one in labs’ own numbers since its release in March 2024 Number one over time 7 models have held number one with claims included since its release in March 2024
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Sources: Official MathVerse leaderboard and labs’ model cards
Qwen’s own number for Qwen3-VL-235B-A22B passed 85% in September 2025
That was 18 months after its release. By the same age, 31% of the benchmarks from 2024 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official MathVerse leaderboard and labs’ model cards
Labs headline it less often now, last in October 2026
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Oct 7, 2026 | 86.5 | MathVerse_MINI | |
| Mar 4, 2026 | 44.9 | MathVerse_MINI (default mode) | |
| Feb 6, 2026 | 85.1 | MathVerse | |
| Nov 11, 2025 | 81 | MathVerse | |
| Sep 23, 2025 | 85 | MathVerse_mini | |
| Aug 25, 2025 | 68.5 | MathVerse (vision-only) | |
| Aug 11, 2025 | GLM-4.5V | 72.1 | MathVerse |
11 system cards report it, from 8 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Jun 23, 2026 | | 89.7 | MathVerse (Vision-Only) (Table 9) | Capabilities |
| Mar 4, 2026 | | 44.9 | MathVerse MINI, default mode, 43.8 force nothink, 53.1 force thinking (Tables 5 and 6) | Capabilities |
| Feb 4, 2026 | | 85.1 | MathVerse (Table 3) | Capabilities |
| Jan 14, 2026 | | 75.7 | MathVerse (vision), SeRe (Tables 1 and 3). PaCoRe 78.30 | Capabilities |
| Jan 1, 2026 | GLM-4.6V technical report | 75.4 | MathVerse, GLM-4.6V thinking (Table 2) | Capabilities |
| Nov 26, 2025 | | 85 | MathVerse mini (Table 2) | Capabilities |
| Nov 6, 2025 | | 58.2 | MathVerse-Mini (Vision-only), reasoning on (Table 3). Reasoning off 34.3 | Capabilities |
| Aug 25, 2025 | | 68.5 | MathVerse (vision-only) (Table 2, Table 3). With parallel thinking 71.6 | Capabilities |
| Aug 13, 2025 | GLM-4.5V technical report | 72.1 | MathVerse, GLM-4.5V thinking (Table 2) | Capabilities |
| Jul 1, 2025 | GLM-4.1V-9B-Thinking technical report | 68.4 | MathVerse, GLM-4.1V-9B-Thinking thinking (Table 2) | Capabilities |
| Feb 19, 2025 | | 57.6 | MathVerse mini (Table 3) | Capabilities |
More in Vision
- MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy.
Qwen 3.6 Plus 86%
- MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
- CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
- MathVista In use Math reasoning in visual contexts (charts, geometry, puzzles) from 31 datasets. Answer accuracy.
Seed2.1 Pro 90.7%
- MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy.
Qwen 3.8 Max 97.7%
- MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy.
Qwen 3.8 Max 85.9%
Data audit
All 9 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (19 entries), launch posts (7 scores) and system cards (11 scores).
- Pass Models scored 22 models we track, plus 10 other entries on its leaderboard.
- Pass Newest score The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B.
- Note Frontier models None of its models are on the capabilities index.
- Note Headroom Saturated. The best is 89.7%, past 85% since September 2025.
Leaderboard import 3 pass
- Pass Importer We read the board on mathverse-cuhk.github.io: 9 rows for models we track and 10 other entries at the last read.
- Pass Last read November 19th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.