Benchmark Almanac

MMMU-Pro

GPT-5.6 Sol 84.6% leads since July 2026
GPT-5.4 82.1% leads since March 2026
GPT-5.6 Sol 84.6% leads since July 2026
In use

Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts.

Variant (standard vs vision, 4 vs 10 options) often unstated in cards.

Made by
MMMU team
Released
September 2024
Kind
General visual questions
Level
University
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Strongest model tested
GPT-5.6 Sol #13 on the capability index
Last in a launch post
November 2026
Last model added
November 2026
Scores refreshed
November 12th, 2026

In the authors’ words

This paper introduces MMMU-Pro, a robust version of the Massive Multi-discipline Multimodal Understanding and Reasoning (MMMU) benchmark. MMMU-Pro rigorously assesses multimodal models' true understanding and reasoning capabilities through a three-step process based on MMMU: (1) filtering out questions answerable by text-only models, (2) augmenting candidate options, and (3) introducing a vision-only input setting where questions are embedded within images. This setting challenges AI to truly "see" and "read" simultaneously, testing a fundamental human cognitive skill of seamlessly integrating visual and textual information. Results show that model performance is substantially lower on MMMU-Pro than on MMMU, ranging from 16.8% to 26.9% across models. We explore the impact of OCR prompts and Chain of Thought (CoT) reasoning, finding that OCR prompts have minimal effect while CoT generally improves performance. MMMU-Pro provides a more rigorous evaluation tool, closely mimicking real-world scenarios and offering valuable directions for future research in multimodal AI.

Abstract of MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark (arXiv:2409.02813, CC0)
90
Models scored in lab cards
82
Launch posts list it, from 21 labs
45
System cards report it, from 17 labs

GPT-5.6 Sol leads with 84.6%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

GPT-5.4 leads with 82.1% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

GPT-5.6 Sol leads with 84.6%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

90 models. Scroll sideways for the rest.

  1. 84.6 GPT-5.6 Sol
  2. 84.1 Kimi K3.1 Pro
  3. 83.6 Gemini 3.5 Flash
  4. 83.2 GPT-5.5
  5. 82.7 Seed2.1 Pro
  6. 82.7 Qwen3.8 Max (0902)
  7. 82.3 Qwen 3.8 Max
  8. 82.0 GPT-5.6 Terra
  9. 81.7 Intern-S2-397B
  10. 81.6 Kimi K3
  11. 81.6 Inkling 2 Small
  12. 81.5 Gemini 3 Deep Think
  13. 81.2 Gemini 3 Flash
  14. 81.2 GPT-5.4
  15. 81.0 Gemini 3 Pro
  16. 80.5 Gemini 3.1 Pro
  17. 80.4 Muse Spark
  18. 79.5 GPT-5.6 Luna
  19. 79.4 Kimi K2.6
  20. 79.0 Qwen3.5 397B-A17B
  21. 79.0 Qwen3.7-Plus
  22. 78.8 Qwen 3.6 Plus
  23. 78.5 Kimi K2.5
  24. 78.4 GPT-5
  25. 78.4 Seed2.0 Lite (0428)
  26. 78.2 Seed2.0 Pro
  27. 78.1 MiniMax-M3
  28. 78.0 GPT-5.4 Mini
  29. 77.9 MiMo-V2.5
  30. 77.3 Claude Opus 4.6
  31. 76.9 Qwen3.5-122B-A10B
  32. 76.9 Gemma 4 31B IT
  33. 76.8 Gemini 3.1 Flash-Lite
  34. 76.8 MiMo-V2-Omni
  35. 76.0 GPT-5.5 Instant
  36. 76.0 Step 5 Preview
  37. 75.8 Qwen3.6 27B
  38. 75.6 Claude Sonnet 4.6
  39. 75.3 Qwen 3.6 35B-A3B
  40. 75.1 Qwen3.5-35B-A3B
  41. 75.0 Qwen3.5-27B
  42. 74.1 GPT-5 mini
  43. 74.0 Inkling-Small
  44. 74.0 Muse Glimmer
  45. 73.9 Qwen3.5-Omni-Plus
  46. 73.8 Gemma 4 26B A4B
  47. 73.5 Inkling
  48. 73.2 Seed1.8
  49. 72.8 Intern-S1-Pro
  50. 70.1 Qwen3.5-9B
  51. 69.5 GPT-5.4 Nano
  52. 69.3 Qwen3-VL-235B-A22B
  53. 69.1 Gemma 4 12B
  54. 68.6 ERNIE 5.0
  55. 68.6 EXAONE 4.5 33B
  56. 67.6 Seed1.5-VL
  57. 66.4 Seed1.6-Thinking
  58. 66.3 Qwen3.5-4B
  59. 66.0 GLM-4.6V
  60. 65.2 GLM-4.5V
  61. 64.1 STEP3-VL-10B
  62. 63.5 Nova 2.0 Pro Preview
  63. 63.0 Command A+
  64. 62.6 GPT-5 nano
  65. 61.8 Nova 2 Lite
  66. 61.4 Nova 2 Omni Preview
  67. 60.0 Mistral Small 4
  68. 59.6 Llama 4 Maverick
  69. 57.9 Magistral Medium 1.0
  70. 57.1 GLM-4.1V-9B-Thinking
  71. 57.0 Qwen3-Omni-30B-A3B
  72. 56.5 DeepSeek V4.1 Flash
  73. 56.5 DeepSeek-V4.1-Flash Base
  74. 54.3 DiffusionGemma 26B A4B
  75. 52.6 Gemma 4 E4B
  76. 52.2 Llama 4 Scout
  77. 51.8 Magistral Small 1.0
  78. 51.1 Qwen2.5-VL-72B
  79. 50.3 Qwen3.5-2B
  80. 49.3 Mistral Small 3.1
  81. 46.3 Kimi-VL-A3B-Thinking-2506
  82. 44.2 Gemma 4 E2B
  83. 42.0 Nemotron 3.5 Bolt E4B
  84. 41.1 Cosmos3-Nano
  85. 38.5 Phi-4-multimodal
  86. 36.6 Qwen2.5-Omni-7B
  87. 31.2 Qwen3.5-0.8B
  88. 30.5 LFM2.5-VL-3B
  89. 28.0 Nemotron Nano 12B v2 VL
  90. 26.4 Cosmos3-Edge
  1. 1 GPT-5.6 Sol 84.6%
  2. 2 Kimi K3.1 Pro 84.1%
  3. 3 Gemini 3.5 Flash 83.6%
  4. 4 GPT-5.5 83.2%
  5. 5 Seed2.1 Pro 82.7%
  6. 6 Qwen3.8 Max (0902) 82.7%
  7. 7 Qwen 3.8 Max 82.3%
  8. 8 GPT-5.6 Terra 82%
  9. 9 Intern-S2-397B 81.7%
  10. 10 Kimi K3 81.6%
  11. 11 Inkling 2 Small 81.6%
  12. 12 Gemini 3 Deep Think 81.5%
  13. 13 Gemini 3 Flash 81.2%
  14. 14 GPT-5.4 81.2%
  15. 15 Gemini 3 Pro 81%
  16. 16 Gemini 3.1 Pro 80.5%
  17. 17 Muse Spark 80.4%
  18. 18 GPT-5.6 Luna 79.5%
  19. 19 Kimi K2.6 79.4%
  20. 20 Qwen3.5 397B-A17B 79%
  21. 21 Qwen3.7-Plus 79%
  22. 22 Qwen 3.6 Plus 78.8%
  23. 23 Kimi K2.5 78.5%
  24. 24 GPT-5 78.4%
  25. 25 Seed2.0 Lite (0428) 78.4%
  26. 26 Seed2.0 Pro 78.2%
  27. 27 MiniMax-M3 78.1%
  28. 28 GPT-5.4 Mini 78%
  29. 29 MiMo-V2.5 77.9%
  30. 30 Claude Opus 4.6 77.3%
  31. 31 Qwen3.5-122B-A10B 76.9%
  32. 32 Gemma 4 31B IT 76.9%
  33. 33 Gemini 3.1 Flash-Lite 76.8%
  34. 34 MiMo-V2-Omni 76.8%
  35. 35 GPT-5.5 Instant 76%
  36. 36 Step 5 Preview 76%
  37. 37 Qwen3.6 27B 75.8%
  38. 38 Claude Sonnet 4.6 75.6%
  39. 39 Qwen 3.6 35B-A3B 75.3%
  40. 40 Qwen3.5-35B-A3B 75.1%
  41. 41 Qwen3.5-27B 75%
  42. 42 GPT-5 mini 74.1%
  43. 43 Inkling-Small 74%
  44. 44 Muse Glimmer 74%
  45. 45 Qwen3.5-Omni-Plus 73.9%
  46. 46 Gemma 4 26B A4B 73.8%
  47. 47 Inkling 73.5%
  48. 48 Seed1.8 73.2%
  49. 49 Intern-S1-Pro 72.8%
  50. 50 Qwen3.5-9B 70.1%
  51. 51 GPT-5.4 Nano 69.5%
  52. 52 Qwen3-VL-235B-A22B 69.3%
  53. 53 Gemma 4 12B 69.1%
  54. 54 ERNIE 5.0 68.6%
  55. 55 EXAONE 4.5 33B 68.6%
  56. 56 Seed1.5-VL 67.6%
  57. 57 Seed1.6-Thinking 66.4%
  58. 58 Qwen3.5-4B 66.3%
  59. 59 GLM-4.6V 66%
  60. 60 GLM-4.5V 65.2%
  61. 61 STEP3-VL-10B 64.1%
  62. 62 Nova 2.0 Pro Preview 63.5%
  63. 63 Command A+ 63%
  64. 64 GPT-5 nano 62.6%
  65. 65 Nova 2 Lite 61.8%
  66. 66 Nova 2 Omni Preview 61.4%
  67. 67 Mistral Small 4 60%
  68. 68 Llama 4 Maverick 59.6%
  69. 69 Magistral Medium 1.0 57.9%
  70. 70 GLM-4.1V-9B-Thinking 57.1%
  71. 71 Qwen3-Omni-30B-A3B 57%
  72. 72 DeepSeek V4.1 Flash 56.5%
  73. 73 DeepSeek-V4.1-Flash Base 56.5%
  74. 74 DiffusionGemma 26B A4B 54.3%
  75. 75 Gemma 4 E4B 52.6%
  76. 76 Llama 4 Scout 52.2%
  77. 77 Magistral Small 1.0 51.8%
  78. 78 Qwen2.5-VL-72B 51.1%
  79. 79 Qwen3.5-2B 50.3%
  80. 80 Mistral Small 3.1 49.3%
  81. 81 Kimi-VL-A3B-Thinking-2506 46.3%
  82. 82 Gemma 4 E2B 44.2%
  83. 83 Nemotron 3.5 Bolt E4B 42%
  84. 84 Cosmos3-Nano 41.1%
  85. 85 Phi-4-multimodal 38.5%
  86. 86 Qwen2.5-Omni-7B 36.6%
  87. 87 Qwen3.5-0.8B 31.2%
  88. 88 LFM2.5-VL-3B 30.5%
  89. 89 Nemotron Nano 12B v2 VL 28%
  90. 90 Cosmos3-Edge 26.4%

Source: Labs’ model cards

57 models. Scroll sideways for the rest.

  1. 82.1 GPT-5.4
  2. 81.0 Gemini 3 Pro
  3. 80.5 Gemini 3.1 Pro
  4. 80.4 GPT-5.2
  5. 80.4 Muse Spark
  6. 79.0 GPT-5.1
  7. 78.4 GPT-5
  8. 77.3 Claude Opus 4.6
  9. 76.9 Gemma 4 31B IT
  10. 76.4 o3
  11. 75.6 Claude Sonnet 4.6
  12. 73.9 Claude Opus 4.5
  13. 73.8 Gemma 4 26B A4B
  14. 70.1 dots.vlm1
  15. 68.9 Claude Sonnet 4.5
  16. 68.1 Qwen3-VL-235B-A22B
  17. 68.0 Gemini 2.5 Pro
  18. 67.6 Seed1.5-VL
  19. 66.4 Seed1.6-Thinking
  20. 65.2 GLM-4.5V
  21. 57.1 GLM-4.1V-9B-Thinking
  22. 55.4 Skywork-R1V3-38B
  23. 52.6 Gemma 4 E4B
  24. 51.9 GPT-4o
  25. 51.5 Claude 3.5 Sonnet
  26. 48.6 InternVL2_5-78B
  27. 46.2 Qwen2-VL-72B-Instruct
  28. 46.2 Qwen2.5-VL-72B
  29. 46.0 InternVL2.5-38B
  30. 44.2 Gemma 4 E2B
  31. 43.5 Gemini 1.5 Pro
  32. 43.3 MiMo-VL-7B-RL
  33. 42.3 MiMo-VL-7B-SFT
  34. 40.0 InternVL2-Llama3-76B
  35. 39.5 Llama 3.2 90B
  36. 38.3 Qwen2.5-VL-7B
  37. 37.6 GPT-4o mini
  38. 37.1 InternVL2.5-26B
  39. 34.3 InternVL2.5-8B
  40. 34.2 InternVL2-40B
  41. 31.6 Qwen2.5-VL-3B
  42. 31.0 LLaVA-OneVision 72B
  43. 29.0 InternVL2-8B
  44. 28.4 Llama 3.2 11B
  45. 27.2 MiniCPM-V-2_6
  46. 25.3 MAmmoTH-VL-8B
  47. 25.1 LLaVA-NeXT-72B
  48. 24.1 LLaVA-OneVision 7B
  49. 23.8 LLaVA-v1.6-34B
  50. 23.7 InternVL2.5-2B
  51. 22.9 Idefics3-8B-Llama3
  52. 19.7 Phi-3.5-vision-instruct
  53. 19.6 MiniCPM-Llama3-V 2.5
  54. 19.4 InternVL2.5-1B
  55. 17.2 llava-v1.6-vicuna-13b
  56. 17.0 llava-v1.6-mistral-7b
  57. 16.1 llava-v1.6-vicuna-7b
  1. 1 GPT-5.4 thinking, with tools 82.1%
  2. 2 Gemini 3 Pro 81%
  3. 3 Gemini 3.1 Pro thinking, high 80.5%
  4. 4 GPT-5.2 thinking, no python 80.4%
  5. 5 Muse Spark thinking 80.4%
  6. 6 GPT-5.1 thinking 79%
  7. 7 GPT-5 thinking 78.4%
  8. 8 Claude Opus 4.6 with tools 77.3%
  9. 9 Gemma 4 31B IT 76.9%
  10. 10 o3 76.4%
  11. 11 Claude Sonnet 4.6 with tools 75.6%
  12. 12 Claude Opus 4.5 73.9%
  13. 13 Gemma 4 26B A4B 73.8%
  14. 14 dots.vlm1 70.1%
  15. 15 Claude Sonnet 4.5 68.9%
  16. 16 Qwen3-VL-235B-A22B 68.1%
  17. 17 Gemini 2.5 Pro May 2025 68%
  18. 18 Seed1.5-VL thinking 67.6%
  19. 19 Seed1.6-Thinking 66.4%
  20. 20 GLM-4.5V thinking 65.2%
  21. 21 GLM-4.1V-9B-Thinking thinking 57.1%
  22. 22 Skywork-R1V3-38B 55.4%
  23. 23 Gemma 4 E4B 52.6%
  24. 24 GPT-4o May 2024 51.9%
  25. 25 Claude 3.5 Sonnet 51.5%
  26. 26 InternVL2_5-78B 48.6%
  27. 27 Qwen2-VL-72B-Instruct 46.2%
  28. 28 Qwen2.5-VL-72B 46.2%
  29. 29 InternVL2.5-38B 46%
  30. 30 Gemma 4 E2B 44.2%
  31. 31 Gemini 1.5 Pro May 2024 43.5%
  32. 32 MiMo-VL-7B-RL 43.3%
  33. 33 MiMo-VL-7B-SFT 42.3%
  34. 34 InternVL2-Llama3-76B 40%
  35. 35 Llama 3.2 90B 39.5%
  36. 36 Qwen2.5-VL-7B 38.3%
  37. 37 GPT-4o mini 37.6%
  38. 38 InternVL2.5-26B 37.1%
  39. 39 InternVL2.5-8B 34.3%
  40. 40 InternVL2-40B 34.2%
  41. 41 Qwen2.5-VL-3B 31.6%
  42. 42 LLaVA-OneVision 72B 31%
  43. 43 InternVL2-8B 29%
  44. 44 Llama 3.2 11B 28.4%
  45. 45 MiniCPM-V-2_6 27.2%
  46. 46 MAmmoTH-VL-8B 25.3%
  47. 47 LLaVA-NeXT-72B 25.1%
  48. 48 LLaVA-OneVision 7B 24.1%
  49. 49 LLaVA-v1.6-34B 23.8%
  50. 50 InternVL2.5-2B 23.7%
  51. 51 Idefics3-8B-Llama3 22.9%
  52. 52 Phi-3.5-vision-instruct 19.7%
  53. 53 MiniCPM-Llama3-V 2.5 19.6%
  54. 54 InternVL2.5-1B 19.4%
  55. 55 llava-v1.6-vicuna-13b 17.2%
  56. 56 llava-v1.6-mistral-7b 17%
  57. 57 llava-v1.6-vicuna-7b 16.1%

Source: Official MMMU-Pro leaderboard

130 models. Scroll sideways for the rest.

  1. 84.6 GPT-5.6 Sol
  2. 84.1 Kimi K3.1 Pro
  3. 83.6 Gemini 3.5 Flash
  4. 83.2 GPT-5.5
  5. 82.7 Seed2.1 Pro
  6. 82.7 Qwen3.8 Max (0902)
  7. 82.3 Qwen 3.8 Max
  8. 82.1 GPT-5.4
  9. 82.0 GPT-5.6 Terra
  10. 81.7 Intern-S2-397B
  11. 81.6 Kimi K3
  12. 81.6 Inkling 2 Small
  13. 81.5 Gemini 3 Deep Think
  14. 81.2 Gemini 3 Flash
  15. 81.0 Gemini 3 Pro
  16. 80.5 Gemini 3.1 Pro
  17. 80.4 GPT-5.2
  18. 80.4 Muse Spark
  19. 79.5 GPT-5.6 Luna
  20. 79.4 Kimi K2.6
  21. 79.0 GPT-5.1
  22. 79.0 Qwen3.5 397B-A17B
  23. 79.0 Qwen3.7-Plus
  24. 78.8 Qwen 3.6 Plus
  25. 78.5 Kimi K2.5
  26. 78.4 GPT-5
  27. 78.4 Seed2.0 Lite (0428)
  28. 78.2 Seed2.0 Pro
  29. 78.1 MiniMax-M3
  30. 78.0 GPT-5.4 Mini
  31. 77.9 MiMo-V2.5
  32. 77.3 Claude Opus 4.6
  33. 76.9 Qwen3.5-122B-A10B
  34. 76.9 Gemma 4 31B IT
  35. 76.8 Gemini 3.1 Flash-Lite
  36. 76.8 MiMo-V2-Omni
  37. 76.4 o3
  38. 76.0 GPT-5.5 Instant
  39. 76.0 Step 5 Preview
  40. 75.8 Qwen3.6 27B
  41. 75.6 Claude Sonnet 4.6
  42. 75.3 Qwen 3.6 35B-A3B
  43. 75.1 Qwen3.5-35B-A3B
  44. 75.0 Qwen3.5-27B
  45. 74.1 GPT-5 mini
  46. 74.0 Inkling-Small
  47. 74.0 Muse Glimmer
  48. 73.9 Claude Opus 4.5
  49. 73.9 Qwen3.5-Omni-Plus
  50. 73.8 Gemma 4 26B A4B
  51. 73.5 Inkling
  52. 73.2 Seed1.8
  53. 72.8 Intern-S1-Pro
  54. 70.1 dots.vlm1
  55. 70.1 Qwen3.5-9B
  56. 69.5 GPT-5.4 Nano
  57. 69.3 Qwen3-VL-235B-A22B
  58. 69.1 Gemma 4 12B
  59. 68.9 Claude Sonnet 4.5
  60. 68.6 ERNIE 5.0
  61. 68.6 EXAONE 4.5 33B
  62. 68.0 Gemini 2.5 Pro
  63. 67.6 Seed1.5-VL
  64. 66.4 Seed1.6-Thinking
  65. 66.3 Qwen3.5-4B
  66. 66.0 GLM-4.6V
  67. 65.2 GLM-4.5V
  68. 64.1 STEP3-VL-10B
  69. 63.5 Nova 2.0 Pro Preview
  70. 63.0 Command A+
  71. 62.6 GPT-5 nano
  72. 61.8 Nova 2 Lite
  73. 61.4 Nova 2 Omni Preview
  74. 60.0 Mistral Small 4
  75. 59.6 Llama 4 Maverick
  76. 57.9 Magistral Medium 1.0
  77. 57.1 GLM-4.1V-9B-Thinking
  78. 57.0 Qwen3-Omni-30B-A3B
  79. 56.5 DeepSeek V4.1 Flash
  80. 56.5 DeepSeek-V4.1-Flash Base
  81. 55.4 Skywork-R1V3-38B
  82. 54.3 DiffusionGemma 26B A4B
  83. 52.6 Gemma 4 E4B
  84. 52.2 Llama 4 Scout
  85. 51.9 GPT-4o
  86. 51.8 Magistral Small 1.0
  87. 51.5 Claude 3.5 Sonnet
  88. 51.1 Qwen2.5-VL-72B
  89. 50.3 Qwen3.5-2B
  90. 49.3 Mistral Small 3.1
  91. 48.6 InternVL2_5-78B
  92. 46.3 Kimi-VL-A3B-Thinking-2506
  93. 46.2 Qwen2-VL-72B-Instruct
  94. 46.0 InternVL2.5-38B
  95. 44.2 Gemma 4 E2B
  96. 43.5 Gemini 1.5 Pro
  97. 43.3 MiMo-VL-7B-RL
  98. 42.3 MiMo-VL-7B-SFT
  99. 42.0 Nemotron 3.5 Bolt E4B
  100. 41.1 Cosmos3-Nano
  101. 40.0 InternVL2-Llama3-76B
  102. 39.5 Llama 3.2 90B
  103. 38.5 Phi-4-multimodal
  104. 38.3 Qwen2.5-VL-7B
  105. 37.6 GPT-4o mini
  106. 37.1 InternVL2.5-26B
  107. 36.6 Qwen2.5-Omni-7B
  108. 34.3 InternVL2.5-8B
  109. 34.2 InternVL2-40B
  110. 31.6 Qwen2.5-VL-3B
  111. 31.2 Qwen3.5-0.8B
  112. 31.0 LLaVA-OneVision 72B
  113. 30.5 LFM2.5-VL-3B
  114. 29.0 InternVL2-8B
  115. 28.4 Llama 3.2 11B
  116. 28.0 Nemotron Nano 12B v2 VL
  117. 27.2 MiniCPM-V-2_6
  118. 26.4 Cosmos3-Edge
  119. 25.3 MAmmoTH-VL-8B
  120. 25.1 LLaVA-NeXT-72B
  121. 24.1 LLaVA-OneVision 7B
  122. 23.8 LLaVA-v1.6-34B
  123. 23.7 InternVL2.5-2B
  124. 22.9 Idefics3-8B-Llama3
  125. 19.7 Phi-3.5-vision-instruct
  126. 19.6 MiniCPM-Llama3-V 2.5
  127. 19.4 InternVL2.5-1B
  128. 17.2 llava-v1.6-vicuna-13b
  129. 17.0 llava-v1.6-mistral-7b
  130. 16.1 llava-v1.6-vicuna-7b
  1. 1 GPT-5.6 Sol 84.6%
  2. 2 Kimi K3.1 Pro 84.1%
  3. 3 Gemini 3.5 Flash 83.6%
  4. 4 GPT-5.5 83.2%
  5. 5 Seed2.1 Pro 82.7%
  6. 6 Qwen3.8 Max (0902) 82.7%
  7. 7 Qwen 3.8 Max 82.3%
  8. 8 GPT-5.4 thinking, with tools 82.1%
  9. 9 GPT-5.6 Terra 82%
  10. 10 Intern-S2-397B 81.7%
  11. 11 Kimi K3 81.6%
  12. 12 Inkling 2 Small 81.6%
  13. 13 Gemini 3 Deep Think 81.5%
  14. 14 Gemini 3 Flash 81.2%
  15. 15 Gemini 3 Pro 81%
  16. 16 Gemini 3.1 Pro thinking, high 80.5%
  17. 17 GPT-5.2 thinking, no python 80.4%
  18. 18 Muse Spark thinking 80.4%
  19. 19 GPT-5.6 Luna 79.5%
  20. 20 Kimi K2.6 79.4%
  21. 21 GPT-5.1 thinking 79%
  22. 22 Qwen3.5 397B-A17B 79%
  23. 23 Qwen3.7-Plus 79%
  24. 24 Qwen 3.6 Plus 78.8%
  25. 25 Kimi K2.5 78.5%
  26. 26 GPT-5 thinking 78.4%
  27. 27 Seed2.0 Lite (0428) 78.4%
  28. 28 Seed2.0 Pro 78.2%
  29. 29 MiniMax-M3 78.1%
  30. 30 GPT-5.4 Mini 78%
  31. 31 MiMo-V2.5 77.9%
  32. 32 Claude Opus 4.6 with tools 77.3%
  33. 33 Qwen3.5-122B-A10B 76.9%
  34. 34 Gemma 4 31B IT 76.9%
  35. 35 Gemini 3.1 Flash-Lite 76.8%
  36. 36 MiMo-V2-Omni 76.8%
  37. 37 o3 76.4%
  38. 38 GPT-5.5 Instant 76%
  39. 39 Step 5 Preview 76%
  40. 40 Qwen3.6 27B 75.8%
  41. 41 Claude Sonnet 4.6 with tools 75.6%
  42. 42 Qwen 3.6 35B-A3B 75.3%
  43. 43 Qwen3.5-35B-A3B 75.1%
  44. 44 Qwen3.5-27B 75%
  45. 45 GPT-5 mini 74.1%
  46. 46 Inkling-Small 74%
  47. 47 Muse Glimmer 74%
  48. 48 Claude Opus 4.5 73.9%
  49. 49 Qwen3.5-Omni-Plus 73.9%
  50. 50 Gemma 4 26B A4B 73.8%
  51. 51 Inkling 73.5%
  52. 52 Seed1.8 73.2%
  53. 53 Intern-S1-Pro 72.8%
  54. 54 dots.vlm1 70.1%
  55. 55 Qwen3.5-9B 70.1%
  56. 56 GPT-5.4 Nano 69.5%
  57. 57 Qwen3-VL-235B-A22B 69.3%
  58. 58 Gemma 4 12B 69.1%
  59. 59 Claude Sonnet 4.5 68.9%
  60. 60 ERNIE 5.0 68.6%
  61. 61 EXAONE 4.5 33B 68.6%
  62. 62 Gemini 2.5 Pro May 2025 68%
  63. 63 Seed1.5-VL thinking 67.6%
  64. 64 Seed1.6-Thinking 66.4%
  65. 65 Qwen3.5-4B 66.3%
  66. 66 GLM-4.6V 66%
  67. 67 GLM-4.5V thinking 65.2%
  68. 68 STEP3-VL-10B 64.1%
  69. 69 Nova 2.0 Pro Preview 63.5%
  70. 70 Command A+ 63%
  71. 71 GPT-5 nano 62.6%
  72. 72 Nova 2 Lite 61.8%
  73. 73 Nova 2 Omni Preview 61.4%
  74. 74 Mistral Small 4 60%
  75. 75 Llama 4 Maverick 59.6%
  76. 76 Magistral Medium 1.0 57.9%
  77. 77 GLM-4.1V-9B-Thinking thinking 57.1%
  78. 78 Qwen3-Omni-30B-A3B 57%
  79. 79 DeepSeek V4.1 Flash 56.5%
  80. 80 DeepSeek-V4.1-Flash Base 56.5%
  81. 81 Skywork-R1V3-38B 55.4%
  82. 82 DiffusionGemma 26B A4B 54.3%
  83. 83 Gemma 4 E4B 52.6%
  84. 84 Llama 4 Scout 52.2%
  85. 85 GPT-4o May 2024 51.9%
  86. 86 Magistral Small 1.0 51.8%
  87. 87 Claude 3.5 Sonnet 51.5%
  88. 88 Qwen2.5-VL-72B 51.1%
  89. 89 Qwen3.5-2B 50.3%
  90. 90 Mistral Small 3.1 49.3%
  91. 91 InternVL2_5-78B 48.6%
  92. 92 Kimi-VL-A3B-Thinking-2506 46.3%
  93. 93 Qwen2-VL-72B-Instruct 46.2%
  94. 94 InternVL2.5-38B 46%
  95. 95 Gemma 4 E2B 44.2%
  96. 96 Gemini 1.5 Pro May 2024 43.5%
  97. 97 MiMo-VL-7B-RL 43.3%
  98. 98 MiMo-VL-7B-SFT 42.3%
  99. 99 Nemotron 3.5 Bolt E4B 42%
  100. 100 Cosmos3-Nano 41.1%
  101. 101 InternVL2-Llama3-76B 40%
  102. 102 Llama 3.2 90B 39.5%
  103. 103 Phi-4-multimodal 38.5%
  104. 104 Qwen2.5-VL-7B 38.3%
  105. 105 GPT-4o mini 37.6%
  106. 106 InternVL2.5-26B 37.1%
  107. 107 Qwen2.5-Omni-7B 36.6%
  108. 108 InternVL2.5-8B 34.3%
  109. 109 InternVL2-40B 34.2%
  110. 110 Qwen2.5-VL-3B 31.6%
  111. 111 Qwen3.5-0.8B 31.2%
  112. 112 LLaVA-OneVision 72B 31%
  113. 113 LFM2.5-VL-3B 30.5%
  114. 114 InternVL2-8B 29%
  115. 115 Llama 3.2 11B 28.4%
  116. 116 Nemotron Nano 12B v2 VL 28%
  117. 117 MiniCPM-V-2_6 27.2%
  118. 118 Cosmos3-Edge 26.4%
  119. 119 MAmmoTH-VL-8B 25.3%
  120. 120 LLaVA-NeXT-72B 25.1%
  121. 121 LLaVA-OneVision 7B 24.1%
  122. 122 LLaVA-v1.6-34B 23.8%
  123. 123 InternVL2.5-2B 23.7%
  124. 124 Idefics3-8B-Llama3 22.9%
  125. 125 Phi-3.5-vision-instruct 19.7%
  126. 126 MiniCPM-Llama3-V 2.5 19.6%
  127. 127 InternVL2.5-1B 19.4%
  128. 128 llava-v1.6-vicuna-13b 17.2%
  129. 129 llava-v1.6-mistral-7b 17%
  130. 130 llava-v1.6-vicuna-7b 16.1%

Sources: Labs’ model cards and official MMMU-Pro leaderboard

#ModelScoreSettingDateSource
1 GPT-5.6 Sol 84.6% Jul 2026 Lab model card
2 Kimi K3.1 Pro 84.1% Oct 2026 Lab model card
3 Gemini 3.5 Flash 83.6% May 2026 Lab model card
4 GPT-5.5 83.2% Apr 2026 Lab model card
5 Seed2.1 Pro 82.7% Jun 2026 Lab model card
6 Qwen3.8 Max (0902) 82.7% Sep 2026 Lab model card
7 Qwen 3.8 Max 82.3% Aug 2026 Lab model card
8 GPT-5.6 Terra 82% Jul 2026 Lab model card
9 Intern-S2-397B 81.7% Sep 2026 Lab model card
10 Kimi K3 81.6% Jul 2026 Lab model card
11 Inkling 2 Small 81.6% Nov 2026 Lab model card
12 Gemini 3 Deep Think 81.5% Feb 2026 Lab model card
13 Gemini 3 Flash 81.2% Dec 2025 Lab model card
14 GPT-5.4 81.2% Mar 2026 Lab model card
15 Gemini 3 Pro 81% Nov 2025 Lab model card
16 Gemini 3.1 Pro 80.5% Feb 2026 Lab model card
17 Muse Spark 80.4% Apr 2026 Lab model card
18 GPT-5.6 Luna 79.5% Jul 2026 Lab model card
19 Kimi K2.6 79.4% Apr 2026 Lab model card
20 Qwen3.5 397B-A17B 79% Feb 2026 Lab model card
21 Qwen3.7-Plus 79% Jun 2026 Lab model card
22 Qwen 3.6 Plus 78.8% Apr 2026 Lab model card
23 Kimi K2.5 78.5% Jan 2026 Lab model card
24 GPT-5 78.4% Aug 2025 Lab model card
25 Seed2.0 Lite (0428) 78.4% Apr 2026 Lab model card
26 Seed2.0 Pro 78.2% Feb 2026 Lab model card
27 MiniMax-M3 78.1% Jun 2026 Lab model card
28 GPT-5.4 Mini 78% Mar 2026 Lab model card
29 MiMo-V2.5 77.9% Apr 2026 Lab model card
30 Claude Opus 4.6 77.3% Feb 2026 Lab model card
31 Qwen3.5-122B-A10B 76.9% Feb 2026 Lab model card
32 Gemma 4 31B IT 76.9% Apr 2026 Lab model card
33 Gemini 3.1 Flash-Lite 76.8% Mar 2026 Lab model card
34 MiMo-V2-Omni 76.8% Mar 2026 Lab model card
35 GPT-5.5 Instant 76% May 2026 Lab model card
36 Step 5 Preview 76% Sep 2026 Lab model card
37 Qwen3.6 27B 75.8% Apr 2026 Lab model card
38 Claude Sonnet 4.6 75.6% Feb 2026 Lab model card
39 Qwen 3.6 35B-A3B 75.3% Apr 2026 Lab model card
40 Qwen3.5-35B-A3B 75.1% Feb 2026 Lab model card
41 Qwen3.5-27B 75% Feb 2026 Lab model card
42 GPT-5 mini 74.1% Aug 2025 Lab model card
43 Inkling-Small 74% Jul 2026 Lab model card
44 Muse Glimmer 74% Aug 2026 Lab model card
45 Qwen3.5-Omni-Plus 73.9% Mar 2026 Lab model card
46 Gemma 4 26B A4B 73.8% Apr 2026 Lab model card
47 Inkling 73.5% Jul 2026 Lab model card
48 Seed1.8 73.2% Dec 2025 Lab model card
49 Intern-S1-Pro 72.8% Feb 2026 Lab model card
50 Qwen3.5-9B 70.1% Mar 2026 Lab model card
51 GPT-5.4 Nano 69.5% Mar 2026 Lab model card
52 Qwen3-VL-235B-A22B 69.3% Sep 2025 Lab model card
53 Gemma 4 12B 69.1% Jun 2026 Lab model card
54 ERNIE 5.0 68.6% Feb 2026 Lab model card
55 EXAONE 4.5 33B 68.6% Apr 2026 Lab model card
56 Seed1.5-VL 67.6% May 2025 Lab model card
57 Seed1.6-Thinking 66.4% Jun 2025 Lab model card
58 Qwen3.5-4B 66.3% Mar 2026 Lab model card
59 GLM-4.6V 66% Dec 2025 Lab model card
60 GLM-4.5V 65.2% Aug 2025 Lab model card
61 STEP3-VL-10B 64.1% Jan 2026 Lab model card
62 Nova 2.0 Pro Preview 63.5% Dec 2025 Lab model card
63 Command A+ 63% May 2026 Lab model card
64 GPT-5 nano 62.6% Aug 2025 Lab model card
65 Nova 2 Lite 61.8% Dec 2025 Lab model card
66 Nova 2 Omni Preview 61.4% Dec 2025 Lab model card
67 Mistral Small 4 60% Mar 2026 Lab model card
68 Llama 4 Maverick 59.6% Apr 2025 Lab model card
69 Magistral Medium 1.0 57.9% Jun 2025 Lab model card
70 GLM-4.1V-9B-Thinking 57.1% Jul 2025 Lab model card
71 Qwen3-Omni-30B-A3B 57% Sep 2025 Lab model card
72 DeepSeek V4.1 Flash 56.5% Sep 2026 Lab model card
73 DeepSeek-V4.1-Flash Base 56.5% Sep 2026 Lab model card
74 DiffusionGemma 26B A4B 54.3% Jun 2026 Lab model card
75 Gemma 4 E4B 52.6% Apr 2026 Lab model card
76 Llama 4 Scout 52.2% Apr 2025 Lab model card
77 Magistral Small 1.0 51.8% Jun 2025 Lab model card
78 Qwen2.5-VL-72B 51.1% Jan 2025 Lab model card
79 Qwen3.5-2B 50.3% Mar 2026 Lab model card
80 Mistral Small 3.1 49.3% Mar 2025 Lab model card
81 Kimi-VL-A3B-Thinking-2506 46.3% Jun 2025 Lab model card
82 Gemma 4 E2B 44.2% Apr 2026 Lab model card
83 Nemotron 3.5 Bolt E4B 42% Oct 2026 Lab model card
84 Cosmos3-Nano 41.1% May 2026 Lab model card
85 Phi-4-multimodal 38.5% Feb 2025 Lab model card
86 Qwen2.5-Omni-7B 36.6% Mar 2025 Lab model card
87 Qwen3.5-0.8B 31.2% Mar 2026 Lab model card
88 LFM2.5-VL-3B 30.5% Aug 2026 Lab model card
89 Nemotron Nano 12B v2 VL 28% Nov 2025 Lab model card
90 Cosmos3-Edge 26.4% May 2026 Lab model card
#ModelScoreSettingDateSource
1 GPT-5.4 82.1% thinking, with tools Mar 2026 Official leaderboard
2 Gemini 3 Pro 81% Nov 2025 Official leaderboard
3 Gemini 3.1 Pro 80.5% thinking, high Feb 2026 Official leaderboard
4 GPT-5.2 80.4% thinking, no python Dec 2025 Official leaderboard
5 Muse Spark 80.4% thinking Apr 2026 Official leaderboard
6 GPT-5.1 79% thinking Nov 2025 Official leaderboard
7 GPT-5 78.4% thinking Aug 2025 Official leaderboard
8 Claude Opus 4.6 77.3% with tools Feb 2026 Official leaderboard
9 Gemma 4 31B IT 76.9% Apr 2026 Official leaderboard
10 o3 76.4% Apr 2025 Official leaderboard
11 Claude Sonnet 4.6 75.6% with tools Feb 2026 Official leaderboard
12 Claude Opus 4.5 73.9% Nov 2025 Official leaderboard
13 Gemma 4 26B A4B 73.8% Apr 2026 Official leaderboard
14 dots.vlm1 70.1% Aug 2025 Official leaderboard
15 Claude Sonnet 4.5 68.9% Sep 2025 Official leaderboard
16 Qwen3-VL-235B-A22B 68.1% Sep 2025 Official leaderboard
17 Gemini 2.5 Pro May 2025 68% May 2025 Official leaderboard
18 Seed1.5-VL 67.6% thinking May 2025 Official leaderboard
19 Seed1.6-Thinking 66.4% Jun 2025 Official leaderboard
20 GLM-4.5V 65.2% thinking Aug 2025 Official leaderboard
21 GLM-4.1V-9B-Thinking 57.1% thinking Jul 2025 Official leaderboard
22 Skywork-R1V3-38B 55.4% Jul 2025 Official leaderboard
23 Gemma 4 E4B 52.6% Mar 2026 Official leaderboard
24 GPT-4o May 2024 51.9% May 2024 Official leaderboard
25 Claude 3.5 Sonnet 51.5% Jun 2024 Official leaderboard
26 InternVL2_5-78B 48.6% Dec 2024 Official leaderboard
27 Qwen2-VL-72B-Instruct 46.2% Aug 2024 Official leaderboard
28 Qwen2.5-VL-72B 46.2% Jan 2025 Official leaderboard
29 InternVL2.5-38B 46% Dec 2024 Official leaderboard
30 Gemma 4 E2B 44.2% Mar 2026 Official leaderboard
31 Gemini 1.5 Pro May 2024 43.5% May 2024 Official leaderboard
32 MiMo-VL-7B-RL 43.3% May 2025 Official leaderboard
33 MiMo-VL-7B-SFT 42.3% May 2025 Official leaderboard
34 InternVL2-Llama3-76B 40% Jul 2024 Official leaderboard
35 Llama 3.2 90B 39.5% Sep 2024 Official leaderboard
36 Qwen2.5-VL-7B 38.3% Jan 2025 Official leaderboard
37 GPT-4o mini 37.6% Jul 2024 Official leaderboard
38 InternVL2.5-26B 37.1% Dec 2024 Official leaderboard
39 InternVL2.5-8B 34.3% Dec 2024 Official leaderboard
40 InternVL2-40B 34.2% Jul 2024 Official leaderboard
41 Qwen2.5-VL-3B 31.6% Jan 2025 Official leaderboard
42 LLaVA-OneVision 72B 31% Aug 2024 Official leaderboard
43 InternVL2-8B 29% Jul 2024 Official leaderboard
44 Llama 3.2 11B 28.4% Sep 2024 Official leaderboard
45 MiniCPM-V-2_6 27.2% Aug 2024 Official leaderboard
46 MAmmoTH-VL-8B 25.3% Nov 2024 Official leaderboard
47 LLaVA-NeXT-72B 25.1% May 2024 Official leaderboard
48 LLaVA-OneVision 7B 24.1% Aug 2024 Official leaderboard
49 LLaVA-v1.6-34B 23.8% Jan 2024 Official leaderboard
50 InternVL2.5-2B 23.7% Dec 2024 Official leaderboard
51 Idefics3-8B-Llama3 22.9% Aug 2024 Official leaderboard
52 Phi-3.5-vision-instruct 19.7% Aug 2024 Official leaderboard
53 MiniCPM-Llama3-V 2.5 19.6% May 2024 Official leaderboard
54 InternVL2.5-1B 19.4% Dec 2024 Official leaderboard
55 llava-v1.6-vicuna-13b 17.2% Jan 2024 Official leaderboard
56 llava-v1.6-mistral-7b 17% Jan 2024 Official leaderboard
57 llava-v1.6-vicuna-7b 16.1% Jan 2024 Official leaderboard
#ModelScoreSettingDateSource
1 GPT-5.6 Sol 84.6% Jul 2026 Lab model card
2 Kimi K3.1 Pro 84.1% Oct 2026 Lab model card
3 Gemini 3.5 Flash 83.6% May 2026 Lab model card
4 GPT-5.5 83.2% Apr 2026 Lab model card
5 Seed2.1 Pro 82.7% Jun 2026 Lab model card
6 Qwen3.8 Max (0902) 82.7% Sep 2026 Lab model card
7 Qwen 3.8 Max 82.3% Aug 2026 Lab model card
8 GPT-5.4 82.1% thinking, with tools Mar 2026 Official leaderboard
9 GPT-5.6 Terra 82% Jul 2026 Lab model card
10 Intern-S2-397B 81.7% Sep 2026 Lab model card
11 Kimi K3 81.6% Jul 2026 Lab model card
12 Inkling 2 Small 81.6% Nov 2026 Lab model card
13 Gemini 3 Deep Think 81.5% Feb 2026 Lab model card
14 Gemini 3 Flash 81.2% Dec 2025 Lab model card
15 Gemini 3 Pro 81% Nov 2025 Official leaderboard
16 Gemini 3.1 Pro 80.5% thinking, high Feb 2026 Official leaderboard
17 GPT-5.2 80.4% thinking, no python Dec 2025 Official leaderboard
18 Muse Spark 80.4% thinking Apr 2026 Official leaderboard
19 GPT-5.6 Luna 79.5% Jul 2026 Lab model card
20 Kimi K2.6 79.4% Apr 2026 Lab model card
21 GPT-5.1 79% thinking Nov 2025 Official leaderboard
22 Qwen3.5 397B-A17B 79% Feb 2026 Lab model card
23 Qwen3.7-Plus 79% Jun 2026 Lab model card
24 Qwen 3.6 Plus 78.8% Apr 2026 Lab model card
25 Kimi K2.5 78.5% Jan 2026 Lab model card
26 GPT-5 78.4% thinking Aug 2025 Official leaderboard
27 Seed2.0 Lite (0428) 78.4% Apr 2026 Lab model card
28 Seed2.0 Pro 78.2% Feb 2026 Lab model card
29 MiniMax-M3 78.1% Jun 2026 Lab model card
30 GPT-5.4 Mini 78% Mar 2026 Lab model card
31 MiMo-V2.5 77.9% Apr 2026 Lab model card
32 Claude Opus 4.6 77.3% with tools Feb 2026 Official leaderboard
33 Qwen3.5-122B-A10B 76.9% Feb 2026 Lab model card
34 Gemma 4 31B IT 76.9% Apr 2026 Official leaderboard
35 Gemini 3.1 Flash-Lite 76.8% Mar 2026 Lab model card
36 MiMo-V2-Omni 76.8% Mar 2026 Lab model card
37 o3 76.4% Apr 2025 Official leaderboard
38 GPT-5.5 Instant 76% May 2026 Lab model card
39 Step 5 Preview 76% Sep 2026 Lab model card
40 Qwen3.6 27B 75.8% Apr 2026 Lab model card
41 Claude Sonnet 4.6 75.6% with tools Feb 2026 Official leaderboard
42 Qwen 3.6 35B-A3B 75.3% Apr 2026 Lab model card
43 Qwen3.5-35B-A3B 75.1% Feb 2026 Lab model card
44 Qwen3.5-27B 75% Feb 2026 Lab model card
45 GPT-5 mini 74.1% Aug 2025 Lab model card
46 Inkling-Small 74% Jul 2026 Lab model card
47 Muse Glimmer 74% Aug 2026 Lab model card
48 Claude Opus 4.5 73.9% Nov 2025 Official leaderboard
49 Qwen3.5-Omni-Plus 73.9% Mar 2026 Lab model card
50 Gemma 4 26B A4B 73.8% Apr 2026 Official leaderboard
51 Inkling 73.5% Jul 2026 Lab model card
52 Seed1.8 73.2% Dec 2025 Lab model card
53 Intern-S1-Pro 72.8% Feb 2026 Lab model card
54 dots.vlm1 70.1% Aug 2025 Official leaderboard
55 Qwen3.5-9B 70.1% Mar 2026 Lab model card
56 GPT-5.4 Nano 69.5% Mar 2026 Lab model card
57 Qwen3-VL-235B-A22B 69.3% Sep 2025 Lab model card
58 Gemma 4 12B 69.1% Jun 2026 Lab model card
59 Claude Sonnet 4.5 68.9% Sep 2025 Official leaderboard
60 ERNIE 5.0 68.6% Feb 2026 Lab model card
61 EXAONE 4.5 33B 68.6% Apr 2026 Lab model card
62 Gemini 2.5 Pro May 2025 68% May 2025 Official leaderboard
63 Seed1.5-VL 67.6% thinking May 2025 Official leaderboard
64 Seed1.6-Thinking 66.4% Jun 2025 Official leaderboard
65 Qwen3.5-4B 66.3% Mar 2026 Lab model card
66 GLM-4.6V 66% Dec 2025 Lab model card
67 GLM-4.5V 65.2% thinking Aug 2025 Official leaderboard
68 STEP3-VL-10B 64.1% Jan 2026 Lab model card
69 Nova 2.0 Pro Preview 63.5% Dec 2025 Lab model card
70 Command A+ 63% May 2026 Lab model card
71 GPT-5 nano 62.6% Aug 2025 Lab model card
72 Nova 2 Lite 61.8% Dec 2025 Lab model card
73 Nova 2 Omni Preview 61.4% Dec 2025 Lab model card
74 Mistral Small 4 60% Mar 2026 Lab model card
75 Llama 4 Maverick 59.6% Apr 2025 Lab model card
76 Magistral Medium 1.0 57.9% Jun 2025 Lab model card
77 GLM-4.1V-9B-Thinking 57.1% thinking Jul 2025 Official leaderboard
78 Qwen3-Omni-30B-A3B 57% Sep 2025 Lab model card
79 DeepSeek V4.1 Flash 56.5% Sep 2026 Lab model card
80 DeepSeek-V4.1-Flash Base 56.5% Sep 2026 Lab model card
81 Skywork-R1V3-38B 55.4% Jul 2025 Official leaderboard
82 DiffusionGemma 26B A4B 54.3% Jun 2026 Lab model card
83 Gemma 4 E4B 52.6% Mar 2026 Official leaderboard
84 Llama 4 Scout 52.2% Apr 2025 Lab model card
85 GPT-4o May 2024 51.9% May 2024 Official leaderboard
86 Magistral Small 1.0 51.8% Jun 2025 Lab model card
87 Claude 3.5 Sonnet 51.5% Jun 2024 Official leaderboard
88 Qwen2.5-VL-72B 51.1% Jan 2025 Lab model card
89 Qwen3.5-2B 50.3% Mar 2026 Lab model card
90 Mistral Small 3.1 49.3% Mar 2025 Lab model card
91 InternVL2_5-78B 48.6% Dec 2024 Official leaderboard
92 Kimi-VL-A3B-Thinking-2506 46.3% Jun 2025 Lab model card
93 Qwen2-VL-72B-Instruct 46.2% Aug 2024 Official leaderboard
94 InternVL2.5-38B 46% Dec 2024 Official leaderboard
95 Gemma 4 E2B 44.2% Mar 2026 Official leaderboard
96 Gemini 1.5 Pro May 2024 43.5% May 2024 Official leaderboard
97 MiMo-VL-7B-RL 43.3% May 2025 Official leaderboard
98 MiMo-VL-7B-SFT 42.3% May 2025 Official leaderboard
99 Nemotron 3.5 Bolt E4B 42% Oct 2026 Lab model card
100 Cosmos3-Nano 41.1% May 2026 Lab model card
101 InternVL2-Llama3-76B 40% Jul 2024 Official leaderboard
102 Llama 3.2 90B 39.5% Sep 2024 Official leaderboard
103 Phi-4-multimodal 38.5% Feb 2025 Lab model card
104 Qwen2.5-VL-7B 38.3% Jan 2025 Official leaderboard
105 GPT-4o mini 37.6% Jul 2024 Official leaderboard
106 InternVL2.5-26B 37.1% Dec 2024 Official leaderboard
107 Qwen2.5-Omni-7B 36.6% Mar 2025 Lab model card
108 InternVL2.5-8B 34.3% Dec 2024 Official leaderboard
109 InternVL2-40B 34.2% Jul 2024 Official leaderboard
110 Qwen2.5-VL-3B 31.6% Jan 2025 Official leaderboard
111 Qwen3.5-0.8B 31.2% Mar 2026 Lab model card
112 LLaVA-OneVision 72B 31% Aug 2024 Official leaderboard
113 LFM2.5-VL-3B 30.5% Aug 2026 Lab model card
114 InternVL2-8B 29% Jul 2024 Official leaderboard
115 Llama 3.2 11B 28.4% Sep 2024 Official leaderboard
116 Nemotron Nano 12B v2 VL 28% Nov 2025 Lab model card
117 MiniCPM-V-2_6 27.2% Aug 2024 Official leaderboard
118 Cosmos3-Edge 26.4% May 2026 Lab model card
119 MAmmoTH-VL-8B 25.3% Nov 2024 Official leaderboard
120 LLaVA-NeXT-72B 25.1% May 2024 Official leaderboard
121 LLaVA-OneVision 7B 24.1% Aug 2024 Official leaderboard
122 LLaVA-v1.6-34B 23.8% Jan 2024 Official leaderboard
123 InternVL2.5-2B 23.7% Dec 2024 Official leaderboard
124 Idefics3-8B-Llama3 22.9% Aug 2024 Official leaderboard
125 Phi-3.5-vision-instruct 19.7% Aug 2024 Official leaderboard
126 MiniCPM-Llama3-V 2.5 19.6% May 2024 Official leaderboard
127 InternVL2.5-1B 19.4% Dec 2024 Official leaderboard
128 llava-v1.6-vicuna-13b 17.2% Jan 2024 Official leaderboard
129 llava-v1.6-mistral-7b 17% Jan 2024 Official leaderboard
130 llava-v1.6-vicuna-7b 16.1% Jan 2024 Official leaderboard

The benchmark's own leaderboard has 61 entries, 57 of them models we track. Switch the scores above to rank them together.

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
GPT-5.4
thinking, n…
82.1%
GPT-5.2
thinking, n…
80.4%
GPT-5.1
thinking
79%
GPT-5
non-thinking thinking
78.4%
Claude Opus 4.6
no tools with tools
77.3%
Claude Sonnet 4.6
no tools
75.6%
Claude Sonnet 4.5
no tools
68.9%
Seed1.5-VL
thinking
67.6%

Source: Official MMMU-Pro leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GPT-5.4 thinking, with tools 82.1%
GPT-5.4 thinking, no tools 81.2%
Gemini 3 Pro 81%
Gemini 3.1 Pro thinking, high 80.5%
GPT-5.2 thinking, no python 80.4%
Muse Spark thinking 80.4%
GPT-5.2 thinking, no tools 79.5%
GPT-5.1 thinking 79%
GPT-5 thinking 78.4%
Claude Opus 4.6 with tools 77.3%
Gemma 4 31B IT 76.9%
o3 76.4%
GPT-5.1 76%
Claude Sonnet 4.6 with tools 75.6%
Claude Sonnet 4.6 no tools 74.5%
Claude Opus 4.5 73.9%
Claude Opus 4.6 no tools 73.9%
Gemma 4 26B A4B 73.8%
dots.vlm1 70.1%
Claude Sonnet 4.5 68.9%
Qwen3-VL-235B-A22B 68.1%
Gemini 2.5 Pro May 2025 68%
Seed1.5-VL thinking 67.6%
Seed1.6-Thinking 66.4%
GLM-4.5V thinking 65.2%
Claude Sonnet 4.5 no tools 63.4%
GPT-5 non-thinking 62.7%
Seed1.5-VL 59.9%
GLM-4.1V-9B-Thinking thinking 57.1%
Skywork-R1V3-38B 55.4%
Gemma 4 E4B 52.6%
GPT-4o May 2024 51.9%
Claude 3.5 Sonnet 51.5%
InternVL2_5-78B 48.6%
Qwen2-VL-72B-Instruct 46.2%
Qwen2.5-VL-72B 46.2%
InternVL2.5-38B 46%
Gemma 4 E2B 44.2%
Gemini 1.5 Pro May 2024 43.5%
MiMo-VL-7B-RL 43.3%
MiMo-VL-7B-SFT 42.3%
InternVL2-Llama3-76B 40%
Llama 3.2 90B 39.5%
Qwen2.5-VL-7B 38.3%
GPT-4o mini 37.6%
InternVL2.5-26B 37.1%
InternVL2.5-8B 34.3%
InternVL2-40B 34.2%
Qwen2.5-VL-3B 31.6%
LLaVA-OneVision 72B 31%
InternVL2-8B 29%
Llama 3.2 11B 28.4%
MiniCPM-V-2_6 27.2%
MAmmoTH-VL-8B 25.3%
LLaVA-NeXT-72B 25.1%
LLaVA-OneVision 7B 24.1%
LLaVA-v1.6-34B 23.8%
InternVL2.5-2B 23.7%
Idefics3-8B-Llama3 22.9%
Phi-3.5-vision-instruct 19.7%
MiniCPM-Llama3-V 2.5 19.6%
InternVL2.5-1B 19.4%
llava-v1.6-vicuna-13b 17.2%
llava-v1.6-mistral-7b 17%
llava-v1.6-vicuna-7b 16.1%

10 models have held number one in labs’ own numbers since its release in September 2024 6 models have held number one across all independent runs since its release in September 2024 12 models have held number one with claims included since its release in September 2024

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

OpenAI Google DeepMind Qwen Meta ByteDance Seed
Own leaderboard Not tested since Apr 2026
GPT-5.4 82.1%
Lab claims 10 leaders
GPT-5.6 Sol 84.6%

Sources: Official MMMU-Pro leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) GPT-5.6 Sol Gemini 3 Flash o3 Llama 4 Maverick GPT-4o LLaVA-NeXT-72B 0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) GPT-5.6 Sol o3

Sources: Official MMMU-Pro leaderboard and labs’ model cards

12 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Qwen
Microsoft
Mistral AI
Meta
ByteDance Seed
Moonshot AI
Z.ai
OpenAI
Google DeepMind
Amazon
Shanghai AI Lab
Anthropic
Baidu
Xiaomi Corp
LG AI Research
Cohere
MiniMax
Thinking Machines
Liquid AI
DeepSeek
StepFun
DateLaunch postScoreAs the post writes it
Nov 5, 2026 Inkling 2 Small 81.6 MMMU Pro (Standard 10)
Oct 26, 2026 GPT-6.1 Terra MMMU-Pro (chart)
Oct 24, 2026 Kimi K3.1 Pro 84.1 MMMU-Pro
Sep 20, 2026 Step 5 Preview 76 MMMU-Pro
Sep 13, 2026 Intern-S2-397B 81.7 MMMU Pro
Sep 10, 2026 DeepSeek V4.1 Flash 56.5 MMMU-Pro (EM)
Sep 2, 2026 Qwen3.8 Max (0902) 82.7 STEM visual reasoning, MMMU-Pro
Aug 12, 2026 LFM2.5-VL-3B 30.5 MMMU-Pro
Aug 10, 2026 Muse Glimmer 74 MMMU Pro
Aug 3, 2026 Qwen 3.8 Max 82.3 MMMU-Pro
Jul 30, 2026 Inkling-Small 74 MMMU Pro Standard 10
Jul 16, 2026 Kimi K3 81.6 MMMU-Pro (without python)
Jul 15, 2026 Inkling 73.5 MMMU Pro Standard 10 (Artificial Analysis)
Jul 9, 2026 GPT-5.6 Luna 79.5 MMMU Pro (with tools) 79.5 (no tools 78.4)
Jul 9, 2026 GPT-5.6 Terra 82 MMMU Pro (with tools) 82 (no tools 80.7)
Jul 9, 2026 GPT-5.6 Sol 84.6 MMMU Pro with tools 84.6% (no tools 83.0%)
Jun 23, 2026 Seed2.1 Pro 82.7 MMMU-Pro (w. Tool) 81.6 (82.7)
Jun 10, 2026 DiffusionGemma 26B A4B 54.3 MMMU Pro
Jun 3, 2026 Gemma 4 12B 69.1 MMMU Pro
Jun 1, 2026 Qwen3.7-Plus 79 MMMU-Pro
Jun 1, 2026 MiniMax-M3 78.1 MMMU-Pro
May 20, 2026 Command A+ 63 MMMU Pro
May 19, 2026 Gemini 3.5 Flash 83.6 MMMU-Pro (No tools)
May 5, 2026 GPT-5.5 Instant 76 MMMU-Pro Standard (Expert Multimodal Reasoning)
Apr 28, 2026 Seed2.0 Lite (0428) 78.4 MMMU_Pro
Apr 23, 2026 GPT-5.5 83.2 MMMU Pro with tools 83.2% (no tools 81.2%)
Apr 22, 2026 Qwen3.6 27B 75.8 MMMU-Pro
Apr 22, 2026 MiMo-V2.5 77.9 MMMU-Pro
Apr 20, 2026 Kimi K2.6 79.4 MMMU-Pro
Apr 15, 2026 Qwen 3.6 35B-A3B 75.3 MMMU-Pro
Apr 9, 2026 EXAONE 4.5 33B 68.6 MMMU-Pro
Apr 8, 2026 Muse Spark 80.4 MMMU Pro (Multimodal Understanding)
Apr 2, 2026 Gemma 4 E4B 52.6 MMMU Pro
Apr 2, 2026 Gemma 4 E2B 44.2 MMMU Pro
Apr 2, 2026 Gemma 4 26B A4B 73.8 MMMU Pro
Apr 2, 2026 Qwen 3.6 Plus 78.8 MMMU-Pro
Apr 2, 2026 Gemma 4 31B IT 76.9 MMMU Pro
Mar 30, 2026 Qwen3.5-Omni-Plus 73.9 MMMU-Pro
Mar 18, 2026 MiMo-V2-Omni 76.8 MMMU-Pro
Mar 17, 2026 GPT-5.4 Nano 69.5 MMMUPro w/ Python 69.5 (without 66.1)
Mar 17, 2026 GPT-5.4 Mini 78 MMMU-Pro with Python, without Python 76.6%
Mar 16, 2026 Mistral Small 4 60 MMMU-Pro (reasoning; instruct = 46.3)
Mar 5, 2026 GPT-5.4 81.2 MMMU Pro (no tools)
Mar 3, 2026 Gemini 3.1 Flash-Lite 76.8 MMMU-Pro (No tools)
Mar 2, 2026 Qwen3.5-0.8B 31.2 MMMU-Pro 31.2/31.4 (with CI / without CI)
Mar 2, 2026 Qwen3.5-2B 50.3 MMMU-Pro 50.3/47.7 (with CI / without CI)
Mar 2, 2026 Qwen3.5-4B 66.3 MMMU-Pro
Mar 2, 2026 Qwen3.5-9B 70.1 MMMU-Pro
Feb 24, 2026 Qwen3.5-35B-A3B 75.1 MMMU-Pro
Feb 24, 2026 Qwen3.5-27B 75 MMMU-Pro
Feb 24, 2026 Qwen3.5-122B-A10B 76.9 MMMU-Pro
Feb 19, 2026 Gemini 3.1 Pro 80.5 MMMU-Pro (No tools)
Feb 17, 2026 Claude Sonnet 4.6 75.6 Visual reasoning MMMU-Pro (with tools)
Feb 16, 2026 Qwen3.5 397B-A17B 79 MMMU-Pro
Feb 12, 2026 Gemini 3 Deep Think 81.5 MMMU-Pro, no tools, 81.5%
Feb 6, 2026 ERNIE 5.0 68.6 MMMU-Pro
Feb 5, 2026 Claude Opus 4.6 77.3 Visual reasoning MMMU Pro (with tools)
Feb 2, 2026 Intern-S1-Pro 72.8 MMMU-Pro
Jan 27, 2026 Kimi K2.5 78.5 MMMU-Pro
Dec 18, 2025 Seed1.8 73.2 MMMU-Pro
Dec 17, 2025 Gemini 3 Flash 81.2 MMMU-Pro
Dec 8, 2025 GLM-4.6V 66 MMMU_Pro
Dec 2, 2025 Nova 2 Omni Preview 61.4 MMMU Pro accuracy
Dec 2, 2025 Nova 2 Lite 61.8 MMMU Pro accuracy
Dec 2, 2025 Nova 2.0 Pro Preview 63.5 MMMU Pro accuracy
Nov 18, 2025 Gemini 3 Pro 81 MMMU-Pro
Sep 23, 2025 Qwen3-VL-235B-A22B 69.3 MMMU_Pro
Sep 22, 2025 Qwen3-Omni-30B-A3B 57 MMMU_pro
Aug 11, 2025 GLM-4.5V 65.2 MMMU Pro
Aug 7, 2025 GPT-5 nano 62.6 MMMU-Pro (avg across standard and vision sets)
Aug 7, 2025 GPT-5 mini 74.1 MMMU-Pro (avg across standard and vision sets)
Aug 7, 2025 GPT-5 78.4 MMMU Pro*** Graduate-level visual problem-solving
Jul 1, 2025 GLM-4.1V-9B-Thinking 57.1 MMMU-Pro
Jun 25, 2025 Seed1.6-Thinking 66.4 MMMU-pro
Jun 21, 2025 Kimi-VL-A3B-Thinking-2506 46.3 MMMU-Pro (Pass@1)
May 12, 2025 Seed1.5-VL 67.6 MMMU-Pro
Apr 5, 2025 Llama 4 Scout 52.2 MMMU Pro (avg of Standard and Vision)
Apr 5, 2025 Llama 4 Maverick 59.6 MMMU Pro (avg of Standard and Vision)
Mar 27, 2025 Qwen2.5-Omni-7B 36.6 MMMU-Prooverall
Mar 17, 2025 Mistral Small 3.1 49.3 MMMU PRO
Feb 26, 2025 Phi-4-multimodal 38.5 MMMU-Pro (std/vision)
Jan 28, 2025 Qwen2.5-VL-72B 51.1 MMMU_Pro

45 system cards report it, from 17 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 23, 2026 Nemotron 3.5 Bolt E4B technical report 42 MMMU-Pro Vision, reasoning, 39.08 non-reasoning (Table 8) Capabilities
Oct 19, 2026 Muse Glimmer technical report 74 MMMU-Pro, high (Tables 2 and 5) Capabilities
Sep 10, 2026 DeepSeek-V4.1-Flash Base model card 56.5 MMMU-Pro, EM, 4-shot Capabilities
Jul 31, 2026 DiffusionGemma 26B A4B technical report 54.3 MMMU Pro, TD, 66.0 no-think (Table 3) Capabilities
Jul 30, 2026 Inkling-Small model card 74 MMMU Pro Standard 10 (5. Evaluations table) Capabilities
Jul 27, 2026 Kimi K3 technical report 81.6 MMMU-Pro without Python, cell 81.6 / 83.4 (without / with Python) (Table 2) Capabilities
Jul 24, 2026 Gemma 4 E4B technical report 52.6 MMMU Pro, max resolution, thinking (Table 6) Capabilities
Jul 24, 2026 Gemma 4 E2B technical report 44.2 MMMU Pro, max resolution, thinking (Table 6) Capabilities
Jul 24, 2026 Gemma 4 31B IT technical report 76.9 MMMU Pro, thinking, max resolution 1120 vision tokens (Table 6). At 280 tokens: 75.8 Capabilities
Jul 24, 2026 Gemma 4 26B A4B technical report 73.8 MMMU Pro, thinking, max resolution 1120 vision tokens (Table 6). At 280 tokens: 73.2 Capabilities
Jul 15, 2026 Inkling model card 73.5 MMMU Pro Standard 10 (5. Evaluations table, effort 0.99) Capabilities
Jun 23, 2026 Seed2.1 Pro model card 81.6 MMMU-Pro, no tools, 82.7 with tools (Table 9) Capabilities
May 31, 2026 Cosmos3-Nano technical report 41.1 MMMU-Pro, Cosmos3-Nano 8B column, Table 10 Capabilities
May 31, 2026 Cosmos3-Edge technical report 26.4 MMMU-Pro, Cosmos3-Edge 2B column, Table 10 Capabilities
May 19, 2026 Gemini 3.5 Flash model card 83.6 MMMU-Pro, no tools (results table, p.4) Capabilities
Apr 17, 2026 Qwen3.5-Omni-Plus technical report 73.9 MMMU-Pro (Table 6) Capabilities
Apr 9, 2026 EXAONE 4.5 33B technical report 68.6 MMMU-Pro (Table 2, Reasoning mode) Capabilities
Mar 26, 2026 Intern-S1-Pro technical report 72.8 MMMU-Pro, thinking (Table 2) Capabilities
Mar 21, 2026 Seed1.8 model card 73.2 MMMU-Pro (Table 2). 65.4 in NoThink mode (2.4 text) Capabilities
Mar 3, 2026 Gemini 3.1 Flash-Lite model card 76.8 MMMU-Pro, no tools (results table, p.4) Capabilities
Feb 19, 2026 Gemini 3.1 Pro model card 80.5 MMMU Pro, no tools (results table, p.4) Capabilities
Feb 17, 2026 Claude Sonnet 4.6 system card 74.5 MMMU-Pro, no tools, adaptive thinking, max effort; 75.6 with an image cropping tool (Table 2.1.A, 2.17.2) Capabilities
Feb 14, 2026 Seed2.0 Pro model card 78.2 MMMU-Pro, Standard (10 options) and Vision combined (Table 8) Capabilities
Feb 5, 2026 Claude Opus 4.6 system card 73.9 MMMU-Pro, no tools, adaptive thinking, max effort, average of Standard (10 options) and Vision (Table 2.3.A, 2.19.2); 77.3 with an image cropping tool Capabilities
Feb 4, 2026 ERNIE 5.0 technical report 68.6 MMMU-Pro (Table 3) Capabilities
Jan 30, 2026 Kimi K2.5 technical report 78.5 MMMU-Pro (Table 4) Capabilities
Jan 14, 2026 STEP3-VL-10B technical report 64.1 MMMU-Pro, SeRe (Tables 1 and 3). PaCoRe 67.18 Capabilities
Jan 1, 2026 GLM-4.6V technical report 66 MMMU Pro, GLM-4.6V thinking (Table 2) Capabilities
Dec 2, 2025 Nova 2 Omni Preview technical report 61.4 MMMU Pro accuracy (Table 6) Capabilities
Dec 2, 2025 Nova 2 Lite technical report 61.8 MMMU Pro accuracy (Table 3) Capabilities
Dec 2, 2025 Nova 2.0 Pro Preview technical report 63.5 MMMU Pro accuracy (Table 3) Capabilities
Nov 26, 2025 Qwen3-VL-235B-A22B technical report 69.3 MMMU-Pro (Table 2) Capabilities
Nov 6, 2025 Nemotron Nano 12B v2 VL technical report 28 MMMU Pro, reasoning on (Table 3). Reasoning off 14.5 Capabilities
Sep 22, 2025 Qwen3-Omni-30B-A3B technical report 57 MMMU-Pro overall, Instruct (Table 9). Thinking 60.5 (Table 10) Capabilities
Aug 13, 2025 GLM-4.5V technical report 65.2 MMMU Pro, GLM-4.5V thinking (Table 2) Capabilities
Jul 1, 2025 GLM-4.1V-9B-Thinking technical report 57.1 MMMU Pro, GLM-4.1V-9B-Thinking thinking (Table 2) Capabilities
Jun 23, 2025 Kimi-VL-A3B-Thinking-2506 technical report 46.3 MMMU-Pro (avg) (Pass@1) (Table 4) Capabilities
Jun 12, 2025 Magistral Small 1.0 technical report 51.8 MMMU-Pro (Standard), Magistral Small bar, printed label (Figure 10). MMMU-Pro (Vision) 39.2 Capabilities
Jun 12, 2025 Magistral Medium 1.0 technical report 57.9 MMMU-Pro (Standard), printed bar label (Figure 10). MMMU-Pro (Vision) 52.1 Capabilities
May 11, 2025 Seed1.5-VL technical report 67.6 MMMU-Pro, thinking (Table 6). Non-thinking 59.9 Capabilities
Apr 5, 2025 Llama 4 Scout model card 52.2 MMMU Pro, 0-shot, average of Standard and Vision, instruction tuned (Instruction tuned models table) Capabilities
Apr 5, 2025 Llama 4 Maverick model card 59.6 MMMU Pro, 0-shot, average of Standard and Vision, instruction tuned (Instruction tuned models table) Capabilities
Mar 26, 2025 Qwen2.5-Omni-7B technical report 36.6 MMMU-Pro overall (Table 5) Capabilities
Mar 3, 2025 Phi-4-multimodal technical report 38.5 MMMUPro, average of standard 39.7 and vision 37.3 (Table 1) Capabilities
Feb 19, 2025 Qwen2.5-VL-72B technical report 51.1 MMMU-Pro overall (Table 3) Capabilities

The lab's number comes out ahead in 2 of 3 pairs that differ

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 14 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
Qwen2.5-VL-72B 51.1% MMMU_Pro 46.2% Leaderboard +4.9 pts
Qwen3-VL-235B-A22B 69.3% MMMU_Pro 68.1% Leaderboard +1.2 pts
GPT-5.4 81.2% MMMU Pro (no tools) 82.1% Leaderboard, thinking, with tools −0.9 pts
Seed1.5-VL 67.6% MMMU-Pro 67.6% Leaderboard, thinking Same
Seed1.6-Thinking 66.4% MMMU-pro 66.4% Leaderboard Same
GLM-4.1V-9B-Thinking 57.1% MMMU-Pro 57.1% Leaderboard, thinking Same
GPT-5 78.4% MMMU Pro*** Graduate-level visual problem-solving 78.4% Leaderboard, thinking Same
GLM-4.5V 65.2% MMMU Pro 65.2% Leaderboard, thinking Same
Gemini 3 Pro 81% MMMU-Pro 81% Leaderboard Same
Claude Opus 4.6 77.3% Visual reasoning MMMU Pro (with tools) 77.3% Leaderboard, with tools Same
Claude Sonnet 4.6 75.6% Visual reasoning MMMU-Pro (with tools) 75.6% Leaderboard, with tools Same
Gemini 3.1 Pro 80.5% MMMU-Pro (No tools) 80.5% Leaderboard, thinking, high Same
Gemma 4 31B IT 76.9% MMMU Pro 76.9% Leaderboard Same
Gemma 4 26B A4B 73.8% MMMU Pro 73.8% Leaderboard Same
Gemma 4 E2B 44.2% MMMU Pro 44.2% Leaderboard Same
Gemma 4 E4B 52.6% MMMU Pro 52.6% Leaderboard Same
Muse Spark 80.4% MMMU Pro (Multimodal Understanding) 80.4% Leaderboard, thinking Same

Versions

Replaces MMMU

More in Vision

All 281 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers. Seed1.5-VL 96.9%