Benchmark Almanac

MM-IFEval

GPT-4o Aug 2024 64.6% leads since April 2025
MiniCPM-V-4.7-35B-A3B 65.4% leads since October 2026
In use

The benchmark’s paper hasn’t scored a model released after InternVL2.5-26B (December 2024). Labs report their own numbers for 3 newer models, such as STEP3-VL-10B at 61.9%.

Multimodal models follow image-grounded instructions with exact output constraints. Checks mix rule-based verification and a judge model.

Released with the MM-IFEngine training pipeline. 35 upvotes on Hugging Face papers.

Released
April 2025
Kind
Following precise instructions
Level
Everyday
Scored
Checked automatically
Answers
Open-ended
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Images → Text
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
In views
Vision
Strongest model tested
GPT-4o Aug 2024 #167 on the capability index
Last in a launch post
October 2026
Last model added
October 2026
Scores refreshed
November 18th, 2026

In the authors’ words

The Instruction Following (IF) ability measures how well Multi-modal Large Language Models (MLLMs) understand exactly what users are telling them and whether they are doing it right. Existing multimodal instruction following training data is scarce, the benchmarks are simple with atomic instructions, and the evaluation strategies are imprecise for tasks demanding exact output constraints. To address this, we present MM-IFEngine, an effective pipeline to generate high-quality image-instruction pairs. Our MM-IFEngine pipeline yields large-scale, diverse, and high-quality training data MM-IFInstruct-23k, which is suitable for Supervised Fine-Tuning (SFT) and extended as MM-IFDPO-23k for Direct Preference Optimization (DPO). We further introduce MM-IFEval, a challenging and diverse multi-modal instruction-following benchmark that includes (1) both compose-level constraints for output responses and perception-level constraints tied to the input images, and (2) a comprehensive evaluation pipeline incorporating both rule-based assessment and judge model. We conduct SFT and DPO experiments and demonstrate that fine-tuning MLLMs on MM-IFInstruct-23k and MM-IFDPO-23k achieves notable gains on various IF benchmarks, such as MM-IFEval (+10.2%), MIA (+7.6%), and IFEval (+12.3%). We have fully open-sourced the datasets (both SFT and DPO), evaluation code and training scripts at https://github.com/SYuan03/MM-IFEngine.

Abstract of MM-IFEngine: Towards Multimodal Instruction Following (arXiv:2504.07957, CC0)
15
Entries in its paper, 9 models we track
3
Launch posts list it, from 2 labs
2
System cards report it, from 2 labs

GPT-4o leads with 64.6% among models released by December 2024 GPT-4o leads with 64.6% among models released by December 2024

Best score per entry from the benchmark's paper, read on November 18th, 2026. A paper’s table doesn’t add newer models. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

MiniCPM-V-4.7-35B-A3B leads with 65.4%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.

The newest model with a score here came out in December 2024, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 GPT-4o Aug 2024 64.6%
  2. 2 GPT-4o-mini (snapshot unspecified) 62.8%
  3. 3 Claude-3.5-Sonnet (snapshot unspecified) 61.7%
  4. 4 Qwen2-VL-7B-Instruct + MM-IFDPO-23k 52.2%
  5. 5 Qwen2-VL-72B-Instruct 50.8%
  6. 6 LLaVA-NeXT-Llama3-8B + MM-IFDPO-23k 49.3%
  7. 7 InternVL2.5-26B 48.1%
  8. 8 InternVL2.5-8B 46.2%
  9. 9 InternVL2-40B 45%
  10. 10 Qwen2-VL-7B-Instruct 42%
  11. 11 InternVL2-8B 41.9%
  12. 12 LLaVA-NeXT-Llama3-8B 39.7%
  13. 13 MiniCPM-V-2_6 37.4%
  14. 14 LLaVA-OneVision 7B 34%
  15. 15 LLaVA-NeXT-7B 31.6%

Source: The MM-IFEval paper

#EntryScoreSettingDateSource
1 GPT-4o Aug 2024 64.6% Aug 2024 Official leaderboard
2 GPT-4o-mini (snapshot unspecified) 62.8% Official leaderboard
3 Claude-3.5-Sonnet (snapshot unspecified) 61.7% Official leaderboard
4 Qwen2-VL-7B-Instruct + MM-IFDPO-23k 52.2% Official leaderboard
5 Qwen2-VL-72B-Instruct 50.8% Aug 2024 Official leaderboard
6 LLaVA-NeXT-Llama3-8B + MM-IFDPO-23k 49.3% Official leaderboard
7 InternVL2.5-26B 48.1% Dec 2024 Official leaderboard
8 InternVL2.5-8B 46.2% Dec 2024 Official leaderboard
9 InternVL2-40B 45% Jul 2024 Official leaderboard
10 Qwen2-VL-7B-Instruct 42% Aug 2024 Official leaderboard
11 InternVL2-8B 41.9% Jul 2024 Official leaderboard
12 LLaVA-NeXT-Llama3-8B 39.7% Official leaderboard
13 MiniCPM-V-2_6 37.4% Aug 2024 Official leaderboard
14 LLaVA-OneVision 7B 34% Aug 2024 Official leaderboard
15 LLaVA-NeXT-7B 31.6% Official leaderboard
#ModelScoreSettingDateSource
1 GPT-4o Aug 2024 64.6% Aug 2024 Official leaderboard
2 Qwen2-VL-72B-Instruct 50.8% Aug 2024 Official leaderboard
3 InternVL2.5-26B 48.1% Dec 2024 Official leaderboard
4 InternVL2.5-8B 46.2% Dec 2024 Official leaderboard
5 InternVL2-40B 45% Jul 2024 Official leaderboard
6 Qwen2-VL-7B-Instruct 42% Aug 2024 Official leaderboard
7 InternVL2-8B 41.9% Jul 2024 Official leaderboard
8 MiniCPM-V-2_6 37.4% Aug 2024 Official leaderboard
9 LLaVA-OneVision 7B 34% Aug 2024 Official leaderboard
#ModelScoreSettingDateSource
1 MiniCPM-V-4.7-35B-A3B 65.4% Oct 2026 Lab model card
2 GPT-4o Aug 2024 64.6% Aug 2024 Official leaderboard
3 STEP3-VL-10B 61.9% Jan 2026 Lab model card
4 LFM2.5-VL-3B 60.6% Aug 2026 Lab model card
5 Qwen2-VL-72B-Instruct 50.8% Aug 2024 Official leaderboard
6 InternVL2.5-26B 48.1% Dec 2024 Official leaderboard
7 LFM2-VL-1.6B 46.4% Nov 2025 Lab model card
8 InternVL2.5-8B 46.2% Dec 2024 Official leaderboard
9 InternVL2-40B 45% Jul 2024 Official leaderboard
10 Qwen2-VL-7B-Instruct 42% Aug 2024 Official leaderboard
11 InternVL2-8B 41.9% Jul 2024 Official leaderboard
12 MiniCPM-V-2_6 37.4% Aug 2024 Official leaderboard
13 LLaVA-OneVision 7B 34% Aug 2024 Official leaderboard

Labs report scores for 4 models in their cards. Switch the scores above to rank them together.

Rankings by part

We read 2 parts from the benchmark’s paper: Compose level and Perception level. Each ranks models on one part of the test. GPT-4o leads both. Arrows show how far a model moved from its overall rank among the models we track.

Compose level

9 models we track have a score here.

  1. 1 GPT-4o Aug 2024 71.5%
  2. 2 InternVL2.5-26B ↑1 53.5%
  3. 3 Qwen2-VL-72B-Instruct ↓1 53.4%
  4. 4 InternVL2.5-8B 49.6%
  5. 5 InternVL2-40B 48%
  6. 6 InternVL2-8B ↑1 45.2%
  7. 7 Qwen2-VL-7B-Instruct ↓1 42.7%
  8. 8 MiniCPM-V-2_6 39.2%
  9. 9 LLaVA-OneVision 7B 37.4%

Number one over time 2 models have held number one with claims included since its release in April 2025

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s paper and what labs claim in their cards.

OpenAI Liquid AI StepFun OpenBMB (ModelBest)
Benchmark paper Not tested since Apr 2025
GPT-4o 64.6%
Lab claims 3 leaders

Sources: The MM-IFEval paper and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
In the benchmark’s paper Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) MiniCPM-V-4.7-35B-A3B GPT-4o InternVL2-40B 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) MiniCPM-V-4.7-35B-A3B GPT-4o

Sources: Official MM-IFEval leaderboard and labs’ model cards

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Liquid AI
OpenBMB (ModelBest)
DateLaunch postScoreAs the post writes it
Oct 7, 2026 MiniCPM-V-4.7-35B-A3B 65.4 MM-IFEval
Aug 12, 2026 LFM2.5-VL-3B 60.6 MM-IFEval
Aug 12, 2025 LFM2-VL-1.6B 37.7 MM-IFEval

2 system cards report it, from 2 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Jan 14, 2026 STEP3-VL-10B technical report 61.9 MM-IFEval, SeRe (Table 1) Capabilities
Nov 28, 2025 LFM2-VL-1.6B technical report 46.4 MM-IFEval, LFM2-VL-1.6B (Table 8) Capabilities

More in Instructions and chat

All 144 in Instructions and chat
  • IFEval In use Models answer about 500 prompts containing verifiable instructions such as word limits or required keywords. Code checks each instruction for strict and loose accuracy. Gemma 4 31B IT 98.9%
  • IFBench In use 58 new out-of-domain verifiable output constraints. Scored by programmatic constraint checks (strict or loose). Flint 89.2%
  • MultiChallenge In use Multi-turn conversations testing instruction retention, inference memory, versioned editing and self-coherence. Graded by LLM judge with rubrics. GPT-6.1 Sol 82%
  • Multi-IF In use Models follow verifiable instructions over three-turn conversations in eight languages, extending IFEval. Scored by instruction accuracy at each turn. LFM3-1.7B 89.1%
  • AdvancedIF In use Models respond to over 1,600 prompts with complex, multi-turn, and system-level instructions. A judge model checks each response against expert-written rubrics. MAI-Thinking-1 85%
  • ComplexConstraints In use Models answer 75 expert-written professional prompts whose constraints depend on each other, fire conditionally or must be inferred. An LLM judge grades 1,559 rubric criteria one at a time, and a task passes only if all criteria pass. GPT-6 Astra 70.8%

Data audit

9 of 10 checks pass, 1 warning. Look first at the Confidence check. Likely. Some facts rest on secondary sources. Every benchmark

Catalog record 1 warning 1 pass
  • Pass Task access All tasks are public.
  • Warning Confidence Likely. Some facts rest on secondary sources.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the benchmark’s paper (15 entries), launch posts (3 scores) and system cards (2 scores).
  • Pass Models scored 13 models we track, plus 6 other entries on its leaderboard.
  • Pass Newest score The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B.
  • Note Frontier models None of the top 10 has a score. The strongest tested is GPT-4o (#167).
  • Pass Headroom The best is 65.4%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on arxiv.org: 9 rows for models we track and 6 other entries at the last read.
  • Pass Last read November 18th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check