MM-IFEval
The benchmark’s paper hasn’t scored a model released after InternVL2.5-26B (December 2024). Labs report their own numbers for 3 newer models, such as STEP3-VL-10B at 61.9%.
Multimodal models follow image-grounded instructions with exact output constraints. Checks mix rule-based verification and a judge model.
Released with the MM-IFEngine training pipeline. 35 upvotes on Hugging Face papers.
- Made by
- MM-IFEngine team (Ding et al.)
- Released
- April 2025
- Kind
- Following precise instructions
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Open-ended
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
- In views
- Vision
- Strongest model tested
- GPT-4o Aug 2024 #167 on the capability index
- Last in a launch post
- October 2026
- Last model added
- October 2026
- Scores refreshed
- November 18th, 2026
In the authors’ words
The Instruction Following (IF) ability measures how well Multi-modal Large Language Models (MLLMs) understand exactly what users are telling them and whether they are doing it right. Existing multimodal instruction following training data is scarce, the benchmarks are simple with atomic instructions, and the evaluation strategies are imprecise for tasks demanding exact output constraints. To address this, we present MM-IFEngine, an effective pipeline to generate high-quality image-instruction pairs. Our MM-IFEngine pipeline yields large-scale, diverse, and high-quality training data MM-IFInstruct-23k, which is suitable for Supervised Fine-Tuning (SFT) and extended as MM-IFDPO-23k for Direct Preference Optimization (DPO). We further introduce MM-IFEval, a challenging and diverse multi-modal instruction-following benchmark that includes (1) both compose-level constraints for output responses and perception-level constraints tied to the input images, and (2) a comprehensive evaluation pipeline incorporating both rule-based assessment and judge model. We conduct SFT and DPO experiments and demonstrate that fine-tuning MLLMs on MM-IFInstruct-23k and MM-IFDPO-23k achieves notable gains on various IF benchmarks, such as MM-IFEval (+10.2%), MIA (+7.6%), and IFEval (+12.3%). We have fully open-sourced the datasets (both SFT and DPO), evaluation code and training scripts at https://github.com/SYuan03/MM-IFEngine.
GPT-4o leads with 64.6% among models released by December 2024 GPT-4o leads with 64.6% among models released by December 2024
Best score per entry from the benchmark's paper, read on November 18th, 2026. A paper’s table doesn’t add newer models. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
MiniCPM-V-4.7-35B-A3B leads with 65.4%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
The newest model with a score here came out in December 2024, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 64.6 GPT-4o
- 62.8 GPT-4o-mini (snapshot unspecified)
- 61.7 Claude-3.5-Sonnet (snapshot unspecified)
- 52.2 Qwen2-VL-7B-Instruct + MM-IFDPO-23k
- 50.8 Qwen2-VL-72B-Instruct
- 49.3 LLaVA-NeXT-Llama3-8B + MM-IFDPO-23k
- 48.1
InternVL2.5-26B
- 46.2
InternVL2.5-8B
- 45.0
InternVL2-40B
- 42.0 Qwen2-VL-7B-Instruct
- 41.9
InternVL2-8B
- 39.7 LLaVA-NeXT-Llama3-8B
- 37.4 MiniCPM-V-2_6
- 34.0
LLaVA-OneVision 7B
- 31.6 LLaVA-NeXT-7B
- 1 GPT-4o Aug 2024 64.6%
- 2 GPT-4o-mini (snapshot unspecified) 62.8%
- 3 Claude-3.5-Sonnet (snapshot unspecified) 61.7%
- 4 Qwen2-VL-7B-Instruct + MM-IFDPO-23k 52.2%
- 5 Qwen2-VL-72B-Instruct 50.8%
- 6 LLaVA-NeXT-Llama3-8B + MM-IFDPO-23k 49.3%
- 7
InternVL2.5-26B 48.1%
- 8
InternVL2.5-8B 46.2%
- 9
InternVL2-40B 45%
- 10 Qwen2-VL-7B-Instruct 42%
- 11
InternVL2-8B 41.9%
- 12 LLaVA-NeXT-Llama3-8B 39.7%
- 13 MiniCPM-V-2_6 37.4%
- 14
LLaVA-OneVision 7B 34%
- 15 LLaVA-NeXT-7B 31.6%
Source: The MM-IFEval paper
- 1 GPT-4o Aug 2024 64.6%
- 2 Qwen2-VL-72B-Instruct 50.8%
- 3
InternVL2.5-26B 48.1%
- 4
InternVL2.5-8B 46.2%
- 5
InternVL2-40B 45%
- 6 Qwen2-VL-7B-Instruct 42%
- 7
InternVL2-8B 41.9%
- 8 MiniCPM-V-2_6 37.4%
- 9
LLaVA-OneVision 7B 34%
Source: The MM-IFEval paper
- 1
MiniCPM-V-4.7-35B-A3B 65.4%
- 2 GPT-4o Aug 2024 64.6%
- 3
STEP3-VL-10B 61.9%
- 4 LFM2.5-VL-3B 60.6%
- 5 Qwen2-VL-72B-Instruct 50.8%
- 6
InternVL2.5-26B 48.1%
- 7 LFM2-VL-1.6B 46.4%
- 8
InternVL2.5-8B 46.2%
- 9
InternVL2-40B 45%
- 10 Qwen2-VL-7B-Instruct 42%
- 11
InternVL2-8B 41.9%
- 12 MiniCPM-V-2_6 37.4%
- 13
LLaVA-OneVision 7B 34%
Sources: Labs’ model cards and the MM-IFEval paper
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-4o Aug 2024 | 64.6% | Aug 2024 | Official leaderboard | |
| 2 | GPT-4o-mini (snapshot unspecified) | 62.8% | Official leaderboard | ||
| 3 | Claude-3.5-Sonnet (snapshot unspecified) | 61.7% | Official leaderboard | ||
| 4 | Qwen2-VL-7B-Instruct + MM-IFDPO-23k | 52.2% | Official leaderboard | ||
| 5 | Qwen2-VL-72B-Instruct | 50.8% | Aug 2024 | Official leaderboard | |
| 6 | LLaVA-NeXT-Llama3-8B + MM-IFDPO-23k | 49.3% | Official leaderboard | ||
| 7 | | 48.1% | Dec 2024 | Official leaderboard | |
| 8 | | 46.2% | Dec 2024 | Official leaderboard | |
| 9 | | 45% | Jul 2024 | Official leaderboard | |
| 10 | Qwen2-VL-7B-Instruct | 42% | Aug 2024 | Official leaderboard | |
| 11 | | 41.9% | Jul 2024 | Official leaderboard | |
| 12 | LLaVA-NeXT-Llama3-8B | 39.7% | Official leaderboard | ||
| 13 | MiniCPM-V-2_6 | 37.4% | Aug 2024 | Official leaderboard | |
| 14 | | 34% | Aug 2024 | Official leaderboard | |
| 15 | LLaVA-NeXT-7B | 31.6% | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-4o Aug 2024 | 64.6% | Aug 2024 | Official leaderboard | |
| 2 | Qwen2-VL-72B-Instruct | 50.8% | Aug 2024 | Official leaderboard | |
| 3 | | 48.1% | Dec 2024 | Official leaderboard | |
| 4 | | 46.2% | Dec 2024 | Official leaderboard | |
| 5 | | 45% | Jul 2024 | Official leaderboard | |
| 6 | Qwen2-VL-7B-Instruct | 42% | Aug 2024 | Official leaderboard | |
| 7 | | 41.9% | Jul 2024 | Official leaderboard | |
| 8 | MiniCPM-V-2_6 | 37.4% | Aug 2024 | Official leaderboard | |
| 9 | | 34% | Aug 2024 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | 65.4% | Oct 2026 | Lab model card | ||
| 2 | GPT-4o Aug 2024 | 64.6% | Aug 2024 | Official leaderboard | |
| 3 | 61.9% | Jan 2026 | Lab model card | ||
| 4 | LFM2.5-VL-3B | 60.6% | Aug 2026 | Lab model card | |
| 5 | Qwen2-VL-72B-Instruct | 50.8% | Aug 2024 | Official leaderboard | |
| 6 | 48.1% | Dec 2024 | Official leaderboard | ||
| 7 | LFM2-VL-1.6B | 46.4% | Nov 2025 | Lab model card | |
| 8 | 46.2% | Dec 2024 | Official leaderboard | ||
| 9 | 45% | Jul 2024 | Official leaderboard | ||
| 10 | Qwen2-VL-7B-Instruct | 42% | Aug 2024 | Official leaderboard | |
| 11 | 41.9% | Jul 2024 | Official leaderboard | ||
| 12 | MiniCPM-V-2_6 | 37.4% | Aug 2024 | Official leaderboard | |
| 13 | 34% | Aug 2024 | Official leaderboard |
Labs report scores for 4 models in their cards. Switch the scores above to rank them together.
Rankings by part
We read 2 parts from the benchmark’s paper: Compose level and Perception level. Each ranks models on one part of the test. GPT-4o leads both. Arrows show how far a model moved from its overall rank among the models we track.
Compose level
9 models we track have a score here.
- 1 GPT-4o Aug 2024 71.5%
- 2
InternVL2.5-26B ↑1 53.5%
- 3 Qwen2-VL-72B-Instruct ↓1 53.4%
- 4
InternVL2.5-8B 49.6%
- 5
InternVL2-40B 48%
- 6
InternVL2-8B ↑1 45.2%
- 7 Qwen2-VL-7B-Instruct ↓1 42.7%
- 8 MiniCPM-V-2_6 39.2%
- 9
LLaVA-OneVision 7B 37.4%
Perception level
9 models we track have a score here.
- 1 GPT-4o Aug 2024 44%
- 2 Qwen2-VL-72B-Instruct 43%
- 3 Qwen2-VL-7B-Instruct ↑3 40%
- 4
InternVL2-40B ↑1 36%
- 5
InternVL2.5-8B ↓1 36%
- 6
InternVL2-8B ↑1 32%
- 7 MiniCPM-V-2_6 ↑1 32%
- 8
InternVL2.5-26B ↓5 32%
- 9
LLaVA-OneVision 7B 24%
Number one over time 2 models have held number one with claims included since its release in April 2025
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s paper and what labs claim in their cards.
Sources: The MM-IFEval paper and labs’ model cards
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official MM-IFEval leaderboard and labs’ model cards
2 labs put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Oct 7, 2026 | 65.4 | MM-IFEval | |
| Aug 12, 2026 | LFM2.5-VL-3B | 60.6 | MM-IFEval |
| Aug 12, 2025 | LFM2-VL-1.6B | 37.7 | MM-IFEval |
2 system cards report it, from 2 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Jan 14, 2026 | | 61.9 | MM-IFEval, SeRe (Table 1) | Capabilities |
| Nov 28, 2025 | LFM2-VL-1.6B technical report | 46.4 | MM-IFEval, LFM2-VL-1.6B (Table 8) | Capabilities |
More in Instructions and chat
- IFEval In use Models answer about 500 prompts containing verifiable instructions such as word limits or required keywords. Code checks each instruction for strict and loose accuracy.
Gemma 4 31B IT 98.9%
- IFBench In use 58 new out-of-domain verifiable output constraints. Scored by programmatic constraint checks (strict or loose). Flint 89.2%
- MultiChallenge In use Multi-turn conversations testing instruction retention, inference memory, versioned editing and self-coherence. Graded by LLM judge with rubrics. GPT-6.1 Sol 82%
- Multi-IF In use Models follow verifiable instructions over three-turn conversations in eight languages, extending IFEval. Scored by instruction accuracy at each turn. LFM3-1.7B 89.1%
- AdvancedIF In use Models respond to over 1,600 prompts with complex, multi-turn, and system-level instructions. A judge model checks each response against expert-written rubrics.
MAI-Thinking-1 85%
- ComplexConstraints In use Models answer 75 expert-written professional prompts whose constraints depend on each other, fire conditionally or must be inferred. An LLM judge grades 1,559 rubric criteria one at a time, and a task passes only if all criteria pass. GPT-6 Astra 70.8%
Data audit
9 of 10 checks pass, 1 warning. Look first at the Confidence check. Likely. Some facts rest on secondary sources. Every benchmark
Catalog record 1 warning 1 pass
- Pass Task access All tasks are public.
- Warning Confidence Likely. Some facts rest on secondary sources.
Scores 4 pass 1 note
- Pass Independent runs Scores from the benchmark’s paper (15 entries), launch posts (3 scores) and system cards (2 scores).
- Pass Models scored 13 models we track, plus 6 other entries on its leaderboard.
- Pass Newest score The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B.
- Note Frontier models None of the top 10 has a score. The strongest tested is GPT-4o (#167).
- Pass Headroom The best is 65.4%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on arxiv.org: 9 rows for models we track and 6 other entries at the last read.
- Pass Last read November 18th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.