IFBench
The benchmark’s paper hasn’t scored a model released after Llama-3.1-Tulu-3-8B (November 2024). Labs report their own numbers for 80 newer models, such as Nemotron-Cascade 2 30B-A3B at 82.9%.
58 new out-of-domain verifiable output constraints. Scored by programmatic constraint checks (strict or loose).
The maker fixed several verifiers in 2026, including six that mis-scored correct answers (July 17th, 2026), so later runs can score slightly higher. Task set unchanged. An optional two-turn split is in IFBench_multi-turn.
- Made by
- AllenAI
- Released
- July 2025
- Announced on X
- 48K views, 306 likes
- Kind
- Following precise instructions
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Open-ended
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Multi-turn
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
-
Qwen 3.8 Max #30 on the capability index
- Last in a launch post
- October 2026
- Last model added
- October 2026
- Scores refreshed
- November 18th, 2026
In the authors’ words
A crucial factor for successful human and AI interaction is the ability of language models or chatbots to follow human instructions precisely. A common feature of instructions are output constraints like “only answer with yes or no" or “mention the word `abrakadabra' at least 3 times" that the user adds to craft a more useful answer. Even today's strongest models struggle with fulfilling such constraints. We find that most models strongly overfit on a small set of verifiable constraints from the benchmarks that test these abilities, a skill called precise instruction following, and are not able to generalize well to unseen output constraints. We introduce a new benchmark, IFBench, to evaluate precise instruction following generalization on 58 new, diverse, and challenging verifiable out-of-domain constraints. In addition, we perform an extensive analysis of how and on what data models can be trained to improve precise instruction following generalization. Specifically, we carefully design constraint verification modules and show that reinforcement learning with verifiable rewards (RLVR) significantly improves instruction following. In addition to IFBench, we release 29 additional new hand-annotated training constraints and verification functions, RLVR training prompts, and code.
Flint leads with 89.2%
Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.
Llama-3.1-Tulu-3-8B leads with 28.9% across all independent runs
Best score per model across the benchmark’s paper, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.
Flint leads with 89.2%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
- 89.2 Flint
- 82.9
Nemotron-Cascade 2 30B-A3B
- 82.8
Qwen 3.8 Max
- 82.2
Inkling-Small
- 81.7
Nemotron 3 Ultra
- 81.5
Qwen3.8-Omni-Flash
- 81.3
Qwen3.8 Flash
- 80.2
Nova 2.0 Pro Preview
- 80.0
Solar Open 2
- 79.8
Inkling
- 79.7 Beam 1.0
- 79.5
Qwen 3.8 27B
- 79.1
Qwen3.7-Max
- 79.1
Qwen3.7-Plus
- 77.2 Granite 4.2 30B
- 77.0
Muse Glimmer
- 77.0 Mercury 2.5
- 76.8 Mercury 3
- 76.5
Qwen3.5 397B-A17B
- 76.5
Qwen3.5-27B
- 76.1
Qwen3.5-122B-A10B
- 76.0
MiniMax-M2.7
- 76.0
Gemma 4 31B IT
- 75.8
Spark-X2.6-4B
- 75.0
MAI-Code-1-Flash
- 74.7 LFM2.5-24B-A2B
- 74.2
Qwen 3.6 Plus
- 74.2
Nemotron 3 Nano Omni 30B-A3B
- 74.0
Command A+
- 73.1
Solar Pro 3 (March 2026 update)
- 73.0 LFM3-1.7B
- 72.6
K-EXAONE 2.0
- 72.6
Nemotron 3 Super
- 72.0
MiniMax-M2
- 72.0 GLM-5
- 72.0
MiniMax-M2.5
- 72.0
Gemma 4 26B A4B
- 72.0
Nemotron 3.5 Lightning 30B-A3B
- 71.5
Nemotron 3 Nano 30B-A3B
- 71.2
Intern-S1-Pro
- 71.0 Mercury 2
- 70.9
Qwen3-Max-2026-01-23
- 70.9
Nemotron 3.5 Super VL
- 70.8
Nova 2 Lite
- 70.2
Qwen3.5-35B-A3B
- 70.0
MiniMax-M2.1
- 69.0
Mistral Medium 3.5
- 69.0
MAI-Thinking-1
- 68.7
Nova 2 Omni Preview
- 68.1
Olmo 3.1 32B Think
- 67.4
Step 3.5 Flash
- 67.3
K-EXAONE
- 66.2
Nemotron 3.5 Bolt 74B-A9B
- 64.5
Qwen3.5-9B
- 64.0
Nemotron 3.5 Bolt E4B
- 62.6
EXAONE 4.5 33B
- 59.2
Qwen3.5-4B
- 59.2 LFM2.5-2.6B
- 57.6
Ling-2.6-1T
- 56.5 LFM2.5-8B-A1B
- 55.8
Solar Pro 3
- 53.7
Solar Open 100B
- 53.4
Falcon H1R 7B
- 52.6
Qwen3.5-Omni-Plus
- 52.3
Trinity Large Thinking
- 48.0
Mistral Small 4
- 47.7
MiniCPM-V-4.7-35B-A3B
- 47.6
Olmo 3 32B Think
- 47.3 LFM2.5-1.2B-Instruct
- 44.9 LFM2.5-1.2B-Thinking
- 44.0
Gemma 4 E4B
- 43.3
STEP3-VL-10B
- 43.2
Nemotron 3 Nano 4B
- 41.3
Qwen3.5-2B
- 38.0
Gemma 4 E2B
- 34.6 nvidia-nemotron-nano-9b-v2
- 32.0
Cosmos3-Nano
- 25.9 LFM2-8B-A1B
- 25.8 LFM2.5-VL-3B
- 21.0
Qwen3.5-0.8B
- 20.8
Cosmos3-Edge
- 20.7 LFM2-1.2B
- 1 Flint 89.2%
- 2
Nemotron-Cascade 2 30B-A3B 82.9%
- 3
Qwen 3.8 Max 82.8%
- 4
Inkling-Small 82.2%
- 5
Nemotron 3 Ultra 81.7%
- 6
Qwen3.8-Omni-Flash 81.5%
- 7
Qwen3.8 Flash 81.3%
- 8
Nova 2.0 Pro Preview 80.2%
- 9
Solar Open 2 80%
- 10
Inkling 79.8%
- 11 Beam 1.0 79.7%
- 12
Qwen 3.8 27B 79.5%
- 13
Qwen3.7-Max 79.1%
- 14
Qwen3.7-Plus 79.1%
- 15 Granite 4.2 30B 77.2%
- 16
Muse Glimmer 77%
- 17 Mercury 2.5 77%
- 18 Mercury 3 76.8%
- 19
Qwen3.5 397B-A17B 76.5%
- 20
Qwen3.5-27B 76.5%
- 21
Qwen3.5-122B-A10B 76.1%
- 22
MiniMax-M2.7 76%
- 23
Gemma 4 31B IT 76%
- 24
Spark-X2.6-4B 75.8%
- 25
MAI-Code-1-Flash 75%
- 26 LFM2.5-24B-A2B 74.7%
- 27
Qwen 3.6 Plus 74.2%
- 28
Nemotron 3 Nano Omni 30B-A3B 74.2%
- 29
Command A+ 74%
- 30
Solar Pro 3 (March 2026 update) 73.1%
- 31 LFM3-1.7B 73%
- 32
K-EXAONE 2.0 72.6%
- 33
Nemotron 3 Super 72.6%
- 34
MiniMax-M2 72%
- 35 GLM-5 72%
- 36
MiniMax-M2.5 72%
- 37
Gemma 4 26B A4B 72%
- 38
Nemotron 3.5 Lightning 30B-A3B 72%
- 39
Nemotron 3 Nano 30B-A3B 71.5%
- 40
Intern-S1-Pro 71.2%
- 41 Mercury 2 71%
- 42
Qwen3-Max-2026-01-23 70.9%
- 43
Nemotron 3.5 Super VL 70.9%
- 44
Nova 2 Lite 70.8%
- 45
Qwen3.5-35B-A3B 70.2%
- 46
MiniMax-M2.1 70%
- 47
Mistral Medium 3.5 69%
- 48
MAI-Thinking-1 69%
- 49
Nova 2 Omni Preview 68.7%
- 50
Olmo 3.1 32B Think 68.1%
- 51
Step 3.5 Flash 67.4%
- 52
K-EXAONE 67.3%
- 53
Nemotron 3.5 Bolt 74B-A9B 66.2%
- 54
Qwen3.5-9B 64.5%
- 55
Nemotron 3.5 Bolt E4B 64%
- 56
EXAONE 4.5 33B 62.6%
- 57
Qwen3.5-4B 59.2%
- 58 LFM2.5-2.6B 59.2%
- 59
Ling-2.6-1T 57.6%
- 60 LFM2.5-8B-A1B 56.5%
- 61
Solar Pro 3 55.8%
- 62
Solar Open 100B 53.7%
- 63
Falcon H1R 7B 53.4%
- 64
Qwen3.5-Omni-Plus 52.6%
- 65
Trinity Large Thinking 52.3%
- 66
Mistral Small 4 48%
- 67
MiniCPM-V-4.7-35B-A3B 47.7%
- 68
Olmo 3 32B Think 47.6%
- 69 LFM2.5-1.2B-Instruct 47.3%
- 70 LFM2.5-1.2B-Thinking 44.9%
- 71
Gemma 4 E4B 44%
- 72
STEP3-VL-10B 43.3%
- 73
Nemotron 3 Nano 4B 43.2%
- 74
Qwen3.5-2B 41.3%
- 75
Gemma 4 E2B 38%
- 76 nvidia-nemotron-nano-9b-v2 34.6%
- 77
Cosmos3-Nano 32%
- 78 LFM2-8B-A1B 25.9%
- 79 LFM2.5-VL-3B 25.8%
- 80
Qwen3.5-0.8B 21%
- 81
Cosmos3-Edge 20.8%
- 82 LFM2-1.2B 20.7%
Source: Labs’ model cards
Source: The IFBench paper
- 89.2 Flint
- 82.9
Nemotron-Cascade 2 30B-A3B
- 82.8
Qwen 3.8 Max
- 82.2
Inkling-Small
- 81.7
Nemotron 3 Ultra
- 81.5
Qwen3.8-Omni-Flash
- 81.3
Qwen3.8 Flash
- 80.2
Nova 2.0 Pro Preview
- 80.0
Solar Open 2
- 79.8
Inkling
- 79.7 Beam 1.0
- 79.5
Qwen 3.8 27B
- 79.1
Qwen3.7-Max
- 79.1
Qwen3.7-Plus
- 77.2 Granite 4.2 30B
- 77.0
Muse Glimmer
- 77.0 Mercury 2.5
- 76.8 Mercury 3
- 76.5
Qwen3.5 397B-A17B
- 76.5
Qwen3.5-27B
- 76.1
Qwen3.5-122B-A10B
- 76.0
MiniMax-M2.7
- 76.0
Gemma 4 31B IT
- 75.8
Spark-X2.6-4B
- 75.0
MAI-Code-1-Flash
- 74.7 LFM2.5-24B-A2B
- 74.2
Qwen 3.6 Plus
- 74.2
Nemotron 3 Nano Omni 30B-A3B
- 74.0
Command A+
- 73.1
Solar Pro 3 (March 2026 update)
- 73.0 LFM3-1.7B
- 72.6
K-EXAONE 2.0
- 72.6
Nemotron 3 Super
- 72.0
MiniMax-M2
- 72.0 GLM-5
- 72.0
MiniMax-M2.5
- 72.0
Gemma 4 26B A4B
- 72.0
Nemotron 3.5 Lightning 30B-A3B
- 71.5
Nemotron 3 Nano 30B-A3B
- 71.2
Intern-S1-Pro
- 71.0 Mercury 2
- 70.9
Qwen3-Max-2026-01-23
- 70.9
Nemotron 3.5 Super VL
- 70.8
Nova 2 Lite
- 70.2
Qwen3.5-35B-A3B
- 70.0
MiniMax-M2.1
- 69.0
Mistral Medium 3.5
- 69.0
MAI-Thinking-1
- 68.7
Nova 2 Omni Preview
- 68.1
Olmo 3.1 32B Think
- 67.4
Step 3.5 Flash
- 67.3
K-EXAONE
- 66.2
Nemotron 3.5 Bolt 74B-A9B
- 64.5
Qwen3.5-9B
- 64.0
Nemotron 3.5 Bolt E4B
- 62.6
EXAONE 4.5 33B
- 59.2
Qwen3.5-4B
- 59.2 LFM2.5-2.6B
- 57.6
Ling-2.6-1T
- 56.5 LFM2.5-8B-A1B
- 55.8
Solar Pro 3
- 53.7
Solar Open 100B
- 53.4
Falcon H1R 7B
- 52.6
Qwen3.5-Omni-Plus
- 52.3
Trinity Large Thinking
- 48.0
Mistral Small 4
- 47.7
MiniCPM-V-4.7-35B-A3B
- 47.6
Olmo 3 32B Think
- 47.3 LFM2.5-1.2B-Instruct
- 44.9 LFM2.5-1.2B-Thinking
- 44.0
Gemma 4 E4B
- 43.3
STEP3-VL-10B
- 43.2
Nemotron 3 Nano 4B
- 41.3
Qwen3.5-2B
- 38.0
Gemma 4 E2B
- 34.6 nvidia-nemotron-nano-9b-v2
- 32.0
Cosmos3-Nano
- 28.9
Llama-3.1-Tulu-3-8B
- 25.9 LFM2-8B-A1B
- 25.8 LFM2.5-VL-3B
- 21.0
Qwen3.5-0.8B
- 20.8
Cosmos3-Edge
- 20.7 LFM2-1.2B
- 1 Flint 89.2%
- 2
Nemotron-Cascade 2 30B-A3B 82.9%
- 3
Qwen 3.8 Max 82.8%
- 4
Inkling-Small 82.2%
- 5
Nemotron 3 Ultra 81.7%
- 6
Qwen3.8-Omni-Flash 81.5%
- 7
Qwen3.8 Flash 81.3%
- 8
Nova 2.0 Pro Preview 80.2%
- 9
Solar Open 2 80%
- 10
Inkling 79.8%
- 11 Beam 1.0 79.7%
- 12
Qwen 3.8 27B 79.5%
- 13
Qwen3.7-Max 79.1%
- 14
Qwen3.7-Plus 79.1%
- 15 Granite 4.2 30B 77.2%
- 16
Muse Glimmer 77%
- 17 Mercury 2.5 77%
- 18 Mercury 3 76.8%
- 19
Qwen3.5 397B-A17B 76.5%
- 20
Qwen3.5-27B 76.5%
- 21
Qwen3.5-122B-A10B 76.1%
- 22
MiniMax-M2.7 76%
- 23
Gemma 4 31B IT 76%
- 24
Spark-X2.6-4B 75.8%
- 25
MAI-Code-1-Flash 75%
- 26 LFM2.5-24B-A2B 74.7%
- 27
Qwen 3.6 Plus 74.2%
- 28
Nemotron 3 Nano Omni 30B-A3B 74.2%
- 29
Command A+ 74%
- 30
Solar Pro 3 (March 2026 update) 73.1%
- 31 LFM3-1.7B 73%
- 32
K-EXAONE 2.0 72.6%
- 33
Nemotron 3 Super 72.6%
- 34
MiniMax-M2 72%
- 35 GLM-5 72%
- 36
MiniMax-M2.5 72%
- 37
Gemma 4 26B A4B 72%
- 38
Nemotron 3.5 Lightning 30B-A3B 72%
- 39
Nemotron 3 Nano 30B-A3B 71.5%
- 40
Intern-S1-Pro 71.2%
- 41 Mercury 2 71%
- 42
Qwen3-Max-2026-01-23 70.9%
- 43
Nemotron 3.5 Super VL 70.9%
- 44
Nova 2 Lite 70.8%
- 45
Qwen3.5-35B-A3B 70.2%
- 46
MiniMax-M2.1 70%
- 47
Mistral Medium 3.5 69%
- 48
MAI-Thinking-1 69%
- 49
Nova 2 Omni Preview 68.7%
- 50
Olmo 3.1 32B Think 68.1%
- 51
Step 3.5 Flash 67.4%
- 52
K-EXAONE 67.3%
- 53
Nemotron 3.5 Bolt 74B-A9B 66.2%
- 54
Qwen3.5-9B 64.5%
- 55
Nemotron 3.5 Bolt E4B 64%
- 56
EXAONE 4.5 33B 62.6%
- 57
Qwen3.5-4B 59.2%
- 58 LFM2.5-2.6B 59.2%
- 59
Ling-2.6-1T 57.6%
- 60 LFM2.5-8B-A1B 56.5%
- 61
Solar Pro 3 55.8%
- 62
Solar Open 100B 53.7%
- 63
Falcon H1R 7B 53.4%
- 64
Qwen3.5-Omni-Plus 52.6%
- 65
Trinity Large Thinking 52.3%
- 66
Mistral Small 4 48%
- 67
MiniCPM-V-4.7-35B-A3B 47.7%
- 68
Olmo 3 32B Think 47.6%
- 69 LFM2.5-1.2B-Instruct 47.3%
- 70 LFM2.5-1.2B-Thinking 44.9%
- 71
Gemma 4 E4B 44%
- 72
STEP3-VL-10B 43.3%
- 73
Nemotron 3 Nano 4B 43.2%
- 74
Qwen3.5-2B 41.3%
- 75
Gemma 4 E2B 38%
- 76 nvidia-nemotron-nano-9b-v2 34.6%
- 77
Cosmos3-Nano 32%
- 78
Llama-3.1-Tulu-3-8B single-turn; paper Table 3 28.9%
- 79 LFM2-8B-A1B 25.9%
- 80 LFM2.5-VL-3B 25.8%
- 81
Qwen3.5-0.8B 21%
- 82
Cosmos3-Edge 20.8%
- 83 LFM2-1.2B 20.7%
Sources: Labs’ model cards and the IFBench paper
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Flint | 89.2% | Oct 2026 | Lab model card | |
| 2 | | 82.9% | Mar 2026 | Lab model card | |
| 3 | | 82.8% | Aug 2026 | Lab model card | |
| 4 | | 82.2% | Jul 2026 | Lab model card | |
| 5 | | 81.7% | Jun 2026 | Lab model card | |
| 6 | | 81.5% | Sep 2026 | Lab model card | |
| 7 | | 81.3% | Aug 2026 | Lab model card | |
| 8 | | 80.2% | Dec 2025 | Lab model card | |
| 9 | | 80% | Jul 2026 | Lab model card | |
| 10 | | 79.8% | Jul 2026 | Lab model card | |
| 11 | Beam 1.0 | 79.7% | Oct 2026 | Lab model card | |
| 12 | | 79.5% | Aug 2026 | Lab model card | |
| 13 | | 79.1% | May 2026 | Lab model card | |
| 14 | | 79.1% | Jun 2026 | Lab model card | |
| 15 | Granite 4.2 30B | 77.2% | Aug 2026 | Lab model card | |
| 16 | | 77% | Aug 2026 | Lab model card | |
| 17 | Mercury 2.5 | 77% | Sep 2026 | Lab model card | |
| 18 | Mercury 3 | 76.8% | Oct 2026 | Lab model card | |
| 19 | | 76.5% | Feb 2026 | Lab model card | |
| 20 | | 76.5% | Feb 2026 | Lab model card | |
| 21 | | 76.1% | Feb 2026 | Lab model card | |
| 22 | | 76% | May 2026 | Lab model card | |
| 23 | | 76% | Jul 2026 | Lab model card | |
| 24 | | 75.8% | Oct 2026 | Lab model card | |
| 25 | | 75% | Jun 2026 | Lab model card | |
| 26 | LFM2.5-24B-A2B | 74.7% | Oct 2026 | Lab model card | |
| 27 | | 74.2% | Apr 2026 | Lab model card | |
| 28 | | 74.2% | Apr 2026 | Lab model card | |
| 29 | | 74% | May 2026 | Lab model card | |
| 30 | | 73.1% | Mar 2026 | Lab model card | |
| 31 | LFM3-1.7B | 73% | Oct 2026 | Lab model card | |
| 32 | | 72.6% | Jul 2026 | Lab model card | |
| 33 | | 72.6% | Mar 2026 | Lab model card | |
| 34 | | 72% | Oct 2025 | Lab model card | |
| 35 | GLM-5 | 72% | Feb 2026 | Lab model card | |
| 36 | | 72% | May 2026 | Lab model card | |
| 37 | | 72% | Jul 2026 | Lab model card | |
| 38 | | 72% | Oct 2026 | Lab model card | |
| 39 | | 71.5% | Dec 2025 | Lab model card | |
| 40 | | 71.2% | Feb 2026 | Lab model card | |
| 41 | Mercury 2 | 71% | Feb 2026 | Lab model card | |
| 42 | | 70.9% | Jan 2026 | Lab model card | |
| 43 | | 70.9% | Oct 2026 | Lab model card | |
| 44 | | 70.8% | Dec 2025 | Lab model card | |
| 45 | | 70.2% | Feb 2026 | Lab model card | |
| 46 | | 70% | Dec 2025 | Lab model card | |
| 47 | | 69% | May 2026 | Lab model card | |
| 48 | | 69% | Jun 2026 | Lab model card | |
| 49 | | 68.7% | Dec 2025 | Lab model card | |
| 50 | | 68.1% | Dec 2025 | Lab model card | |
| 51 | | 67.4% | Feb 2026 | Lab model card | |
| 52 | | 67.3% | Dec 2025 | Lab model card | |
| 53 | | 66.2% | Oct 2026 | Lab model card | |
| 54 | | 64.5% | Mar 2026 | Lab model card | |
| 55 | | 64% | Oct 2026 | Lab model card | |
| 56 | | 62.6% | Apr 2026 | Lab model card | |
| 57 | | 59.2% | Mar 2026 | Lab model card | |
| 58 | LFM2.5-2.6B | 59.2% | Aug 2026 | Lab model card | |
| 59 | | 57.6% | Jun 2026 | Lab model card | |
| 60 | LFM2.5-8B-A1B | 56.5% | May 2026 | Lab model card | |
| 61 | | 55.8% | Jan 2026 | Lab model card | |
| 62 | | 53.7% | Jan 2026 | Lab model card | |
| 63 | | 53.4% | Jan 2026 | Lab model card | |
| 64 | | 52.6% | Mar 2026 | Lab model card | |
| 65 | | 52.3% | Apr 2026 | Lab model card | |
| 66 | | 48% | Mar 2026 | Lab model card | |
| 67 | | 47.7% | Oct 2026 | Lab model card | |
| 68 | | 47.6% | Nov 2025 | Lab model card | |
| 69 | LFM2.5-1.2B-Instruct | 47.3% | Jan 2026 | Lab model card | |
| 70 | LFM2.5-1.2B-Thinking | 44.9% | Jan 2026 | Lab model card | |
| 71 | | 44% | Jul 2026 | Lab model card | |
| 72 | | 43.3% | Jan 2026 | Lab model card | |
| 73 | | 43.2% | Mar 2026 | Lab model card | |
| 74 | | 41.3% | Mar 2026 | Lab model card | |
| 75 | | 38% | Jul 2026 | Lab model card | |
| 76 | nvidia-nemotron-nano-9b-v2 | 34.6% | Aug 2025 | Lab model card | |
| 77 | | 32% | May 2026 | Lab model card | |
| 78 | LFM2-8B-A1B | 25.9% | Oct 2025 | Lab model card | |
| 79 | LFM2.5-VL-3B | 25.8% | Aug 2026 | Lab model card | |
| 80 | | 21% | Mar 2026 | Lab model card | |
| 81 | | 20.8% | May 2026 | Lab model card | |
| 82 | LFM2-1.2B | 20.7% | Jul 2025 | Lab model card |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | 28.9% | single-turn; paper Table 3 | Nov 2024 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Flint | 89.2% | Oct 2026 | Lab model card | |
| 2 | 82.9% | Mar 2026 | Lab model card | ||
| 3 | 82.8% | Aug 2026 | Lab model card | ||
| 4 | 82.2% | Jul 2026 | Lab model card | ||
| 5 | 81.7% | Jun 2026 | Lab model card | ||
| 6 | 81.5% | Sep 2026 | Lab model card | ||
| 7 | 81.3% | Aug 2026 | Lab model card | ||
| 8 | 80.2% | Dec 2025 | Lab model card | ||
| 9 | 80% | Jul 2026 | Lab model card | ||
| 10 | 79.8% | Jul 2026 | Lab model card | ||
| 11 | Beam 1.0 | 79.7% | Oct 2026 | Lab model card | |
| 12 | 79.5% | Aug 2026 | Lab model card | ||
| 13 | 79.1% | May 2026 | Lab model card | ||
| 14 | 79.1% | Jun 2026 | Lab model card | ||
| 15 | Granite 4.2 30B | 77.2% | Aug 2026 | Lab model card | |
| 16 | 77% | Aug 2026 | Lab model card | ||
| 17 | Mercury 2.5 | 77% | Sep 2026 | Lab model card | |
| 18 | Mercury 3 | 76.8% | Oct 2026 | Lab model card | |
| 19 | 76.5% | Feb 2026 | Lab model card | ||
| 20 | 76.5% | Feb 2026 | Lab model card | ||
| 21 | 76.1% | Feb 2026 | Lab model card | ||
| 22 | 76% | May 2026 | Lab model card | ||
| 23 | 76% | Jul 2026 | Lab model card | ||
| 24 | 75.8% | Oct 2026 | Lab model card | ||
| 25 | 75% | Jun 2026 | Lab model card | ||
| 26 | LFM2.5-24B-A2B | 74.7% | Oct 2026 | Lab model card | |
| 27 | 74.2% | Apr 2026 | Lab model card | ||
| 28 | 74.2% | Apr 2026 | Lab model card | ||
| 29 | 74% | May 2026 | Lab model card | ||
| 30 | 73.1% | Mar 2026 | Lab model card | ||
| 31 | LFM3-1.7B | 73% | Oct 2026 | Lab model card | |
| 32 | 72.6% | Jul 2026 | Lab model card | ||
| 33 | 72.6% | Mar 2026 | Lab model card | ||
| 34 | 72% | Oct 2025 | Lab model card | ||
| 35 | GLM-5 | 72% | Feb 2026 | Lab model card | |
| 36 | 72% | May 2026 | Lab model card | ||
| 37 | 72% | Jul 2026 | Lab model card | ||
| 38 | 72% | Oct 2026 | Lab model card | ||
| 39 | 71.5% | Dec 2025 | Lab model card | ||
| 40 | 71.2% | Feb 2026 | Lab model card | ||
| 41 | Mercury 2 | 71% | Feb 2026 | Lab model card | |
| 42 | 70.9% | Jan 2026 | Lab model card | ||
| 43 | 70.9% | Oct 2026 | Lab model card | ||
| 44 | 70.8% | Dec 2025 | Lab model card | ||
| 45 | 70.2% | Feb 2026 | Lab model card | ||
| 46 | 70% | Dec 2025 | Lab model card | ||
| 47 | 69% | May 2026 | Lab model card | ||
| 48 | 69% | Jun 2026 | Lab model card | ||
| 49 | 68.7% | Dec 2025 | Lab model card | ||
| 50 | 68.1% | Dec 2025 | Lab model card | ||
| 51 | 67.4% | Feb 2026 | Lab model card | ||
| 52 | 67.3% | Dec 2025 | Lab model card | ||
| 53 | 66.2% | Oct 2026 | Lab model card | ||
| 54 | 64.5% | Mar 2026 | Lab model card | ||
| 55 | 64% | Oct 2026 | Lab model card | ||
| 56 | 62.6% | Apr 2026 | Lab model card | ||
| 57 | 59.2% | Mar 2026 | Lab model card | ||
| 58 | LFM2.5-2.6B | 59.2% | Aug 2026 | Lab model card | |
| 59 | 57.6% | Jun 2026 | Lab model card | ||
| 60 | LFM2.5-8B-A1B | 56.5% | May 2026 | Lab model card | |
| 61 | 55.8% | Jan 2026 | Lab model card | ||
| 62 | 53.7% | Jan 2026 | Lab model card | ||
| 63 | 53.4% | Jan 2026 | Lab model card | ||
| 64 | 52.6% | Mar 2026 | Lab model card | ||
| 65 | 52.3% | Apr 2026 | Lab model card | ||
| 66 | 48% | Mar 2026 | Lab model card | ||
| 67 | 47.7% | Oct 2026 | Lab model card | ||
| 68 | 47.6% | Nov 2025 | Lab model card | ||
| 69 | LFM2.5-1.2B-Instruct | 47.3% | Jan 2026 | Lab model card | |
| 70 | LFM2.5-1.2B-Thinking | 44.9% | Jan 2026 | Lab model card | |
| 71 | 44% | Jul 2026 | Lab model card | ||
| 72 | 43.3% | Jan 2026 | Lab model card | ||
| 73 | 43.2% | Mar 2026 | Lab model card | ||
| 74 | 41.3% | Mar 2026 | Lab model card | ||
| 75 | 38% | Jul 2026 | Lab model card | ||
| 76 | nvidia-nemotron-nano-9b-v2 | 34.6% | Aug 2025 | Lab model card | |
| 77 | 32% | May 2026 | Lab model card | ||
| 78 | 28.9% | single-turn; paper Table 3 | Nov 2024 | Official leaderboard | |
| 79 | LFM2-8B-A1B | 25.9% | Oct 2025 | Lab model card | |
| 80 | LFM2.5-VL-3B | 25.8% | Aug 2026 | Lab model card | |
| 81 | 21% | Mar 2026 | Lab model card | ||
| 82 | 20.8% | May 2026 | Lab model card | ||
| 83 | LFM2-1.2B | 20.7% | Jul 2025 | Lab model card |
The benchmark's paper has 5 entries, 1 of them models we track. Switch the scores above to rank them together.
6 models have held number one in labs’ own numbers since its release in July 2025 Number one over time 6 models have held number one with claims included since its release in July 2025
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Labs’ model cards
Recursive Superintelligence’s own number for Flint passed 85% in October 2026
That was 15 months after its release. By the same age, 24% of the benchmarks from 2025 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official IFBench leaderboard and labs’ model cards
14 labs put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Oct 27, 2026 | LFM3-1.7B | 73 | IFBench |
| Oct 23, 2026 | 64 | IFBench | |
| Oct 22, 2026 | 75.8 | IFBench | |
| Oct 21, 2026 | Flint | 89.2 | IFBench |
| Oct 19, 2026 | Mercury 3 | 76.8 | IFBench |
| Oct 15, 2026 | 70.9 | IFBench | |
| Oct 13, 2026 | LFM2.5-24B-A2B | 74.7 | IFBench |
| Oct 7, 2026 | 47.7 | IFBench | |
| Oct 5, 2026 | Beam 1.0 | 79.7 | IFBench |
| Sep 18, 2026 | 81.5 | IFBench | |
| Sep 8, 2026 | Mercury 2.5 | 77 | IFBench |
| Aug 26, 2026 | 81.3 | Instruction following IFBench | |
| Aug 25, 2026 | Granite 4.2 30B | 77.2 | IFBench (prompt) |
| Aug 14, 2026 | 79.5 | Instruction following IFBench | |
| Aug 12, 2026 | LFM2.5-VL-3B | 25.8 | IFBench |
| Aug 11, 2026 | 71.9 | IFBench (loose) | |
| Aug 10, 2026 | 77 | IFBench | |
| Aug 4, 2026 | LFM2.5-2.6B | 59.2 | IFBench |
| Aug 3, 2026 | 82.8 | IFBench | |
| Jul 31, 2026 | 72.6 | IFBench | |
| Jul 30, 2026 | 82.2 | IFBench | |
| Jul 22, 2026 | 80 | IFBench | |
| Jul 15, 2026 | 79.8 | IFBench | |
| Jun 4, 2026 | 81.7 | IFBench (prompt loose) | |
| Jun 2, 2026 | 75 | IF Bench (average) | |
| Jun 2, 2026 | 69 | IF Bench | |
| Jun 1, 2026 | 79.1 | IFBench | |
| May 28, 2026 | LFM2.5-8B-A1B | 56.5 | IFBench |
| May 22, 2026 | 69 | Allenai Ifbench | |
| May 20, 2026 | 74 | IFBench (single turn, loose, prompt accuracy) | |
| May 20, 2026 | 79.1 | IFBench | |
| Apr 29, 2026 | 57 | IFBench | |
| Apr 9, 2026 | 62.6 | IFBench | |
| Apr 2, 2026 | 74.2 | IFBench | |
| Apr 1, 2026 | 52.3 | IFBench | |
| Mar 30, 2026 | 52.6 | IFBench | |
| Mar 24, 2026 | 73.1 | IFBench | |
| Mar 19, 2026 | 82.9 | IFBench (prompt) | |
| Mar 16, 2026 | 43.2 | IFBench-Prompt (reasoning off); Instruction 44.2 | |
| Mar 16, 2026 | 48 | AllenAI IFBench (reasoning; instruct = 35.7) | |
| Mar 11, 2026 | 72.6 | IFBench (prompt) | |
| Mar 2, 2026 | 21 | IFBench | |
| Mar 2, 2026 | 41.3 | IFBench | |
| Mar 2, 2026 | 59.2 | IFBench | |
| Mar 2, 2026 | 64.5 | IFBench | |
| Feb 24, 2026 | 70.2 | IFBench | |
| Feb 24, 2026 | 76.5 | IFBench | |
| Feb 24, 2026 | Mercury 2 | 71 | IFBench |
| Feb 24, 2026 | 76.1 | IFBench | |
| Feb 16, 2026 | 76.5 | IFBench | |
| Feb 12, 2026 | 70 | IFBench | |
| Feb 2, 2026 | 71.2 | IFBench | |
| Jan 26, 2026 | 55.8 | IFBench | |
| Jan 23, 2026 | 70.9 | IFBench | |
| Jan 20, 2026 | LFM2.5-1.2B-Thinking | 44.9 | IFBench (± 0.73) |
| Jan 5, 2026 | 53.4 | IFBench | |
| Jan 5, 2026 | LFM2.5-1.2B-Instruct | 47.3 | IFBench |
| Jan 5, 2026 | 53.7 | IFBench | |
| Dec 31, 2025 | 67.3 | IFBench | |
| Dec 23, 2025 | 70 | IFBench | |
| Dec 15, 2025 | 71.5 | IFBench (prompt) | |
| Dec 12, 2025 | 68.1 | IFBench | |
| Dec 2, 2025 | 68.7 | IF-Bench prompt loose acc | |
| Dec 2, 2025 | 70.8 | IF-Bench prompt loose acc | |
| Dec 2, 2025 | 80.2 | IF-Bench prompt loose acc | |
| Nov 20, 2025 | 47.6 | IFBench | |
| Oct 27, 2025 | 72 | IFBench (AA) | |
| Oct 7, 2025 | LFM2-8B-A1B | 25.9 | IFBench |
| Jul 10, 2025 | LFM2-1.2B | 20.7 | IFBench |
43 system cards report it, from 17 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Oct 23, 2026 | | 64 | IFBench, released Q4_K_M imatrix checkpoint (Table 5). BF16 63.79 (Table 3) | Capabilities |
| Oct 19, 2026 | | 77 | IFBench, high (Tables 2 and 5) | Capabilities |
| Oct 15, 2026 | | 70.9 | IFBench, strict (Table 10) | Capabilities |
| Oct 15, 2026 | | 72 | IFBench (loose), BF16 acc. (Table 22) | Capabilities |
| Oct 15, 2026 | | 66.2 | IFBench (instruction loose), released 24GB checkpoint (Table 7). BF16 68.71 | Capabilities |
| Sep 22, 2026 | | 81.5 | IFBench (Table 2) | Capabilities |
| Aug 5, 2026 | | 72.6 | IFBench (Table 6, Reasoning mode) | Capabilities |
| Jul 30, 2026 | | 82.2 | IFBench (5. Evaluations table) | Capabilities |
| Jul 24, 2026 | | 44 | IFBench, thinking (Table 5) | Capabilities |
| Jul 24, 2026 | | 38 | IFBench, thinking (Table 5) | Capabilities |
| Jul 24, 2026 | | 76 | IFBench, thinking (Table 5) | Capabilities |
| Jul 24, 2026 | | 72 | IFBench, thinking (Table 5) | Capabilities |
| Jul 22, 2026 | | 80 | IFBench (Table 3) | Capabilities |
| Jul 15, 2026 | | 79.8 | IFBench (5. Evaluations table, effort 0.99) | Capabilities |
| Jun 13, 2026 | | 57.6 | IFBench, Mean@5 (Table 5) | Capabilities |
| Jun 9, 2026 | | 81.7 | IFBench (prompt loose) (Table 10) | Capabilities |
| Jun 2, 2026 | | 69 | IF Bench (Table 12, rounded) | Capabilities |
| Jun 2, 2026 | | 75 | IF Bench, average (Instruction following table) | Capabilities |
| May 31, 2026 | | 32 | IFBench, Cosmos3-Nano 8B column, Table 10 | Capabilities |
| May 31, 2026 | | 20.8 | IFBench, Cosmos3-Edge 2B column, Table 10 | Capabilities |
| May 26, 2026 | | 76 | IFBench, AA Index v4.0 protocol (Table 4) | Capabilities |
| May 26, 2026 | | 72 | IFBench, AA Index v4.0 protocol (Table 4) | Capabilities |
| Apr 27, 2026 | | 74.2 | IFBench (prompt) (Table 10) | Capabilities |
| Apr 17, 2026 | | 52.6 | IFBench (Table 4) | Capabilities |
| Apr 9, 2026 | | 62.6 | IFBench (Table 3, Reasoning mode) | Capabilities |
| Mar 26, 2026 | | 71.2 | IFBench, thinking (Table 2) | Capabilities |
| Mar 19, 2026 | | 82.9 | IFBench (prompt) (Table 1) | Capabilities |
| Mar 11, 2026 | | 72.6 | IFBench (prompt) (Table 5). Table 8 BF16 prints 72.58 | Capabilities |
| Feb 17, 2026 | GLM-5 technical report | 72 | IFBench, Instruction Following panel (Figure 11) | Capabilities |
| Feb 11, 2026 | | 67.4 | IFBench (Table 5). PaCoRe 56.8 | Capabilities |
| Jan 14, 2026 | | 43.3 | IFBench, SeRe (Table 2) | Capabilities |
| Jan 11, 2026 | | 53.7 | IFBench (Table 6) | Capabilities |
| Jan 5, 2026 | | 53.4 | IFBench (Table 6) | Capabilities |
| Jan 5, 2026 | | 67.3 | IFBench, prompt-loose (Table 3, Reasoning mode). Non-reasoning 40.5 (Table 4) | Capabilities |
| Dec 23, 2025 | | 71.5 | IFBench (prompt) (Table 3) | Capabilities |
| Dec 15, 2025 | | 47.6 | IFBench, Final Think 3.0 column (Tables 14 and 24) | Capabilities |
| Dec 15, 2025 | | 68.1 | IFBench (Tables 1, 14 and 24) | Capabilities |
| Dec 2, 2025 | | 68.7 | IF-Bench prompt loose acc (Table 5) | Capabilities |
| Dec 2, 2025 | | 70.8 | IF-Bench prompt loose acc (Table 1) | Capabilities |
| Dec 2, 2025 | | 80.2 | IF-Bench prompt loose acc (Table 1) | Capabilities |
| Nov 28, 2025 | LFM2-8B-A1B technical report | 25.9 | IFBench, LFM2-8B-A1B (Table 7) | Capabilities |
| Nov 28, 2025 | LFM2-1.2B technical report | 20.7 | IFBench, LFM2-1.2B (Table 6) | Capabilities |
| Aug 20, 2025 | | 34.6 | IFBench (Instr. Following), NVIDIA-Nemotron-Nano-9B-v2 (Figure 1) | Capabilities |
3 logged changes that move scores
Changes by the maintainers, so a score from before a change may not compare with one from after it. Newest first, each with the maintainers' own source. The list may be incomplete. What we log.
- Grader changed July 17th, 2026
The maintainers fixed six instruction verifiers that incorrectly accepted or rejected responses. Scores can move in either direction without changing the task set. Source
- Grader changed January 4th, 2026
The maintainers fixed verifier crashes, case handling, tokenization, and word-boundary matching. The revised checks can assign different results to identical responses. Source
- Scores recomputed November 11th, 2025
The maintainers revised the leaderboard after fixing IFBench verifiers. Earlier published scores can differ from the revised results. Source
Versions
More in Instructions and chat
- IFEval In use Models answer about 500 prompts containing verifiable instructions such as word limits or required keywords. Code checks each instruction for strict and loose accuracy.
Gemma 4 31B IT 98.9%
- MultiChallenge In use Multi-turn conversations testing instruction retention, inference memory, versioned editing and self-coherence. Graded by LLM judge with rubrics. GPT-6.1 Sol 82%
- Multi-IF In use Models follow verifiable instructions over three-turn conversations in eight languages, extending IFEval. Scored by instruction accuracy at each turn. LFM3-1.7B 89.1%
- AdvancedIF In use Models respond to over 1,600 prompts with complex, multi-turn, and system-level instructions. A judge model checks each response against expert-written rubrics.
MAI-Thinking-1 85%
- MM-IFEval In use Multimodal models follow image-grounded instructions with exact output constraints. Checks mix rule-based verification and a judge model. GPT-4o 64.6%
- ComplexConstraints In use Models answer 75 expert-written professional prompts whose constraints depend on each other, fire conditionally or must be inferred. An LLM judge grades 1,559 rubric criteria one at a time, and a task passes only if all criteria pass. GPT-6 Astra 70.8%
Data audit
All 9 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
- Pass Independent runs Scores from the benchmark’s paper (5 entries), launch posts (69 scores) and system cards (43 scores).
- Pass Models scored 83 models we track, plus 4 other entries on its leaderboard.
- Pass Newest score The newest is from October 2026, for LFM3-1.7B.
- Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30).
- Note Headroom Saturated. The best is 89.2%, past 85% since October 2026.
Leaderboard import 3 pass
- Pass Importer We read the board on arxiv.org: 1 row for models we track and 4 other entries at the last read.
- Pass Last read November 18th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.