Benchmark Almanac

IFBench

Flint 89.2% leads since October 2026
Flint 89.2% leads since October 2026
In use

The benchmark’s paper hasn’t scored a model released after Llama-3.1-Tulu-3-8B (November 2024). Labs report their own numbers for 80 newer models, such as Nemotron-Cascade 2 30B-A3B at 82.9%.

58 new out-of-domain verifiable output constraints. Scored by programmatic constraint checks (strict or loose).

The maker fixed several verifiers in 2026, including six that mis-scored correct answers (July 17th, 2026), so later runs can score slightly higher. Task set unchanged. An optional two-turn split is in IFBench_multi-turn.

Made by
AllenAI
Released
July 2025
Announced on X
48K views, 306 likes
Kind
Following precise instructions
Level
Everyday
Scored
Checked automatically
Answers
Open-ended
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Text
Interaction
Multi-turn
Refresh
One fixed set
Run by
Anyone
Strongest model tested
Qwen 3.8 Max #30 on the capability index
Last in a launch post
October 2026
Last model added
October 2026
Scores refreshed
November 18th, 2026

In the authors’ words

A crucial factor for successful human and AI interaction is the ability of language models or chatbots to follow human instructions precisely. A common feature of instructions are output constraints like “only answer with yes or no" or “mention the word `abrakadabra' at least 3 times" that the user adds to craft a more useful answer. Even today's strongest models struggle with fulfilling such constraints. We find that most models strongly overfit on a small set of verifiable constraints from the benchmarks that test these abilities, a skill called precise instruction following, and are not able to generalize well to unseen output constraints. We introduce a new benchmark, IFBench, to evaluate precise instruction following generalization on 58 new, diverse, and challenging verifiable out-of-domain constraints. In addition, we perform an extensive analysis of how and on what data models can be trained to improve precise instruction following generalization. Specifically, we carefully design constraint verification modules and show that reinforcement learning with verifiable rewards (RLVR) significantly improves instruction following. In addition to IFBench, we release 29 additional new hand-annotated training constraints and verification functions, RLVR training prompts, and code.

Abstract of Generalizing Verifiable Instruction Following (arXiv:2507.02833, CC0)
82
Models scored in lab cards
69
Launch posts list it, from 23 labs
43
System cards report it, from 17 labs

Flint leads with 89.2%

Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.

Llama-3.1-Tulu-3-8B leads with 28.9% across all independent runs

Best score per model across the benchmark’s paper, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.

Flint leads with 89.2%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.

82 models. Scroll sideways for the rest.

  1. 89.2 Flint
  2. 82.9 Nemotron-Cascade 2 30B-A3B
  3. 82.8 Qwen 3.8 Max
  4. 82.2 Inkling-Small
  5. 81.7 Nemotron 3 Ultra
  6. 81.5 Qwen3.8-Omni-Flash
  7. 81.3 Qwen3.8 Flash
  8. 80.2 Nova 2.0 Pro Preview
  9. 80.0 Solar Open 2
  10. 79.8 Inkling
  11. 79.7 Beam 1.0
  12. 79.5 Qwen 3.8 27B
  13. 79.1 Qwen3.7-Max
  14. 79.1 Qwen3.7-Plus
  15. 77.2 Granite 4.2 30B
  16. 77.0 Muse Glimmer
  17. 77.0 Mercury 2.5
  18. 76.8 Mercury 3
  19. 76.5 Qwen3.5 397B-A17B
  20. 76.5 Qwen3.5-27B
  21. 76.1 Qwen3.5-122B-A10B
  22. 76.0 MiniMax-M2.7
  23. 76.0 Gemma 4 31B IT
  24. 75.8 Spark-X2.6-4B
  25. 75.0 MAI-Code-1-Flash
  26. 74.7 LFM2.5-24B-A2B
  27. 74.2 Qwen 3.6 Plus
  28. 74.2 Nemotron 3 Nano Omni 30B-A3B
  29. 74.0 Command A+
  30. 73.1 Solar Pro 3 (March 2026 update)
  31. 73.0 LFM3-1.7B
  32. 72.6 K-EXAONE 2.0
  33. 72.6 Nemotron 3 Super
  34. 72.0 MiniMax-M2
  35. 72.0 GLM-5
  36. 72.0 MiniMax-M2.5
  37. 72.0 Gemma 4 26B A4B
  38. 72.0 Nemotron 3.5 Lightning 30B-A3B
  39. 71.5 Nemotron 3 Nano 30B-A3B
  40. 71.2 Intern-S1-Pro
  41. 71.0 Mercury 2
  42. 70.9 Qwen3-Max-2026-01-23
  43. 70.9 Nemotron 3.5 Super VL
  44. 70.8 Nova 2 Lite
  45. 70.2 Qwen3.5-35B-A3B
  46. 70.0 MiniMax-M2.1
  47. 69.0 Mistral Medium 3.5
  48. 69.0 MAI-Thinking-1
  49. 68.7 Nova 2 Omni Preview
  50. 68.1 Olmo 3.1 32B Think
  51. 67.4 Step 3.5 Flash
  52. 67.3 K-EXAONE
  53. 66.2 Nemotron 3.5 Bolt 74B-A9B
  54. 64.5 Qwen3.5-9B
  55. 64.0 Nemotron 3.5 Bolt E4B
  56. 62.6 EXAONE 4.5 33B
  57. 59.2 Qwen3.5-4B
  58. 59.2 LFM2.5-2.6B
  59. 57.6 Ling-2.6-1T
  60. 56.5 LFM2.5-8B-A1B
  61. 55.8 Solar Pro 3
  62. 53.7 Solar Open 100B
  63. 53.4 Falcon H1R 7B
  64. 52.6 Qwen3.5-Omni-Plus
  65. 52.3 Trinity Large Thinking
  66. 48.0 Mistral Small 4
  67. 47.7 MiniCPM-V-4.7-35B-A3B
  68. 47.6 Olmo 3 32B Think
  69. 47.3 LFM2.5-1.2B-Instruct
  70. 44.9 LFM2.5-1.2B-Thinking
  71. 44.0 Gemma 4 E4B
  72. 43.3 STEP3-VL-10B
  73. 43.2 Nemotron 3 Nano 4B
  74. 41.3 Qwen3.5-2B
  75. 38.0 Gemma 4 E2B
  76. 34.6 nvidia-nemotron-nano-9b-v2
  77. 32.0 Cosmos3-Nano
  78. 25.9 LFM2-8B-A1B
  79. 25.8 LFM2.5-VL-3B
  80. 21.0 Qwen3.5-0.8B
  81. 20.8 Cosmos3-Edge
  82. 20.7 LFM2-1.2B
  1. 1 Flint 89.2%
  2. 2 Nemotron-Cascade 2 30B-A3B 82.9%
  3. 3 Qwen 3.8 Max 82.8%
  4. 4 Inkling-Small 82.2%
  5. 5 Nemotron 3 Ultra 81.7%
  6. 6 Qwen3.8-Omni-Flash 81.5%
  7. 7 Qwen3.8 Flash 81.3%
  8. 8 Nova 2.0 Pro Preview 80.2%
  9. 9 Solar Open 2 80%
  10. 10 Inkling 79.8%
  11. 11 Beam 1.0 79.7%
  12. 12 Qwen 3.8 27B 79.5%
  13. 13 Qwen3.7-Max 79.1%
  14. 14 Qwen3.7-Plus 79.1%
  15. 15 Granite 4.2 30B 77.2%
  16. 16 Muse Glimmer 77%
  17. 17 Mercury 2.5 77%
  18. 18 Mercury 3 76.8%
  19. 19 Qwen3.5 397B-A17B 76.5%
  20. 20 Qwen3.5-27B 76.5%
  21. 21 Qwen3.5-122B-A10B 76.1%
  22. 22 MiniMax-M2.7 76%
  23. 23 Gemma 4 31B IT 76%
  24. 24 Spark-X2.6-4B 75.8%
  25. 25 MAI-Code-1-Flash 75%
  26. 26 LFM2.5-24B-A2B 74.7%
  27. 27 Qwen 3.6 Plus 74.2%
  28. 28 Nemotron 3 Nano Omni 30B-A3B 74.2%
  29. 29 Command A+ 74%
  30. 30 Solar Pro 3 (March 2026 update) 73.1%
  31. 31 LFM3-1.7B 73%
  32. 32 K-EXAONE 2.0 72.6%
  33. 33 Nemotron 3 Super 72.6%
  34. 34 MiniMax-M2 72%
  35. 35 GLM-5 72%
  36. 36 MiniMax-M2.5 72%
  37. 37 Gemma 4 26B A4B 72%
  38. 38 Nemotron 3.5 Lightning 30B-A3B 72%
  39. 39 Nemotron 3 Nano 30B-A3B 71.5%
  40. 40 Intern-S1-Pro 71.2%
  41. 41 Mercury 2 71%
  42. 42 Qwen3-Max-2026-01-23 70.9%
  43. 43 Nemotron 3.5 Super VL 70.9%
  44. 44 Nova 2 Lite 70.8%
  45. 45 Qwen3.5-35B-A3B 70.2%
  46. 46 MiniMax-M2.1 70%
  47. 47 Mistral Medium 3.5 69%
  48. 48 MAI-Thinking-1 69%
  49. 49 Nova 2 Omni Preview 68.7%
  50. 50 Olmo 3.1 32B Think 68.1%
  51. 51 Step 3.5 Flash 67.4%
  52. 52 K-EXAONE 67.3%
  53. 53 Nemotron 3.5 Bolt 74B-A9B 66.2%
  54. 54 Qwen3.5-9B 64.5%
  55. 55 Nemotron 3.5 Bolt E4B 64%
  56. 56 EXAONE 4.5 33B 62.6%
  57. 57 Qwen3.5-4B 59.2%
  58. 58 LFM2.5-2.6B 59.2%
  59. 59 Ling-2.6-1T 57.6%
  60. 60 LFM2.5-8B-A1B 56.5%
  61. 61 Solar Pro 3 55.8%
  62. 62 Solar Open 100B 53.7%
  63. 63 Falcon H1R 7B 53.4%
  64. 64 Qwen3.5-Omni-Plus 52.6%
  65. 65 Trinity Large Thinking 52.3%
  66. 66 Mistral Small 4 48%
  67. 67 MiniCPM-V-4.7-35B-A3B 47.7%
  68. 68 Olmo 3 32B Think 47.6%
  69. 69 LFM2.5-1.2B-Instruct 47.3%
  70. 70 LFM2.5-1.2B-Thinking 44.9%
  71. 71 Gemma 4 E4B 44%
  72. 72 STEP3-VL-10B 43.3%
  73. 73 Nemotron 3 Nano 4B 43.2%
  74. 74 Qwen3.5-2B 41.3%
  75. 75 Gemma 4 E2B 38%
  76. 76 nvidia-nemotron-nano-9b-v2 34.6%
  77. 77 Cosmos3-Nano 32%
  78. 78 LFM2-8B-A1B 25.9%
  79. 79 LFM2.5-VL-3B 25.8%
  80. 80 Qwen3.5-0.8B 21%
  81. 81 Cosmos3-Edge 20.8%
  82. 82 LFM2-1.2B 20.7%

Source: Labs’ model cards

Source: The IFBench paper

83 models. Scroll sideways for the rest.

  1. 89.2 Flint
  2. 82.9 Nemotron-Cascade 2 30B-A3B
  3. 82.8 Qwen 3.8 Max
  4. 82.2 Inkling-Small
  5. 81.7 Nemotron 3 Ultra
  6. 81.5 Qwen3.8-Omni-Flash
  7. 81.3 Qwen3.8 Flash
  8. 80.2 Nova 2.0 Pro Preview
  9. 80.0 Solar Open 2
  10. 79.8 Inkling
  11. 79.7 Beam 1.0
  12. 79.5 Qwen 3.8 27B
  13. 79.1 Qwen3.7-Max
  14. 79.1 Qwen3.7-Plus
  15. 77.2 Granite 4.2 30B
  16. 77.0 Muse Glimmer
  17. 77.0 Mercury 2.5
  18. 76.8 Mercury 3
  19. 76.5 Qwen3.5 397B-A17B
  20. 76.5 Qwen3.5-27B
  21. 76.1 Qwen3.5-122B-A10B
  22. 76.0 MiniMax-M2.7
  23. 76.0 Gemma 4 31B IT
  24. 75.8 Spark-X2.6-4B
  25. 75.0 MAI-Code-1-Flash
  26. 74.7 LFM2.5-24B-A2B
  27. 74.2 Qwen 3.6 Plus
  28. 74.2 Nemotron 3 Nano Omni 30B-A3B
  29. 74.0 Command A+
  30. 73.1 Solar Pro 3 (March 2026 update)
  31. 73.0 LFM3-1.7B
  32. 72.6 K-EXAONE 2.0
  33. 72.6 Nemotron 3 Super
  34. 72.0 MiniMax-M2
  35. 72.0 GLM-5
  36. 72.0 MiniMax-M2.5
  37. 72.0 Gemma 4 26B A4B
  38. 72.0 Nemotron 3.5 Lightning 30B-A3B
  39. 71.5 Nemotron 3 Nano 30B-A3B
  40. 71.2 Intern-S1-Pro
  41. 71.0 Mercury 2
  42. 70.9 Qwen3-Max-2026-01-23
  43. 70.9 Nemotron 3.5 Super VL
  44. 70.8 Nova 2 Lite
  45. 70.2 Qwen3.5-35B-A3B
  46. 70.0 MiniMax-M2.1
  47. 69.0 Mistral Medium 3.5
  48. 69.0 MAI-Thinking-1
  49. 68.7 Nova 2 Omni Preview
  50. 68.1 Olmo 3.1 32B Think
  51. 67.4 Step 3.5 Flash
  52. 67.3 K-EXAONE
  53. 66.2 Nemotron 3.5 Bolt 74B-A9B
  54. 64.5 Qwen3.5-9B
  55. 64.0 Nemotron 3.5 Bolt E4B
  56. 62.6 EXAONE 4.5 33B
  57. 59.2 Qwen3.5-4B
  58. 59.2 LFM2.5-2.6B
  59. 57.6 Ling-2.6-1T
  60. 56.5 LFM2.5-8B-A1B
  61. 55.8 Solar Pro 3
  62. 53.7 Solar Open 100B
  63. 53.4 Falcon H1R 7B
  64. 52.6 Qwen3.5-Omni-Plus
  65. 52.3 Trinity Large Thinking
  66. 48.0 Mistral Small 4
  67. 47.7 MiniCPM-V-4.7-35B-A3B
  68. 47.6 Olmo 3 32B Think
  69. 47.3 LFM2.5-1.2B-Instruct
  70. 44.9 LFM2.5-1.2B-Thinking
  71. 44.0 Gemma 4 E4B
  72. 43.3 STEP3-VL-10B
  73. 43.2 Nemotron 3 Nano 4B
  74. 41.3 Qwen3.5-2B
  75. 38.0 Gemma 4 E2B
  76. 34.6 nvidia-nemotron-nano-9b-v2
  77. 32.0 Cosmos3-Nano
  78. 28.9 Llama-3.1-Tulu-3-8B
  79. 25.9 LFM2-8B-A1B
  80. 25.8 LFM2.5-VL-3B
  81. 21.0 Qwen3.5-0.8B
  82. 20.8 Cosmos3-Edge
  83. 20.7 LFM2-1.2B
  1. 1 Flint 89.2%
  2. 2 Nemotron-Cascade 2 30B-A3B 82.9%
  3. 3 Qwen 3.8 Max 82.8%
  4. 4 Inkling-Small 82.2%
  5. 5 Nemotron 3 Ultra 81.7%
  6. 6 Qwen3.8-Omni-Flash 81.5%
  7. 7 Qwen3.8 Flash 81.3%
  8. 8 Nova 2.0 Pro Preview 80.2%
  9. 9 Solar Open 2 80%
  10. 10 Inkling 79.8%
  11. 11 Beam 1.0 79.7%
  12. 12 Qwen 3.8 27B 79.5%
  13. 13 Qwen3.7-Max 79.1%
  14. 14 Qwen3.7-Plus 79.1%
  15. 15 Granite 4.2 30B 77.2%
  16. 16 Muse Glimmer 77%
  17. 17 Mercury 2.5 77%
  18. 18 Mercury 3 76.8%
  19. 19 Qwen3.5 397B-A17B 76.5%
  20. 20 Qwen3.5-27B 76.5%
  21. 21 Qwen3.5-122B-A10B 76.1%
  22. 22 MiniMax-M2.7 76%
  23. 23 Gemma 4 31B IT 76%
  24. 24 Spark-X2.6-4B 75.8%
  25. 25 MAI-Code-1-Flash 75%
  26. 26 LFM2.5-24B-A2B 74.7%
  27. 27 Qwen 3.6 Plus 74.2%
  28. 28 Nemotron 3 Nano Omni 30B-A3B 74.2%
  29. 29 Command A+ 74%
  30. 30 Solar Pro 3 (March 2026 update) 73.1%
  31. 31 LFM3-1.7B 73%
  32. 32 K-EXAONE 2.0 72.6%
  33. 33 Nemotron 3 Super 72.6%
  34. 34 MiniMax-M2 72%
  35. 35 GLM-5 72%
  36. 36 MiniMax-M2.5 72%
  37. 37 Gemma 4 26B A4B 72%
  38. 38 Nemotron 3.5 Lightning 30B-A3B 72%
  39. 39 Nemotron 3 Nano 30B-A3B 71.5%
  40. 40 Intern-S1-Pro 71.2%
  41. 41 Mercury 2 71%
  42. 42 Qwen3-Max-2026-01-23 70.9%
  43. 43 Nemotron 3.5 Super VL 70.9%
  44. 44 Nova 2 Lite 70.8%
  45. 45 Qwen3.5-35B-A3B 70.2%
  46. 46 MiniMax-M2.1 70%
  47. 47 Mistral Medium 3.5 69%
  48. 48 MAI-Thinking-1 69%
  49. 49 Nova 2 Omni Preview 68.7%
  50. 50 Olmo 3.1 32B Think 68.1%
  51. 51 Step 3.5 Flash 67.4%
  52. 52 K-EXAONE 67.3%
  53. 53 Nemotron 3.5 Bolt 74B-A9B 66.2%
  54. 54 Qwen3.5-9B 64.5%
  55. 55 Nemotron 3.5 Bolt E4B 64%
  56. 56 EXAONE 4.5 33B 62.6%
  57. 57 Qwen3.5-4B 59.2%
  58. 58 LFM2.5-2.6B 59.2%
  59. 59 Ling-2.6-1T 57.6%
  60. 60 LFM2.5-8B-A1B 56.5%
  61. 61 Solar Pro 3 55.8%
  62. 62 Solar Open 100B 53.7%
  63. 63 Falcon H1R 7B 53.4%
  64. 64 Qwen3.5-Omni-Plus 52.6%
  65. 65 Trinity Large Thinking 52.3%
  66. 66 Mistral Small 4 48%
  67. 67 MiniCPM-V-4.7-35B-A3B 47.7%
  68. 68 Olmo 3 32B Think 47.6%
  69. 69 LFM2.5-1.2B-Instruct 47.3%
  70. 70 LFM2.5-1.2B-Thinking 44.9%
  71. 71 Gemma 4 E4B 44%
  72. 72 STEP3-VL-10B 43.3%
  73. 73 Nemotron 3 Nano 4B 43.2%
  74. 74 Qwen3.5-2B 41.3%
  75. 75 Gemma 4 E2B 38%
  76. 76 nvidia-nemotron-nano-9b-v2 34.6%
  77. 77 Cosmos3-Nano 32%
  78. 78 Llama-3.1-Tulu-3-8B single-turn; paper Table 3 28.9%
  79. 79 LFM2-8B-A1B 25.9%
  80. 80 LFM2.5-VL-3B 25.8%
  81. 81 Qwen3.5-0.8B 21%
  82. 82 Cosmos3-Edge 20.8%
  83. 83 LFM2-1.2B 20.7%

Sources: Labs’ model cards and the IFBench paper

#ModelScoreSettingDateSource
1 Flint 89.2% Oct 2026 Lab model card
2 Nemotron-Cascade 2 30B-A3B 82.9% Mar 2026 Lab model card
3 Qwen 3.8 Max 82.8% Aug 2026 Lab model card
4 Inkling-Small 82.2% Jul 2026 Lab model card
5 Nemotron 3 Ultra 81.7% Jun 2026 Lab model card
6 Qwen3.8-Omni-Flash 81.5% Sep 2026 Lab model card
7 Qwen3.8 Flash 81.3% Aug 2026 Lab model card
8 Nova 2.0 Pro Preview 80.2% Dec 2025 Lab model card
9 Solar Open 2 80% Jul 2026 Lab model card
10 Inkling 79.8% Jul 2026 Lab model card
11 Beam 1.0 79.7% Oct 2026 Lab model card
12 Qwen 3.8 27B 79.5% Aug 2026 Lab model card
13 Qwen3.7-Max 79.1% May 2026 Lab model card
14 Qwen3.7-Plus 79.1% Jun 2026 Lab model card
15 Granite 4.2 30B 77.2% Aug 2026 Lab model card
16 Muse Glimmer 77% Aug 2026 Lab model card
17 Mercury 2.5 77% Sep 2026 Lab model card
18 Mercury 3 76.8% Oct 2026 Lab model card
19 Qwen3.5 397B-A17B 76.5% Feb 2026 Lab model card
20 Qwen3.5-27B 76.5% Feb 2026 Lab model card
21 Qwen3.5-122B-A10B 76.1% Feb 2026 Lab model card
22 MiniMax-M2.7 76% May 2026 Lab model card
23 Gemma 4 31B IT 76% Jul 2026 Lab model card
24 Spark-X2.6-4B 75.8% Oct 2026 Lab model card
25 MAI-Code-1-Flash 75% Jun 2026 Lab model card
26 LFM2.5-24B-A2B 74.7% Oct 2026 Lab model card
27 Qwen 3.6 Plus 74.2% Apr 2026 Lab model card
28 Nemotron 3 Nano Omni 30B-A3B 74.2% Apr 2026 Lab model card
29 Command A+ 74% May 2026 Lab model card
30 Solar Pro 3 (March 2026 update) 73.1% Mar 2026 Lab model card
31 LFM3-1.7B 73% Oct 2026 Lab model card
32 K-EXAONE 2.0 72.6% Jul 2026 Lab model card
33 Nemotron 3 Super 72.6% Mar 2026 Lab model card
34 MiniMax-M2 72% Oct 2025 Lab model card
35 GLM-5 72% Feb 2026 Lab model card
36 MiniMax-M2.5 72% May 2026 Lab model card
37 Gemma 4 26B A4B 72% Jul 2026 Lab model card
38 Nemotron 3.5 Lightning 30B-A3B 72% Oct 2026 Lab model card
39 Nemotron 3 Nano 30B-A3B 71.5% Dec 2025 Lab model card
40 Intern-S1-Pro 71.2% Feb 2026 Lab model card
41 Mercury 2 71% Feb 2026 Lab model card
42 Qwen3-Max-2026-01-23 70.9% Jan 2026 Lab model card
43 Nemotron 3.5 Super VL 70.9% Oct 2026 Lab model card
44 Nova 2 Lite 70.8% Dec 2025 Lab model card
45 Qwen3.5-35B-A3B 70.2% Feb 2026 Lab model card
46 MiniMax-M2.1 70% Dec 2025 Lab model card
47 Mistral Medium 3.5 69% May 2026 Lab model card
48 MAI-Thinking-1 69% Jun 2026 Lab model card
49 Nova 2 Omni Preview 68.7% Dec 2025 Lab model card
50 Olmo 3.1 32B Think 68.1% Dec 2025 Lab model card
51 Step 3.5 Flash 67.4% Feb 2026 Lab model card
52 K-EXAONE 67.3% Dec 2025 Lab model card
53 Nemotron 3.5 Bolt 74B-A9B 66.2% Oct 2026 Lab model card
54 Qwen3.5-9B 64.5% Mar 2026 Lab model card
55 Nemotron 3.5 Bolt E4B 64% Oct 2026 Lab model card
56 EXAONE 4.5 33B 62.6% Apr 2026 Lab model card
57 Qwen3.5-4B 59.2% Mar 2026 Lab model card
58 LFM2.5-2.6B 59.2% Aug 2026 Lab model card
59 Ling-2.6-1T 57.6% Jun 2026 Lab model card
60 LFM2.5-8B-A1B 56.5% May 2026 Lab model card
61 Solar Pro 3 55.8% Jan 2026 Lab model card
62 Solar Open 100B 53.7% Jan 2026 Lab model card
63 Falcon H1R 7B 53.4% Jan 2026 Lab model card
64 Qwen3.5-Omni-Plus 52.6% Mar 2026 Lab model card
65 Trinity Large Thinking 52.3% Apr 2026 Lab model card
66 Mistral Small 4 48% Mar 2026 Lab model card
67 MiniCPM-V-4.7-35B-A3B 47.7% Oct 2026 Lab model card
68 Olmo 3 32B Think 47.6% Nov 2025 Lab model card
69 LFM2.5-1.2B-Instruct 47.3% Jan 2026 Lab model card
70 LFM2.5-1.2B-Thinking 44.9% Jan 2026 Lab model card
71 Gemma 4 E4B 44% Jul 2026 Lab model card
72 STEP3-VL-10B 43.3% Jan 2026 Lab model card
73 Nemotron 3 Nano 4B 43.2% Mar 2026 Lab model card
74 Qwen3.5-2B 41.3% Mar 2026 Lab model card
75 Gemma 4 E2B 38% Jul 2026 Lab model card
76 nvidia-nemotron-nano-9b-v2 34.6% Aug 2025 Lab model card
77 Cosmos3-Nano 32% May 2026 Lab model card
78 LFM2-8B-A1B 25.9% Oct 2025 Lab model card
79 LFM2.5-VL-3B 25.8% Aug 2026 Lab model card
80 Qwen3.5-0.8B 21% Mar 2026 Lab model card
81 Cosmos3-Edge 20.8% May 2026 Lab model card
82 LFM2-1.2B 20.7% Jul 2025 Lab model card
#ModelScoreSettingDateSource
1 Llama-3.1-Tulu-3-8B 28.9% single-turn; paper Table 3 Nov 2024 Official leaderboard
#ModelScoreSettingDateSource
1 Flint 89.2% Oct 2026 Lab model card
2 Nemotron-Cascade 2 30B-A3B 82.9% Mar 2026 Lab model card
3 Qwen 3.8 Max 82.8% Aug 2026 Lab model card
4 Inkling-Small 82.2% Jul 2026 Lab model card
5 Nemotron 3 Ultra 81.7% Jun 2026 Lab model card
6 Qwen3.8-Omni-Flash 81.5% Sep 2026 Lab model card
7 Qwen3.8 Flash 81.3% Aug 2026 Lab model card
8 Nova 2.0 Pro Preview 80.2% Dec 2025 Lab model card
9 Solar Open 2 80% Jul 2026 Lab model card
10 Inkling 79.8% Jul 2026 Lab model card
11 Beam 1.0 79.7% Oct 2026 Lab model card
12 Qwen 3.8 27B 79.5% Aug 2026 Lab model card
13 Qwen3.7-Max 79.1% May 2026 Lab model card
14 Qwen3.7-Plus 79.1% Jun 2026 Lab model card
15 Granite 4.2 30B 77.2% Aug 2026 Lab model card
16 Muse Glimmer 77% Aug 2026 Lab model card
17 Mercury 2.5 77% Sep 2026 Lab model card
18 Mercury 3 76.8% Oct 2026 Lab model card
19 Qwen3.5 397B-A17B 76.5% Feb 2026 Lab model card
20 Qwen3.5-27B 76.5% Feb 2026 Lab model card
21 Qwen3.5-122B-A10B 76.1% Feb 2026 Lab model card
22 MiniMax-M2.7 76% May 2026 Lab model card
23 Gemma 4 31B IT 76% Jul 2026 Lab model card
24 Spark-X2.6-4B 75.8% Oct 2026 Lab model card
25 MAI-Code-1-Flash 75% Jun 2026 Lab model card
26 LFM2.5-24B-A2B 74.7% Oct 2026 Lab model card
27 Qwen 3.6 Plus 74.2% Apr 2026 Lab model card
28 Nemotron 3 Nano Omni 30B-A3B 74.2% Apr 2026 Lab model card
29 Command A+ 74% May 2026 Lab model card
30 Solar Pro 3 (March 2026 update) 73.1% Mar 2026 Lab model card
31 LFM3-1.7B 73% Oct 2026 Lab model card
32 K-EXAONE 2.0 72.6% Jul 2026 Lab model card
33 Nemotron 3 Super 72.6% Mar 2026 Lab model card
34 MiniMax-M2 72% Oct 2025 Lab model card
35 GLM-5 72% Feb 2026 Lab model card
36 MiniMax-M2.5 72% May 2026 Lab model card
37 Gemma 4 26B A4B 72% Jul 2026 Lab model card
38 Nemotron 3.5 Lightning 30B-A3B 72% Oct 2026 Lab model card
39 Nemotron 3 Nano 30B-A3B 71.5% Dec 2025 Lab model card
40 Intern-S1-Pro 71.2% Feb 2026 Lab model card
41 Mercury 2 71% Feb 2026 Lab model card
42 Qwen3-Max-2026-01-23 70.9% Jan 2026 Lab model card
43 Nemotron 3.5 Super VL 70.9% Oct 2026 Lab model card
44 Nova 2 Lite 70.8% Dec 2025 Lab model card
45 Qwen3.5-35B-A3B 70.2% Feb 2026 Lab model card
46 MiniMax-M2.1 70% Dec 2025 Lab model card
47 Mistral Medium 3.5 69% May 2026 Lab model card
48 MAI-Thinking-1 69% Jun 2026 Lab model card
49 Nova 2 Omni Preview 68.7% Dec 2025 Lab model card
50 Olmo 3.1 32B Think 68.1% Dec 2025 Lab model card
51 Step 3.5 Flash 67.4% Feb 2026 Lab model card
52 K-EXAONE 67.3% Dec 2025 Lab model card
53 Nemotron 3.5 Bolt 74B-A9B 66.2% Oct 2026 Lab model card
54 Qwen3.5-9B 64.5% Mar 2026 Lab model card
55 Nemotron 3.5 Bolt E4B 64% Oct 2026 Lab model card
56 EXAONE 4.5 33B 62.6% Apr 2026 Lab model card
57 Qwen3.5-4B 59.2% Mar 2026 Lab model card
58 LFM2.5-2.6B 59.2% Aug 2026 Lab model card
59 Ling-2.6-1T 57.6% Jun 2026 Lab model card
60 LFM2.5-8B-A1B 56.5% May 2026 Lab model card
61 Solar Pro 3 55.8% Jan 2026 Lab model card
62 Solar Open 100B 53.7% Jan 2026 Lab model card
63 Falcon H1R 7B 53.4% Jan 2026 Lab model card
64 Qwen3.5-Omni-Plus 52.6% Mar 2026 Lab model card
65 Trinity Large Thinking 52.3% Apr 2026 Lab model card
66 Mistral Small 4 48% Mar 2026 Lab model card
67 MiniCPM-V-4.7-35B-A3B 47.7% Oct 2026 Lab model card
68 Olmo 3 32B Think 47.6% Nov 2025 Lab model card
69 LFM2.5-1.2B-Instruct 47.3% Jan 2026 Lab model card
70 LFM2.5-1.2B-Thinking 44.9% Jan 2026 Lab model card
71 Gemma 4 E4B 44% Jul 2026 Lab model card
72 STEP3-VL-10B 43.3% Jan 2026 Lab model card
73 Nemotron 3 Nano 4B 43.2% Mar 2026 Lab model card
74 Qwen3.5-2B 41.3% Mar 2026 Lab model card
75 Gemma 4 E2B 38% Jul 2026 Lab model card
76 nvidia-nemotron-nano-9b-v2 34.6% Aug 2025 Lab model card
77 Cosmos3-Nano 32% May 2026 Lab model card
78 Llama-3.1-Tulu-3-8B 28.9% single-turn; paper Table 3 Nov 2024 Official leaderboard
79 LFM2-8B-A1B 25.9% Oct 2025 Lab model card
80 LFM2.5-VL-3B 25.8% Aug 2026 Lab model card
81 Qwen3.5-0.8B 21% Mar 2026 Lab model card
82 Cosmos3-Edge 20.8% May 2026 Lab model card
83 LFM2-1.2B 20.7% Jul 2025 Lab model card

The benchmark's paper has 5 entries, 1 of them models we track. Switch the scores above to rank them together.

6 models have held number one in labs’ own numbers since its release in July 2025 Number one over time 6 models have held number one with claims included since its release in July 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

Liquid AI Other labs MiniMax Amazon NVIDIA Recursive Superintelligence
Lab claims Passed 85% in Oct 2026, by Flint
Flint 89.2%

Source: Labs’ model cards

Recursive Superintelligence’s own number for Flint passed 85% in October 2026

That was 15 months after its release. By the same age, 24% of the benchmarks from 2025 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
In the benchmark’s paper Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Flint Nemotron-Cascade 2 30B-A3B MiniMax-M2 nvidia-nemotron-nano-9b-v2 Llama-3.1-Tulu-3-8B 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Flint Nemotron-Cascade 2 30B-A3B

Sources: Official IFBench leaderboard and labs’ model cards

14 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Liquid AI
MiniMax
Ai2
Amazon
NVIDIA
LG AI Research
Upstage
Technology Innovation Institute
Qwen
Shanghai AI Lab
Inception Labs
Mistral AI
Arcee AI
Ant Group
Cohere
Microsoft
Thinking Machines
Meta
IBM
Reflection
OpenBMB (ModelBest)
Recursive Superintelligence
iFlytek
DateLaunch postScoreAs the post writes it
Oct 27, 2026 LFM3-1.7B 73 IFBench
Oct 23, 2026 Nemotron 3.5 Bolt E4B 64 IFBench
Oct 22, 2026 Spark-X2.6-4B 75.8 IFBench
Oct 21, 2026 Flint 89.2 IFBench
Oct 19, 2026 Mercury 3 76.8 IFBench
Oct 15, 2026 Nemotron 3.5 Super VL 70.9 IFBench
Oct 13, 2026 LFM2.5-24B-A2B 74.7 IFBench
Oct 7, 2026 MiniCPM-V-4.7-35B-A3B 47.7 IFBench
Oct 5, 2026 Beam 1.0 79.7 IFBench
Sep 18, 2026 Qwen3.8-Omni-Flash 81.5 IFBench
Sep 8, 2026 Mercury 2.5 77 IFBench
Aug 26, 2026 Qwen3.8 Flash 81.3 Instruction following IFBench
Aug 25, 2026 Granite 4.2 30B 77.2 IFBench (prompt)
Aug 14, 2026 Qwen 3.8 27B 79.5 Instruction following IFBench
Aug 12, 2026 LFM2.5-VL-3B 25.8 IFBench
Aug 11, 2026 Nemotron 3.5 Lightning 30B-A3B 71.9 IFBench (loose)
Aug 10, 2026 Muse Glimmer 77 IFBench
Aug 4, 2026 LFM2.5-2.6B 59.2 IFBench
Aug 3, 2026 Qwen 3.8 Max 82.8 IFBench
Jul 31, 2026 K-EXAONE 2.0 72.6 IFBench
Jul 30, 2026 Inkling-Small 82.2 IFBench
Jul 22, 2026 Solar Open 2 80 IFBench
Jul 15, 2026 Inkling 79.8 IFBench
Jun 4, 2026 Nemotron 3 Ultra 81.7 IFBench (prompt loose)
Jun 2, 2026 MAI-Code-1-Flash 75 IF Bench (average)
Jun 2, 2026 MAI-Thinking-1 69 IF Bench
Jun 1, 2026 Qwen3.7-Plus 79.1 IFBench
May 28, 2026 LFM2.5-8B-A1B 56.5 IFBench
May 22, 2026 Mistral Medium 3.5 69 Allenai Ifbench
May 20, 2026 Command A+ 74 IFBench (single turn, loose, prompt accuracy)
May 20, 2026 Qwen3.7-Max 79.1 IFBench
Apr 29, 2026 Ling-2.6-1T 57 IFBench
Apr 9, 2026 EXAONE 4.5 33B 62.6 IFBench
Apr 2, 2026 Qwen 3.6 Plus 74.2 IFBench
Apr 1, 2026 Trinity Large Thinking 52.3 IFBench
Mar 30, 2026 Qwen3.5-Omni-Plus 52.6 IFBench
Mar 24, 2026 Solar Pro 3 (March 2026 update) 73.1 IFBench
Mar 19, 2026 Nemotron-Cascade 2 30B-A3B 82.9 IFBench (prompt)
Mar 16, 2026 Nemotron 3 Nano 4B 43.2 IFBench-Prompt (reasoning off); Instruction 44.2
Mar 16, 2026 Mistral Small 4 48 AllenAI IFBench (reasoning; instruct = 35.7)
Mar 11, 2026 Nemotron 3 Super 72.6 IFBench (prompt)
Mar 2, 2026 Qwen3.5-0.8B 21 IFBench
Mar 2, 2026 Qwen3.5-2B 41.3 IFBench
Mar 2, 2026 Qwen3.5-4B 59.2 IFBench
Mar 2, 2026 Qwen3.5-9B 64.5 IFBench
Feb 24, 2026 Qwen3.5-35B-A3B 70.2 IFBench
Feb 24, 2026 Qwen3.5-27B 76.5 IFBench
Feb 24, 2026 Mercury 2 71 IFBench
Feb 24, 2026 Qwen3.5-122B-A10B 76.1 IFBench
Feb 16, 2026 Qwen3.5 397B-A17B 76.5 IFBench
Feb 12, 2026 MiniMax-M2.5 70 IFBench
Feb 2, 2026 Intern-S1-Pro 71.2 IFBench
Jan 26, 2026 Solar Pro 3 55.8 IFBench
Jan 23, 2026 Qwen3-Max-2026-01-23 70.9 IFBench
Jan 20, 2026 LFM2.5-1.2B-Thinking 44.9 IFBench (± 0.73)
Jan 5, 2026 Falcon H1R 7B 53.4 IFBench
Jan 5, 2026 LFM2.5-1.2B-Instruct 47.3 IFBench
Jan 5, 2026 Solar Open 100B 53.7 IFBench
Dec 31, 2025 K-EXAONE 67.3 IFBench
Dec 23, 2025 MiniMax-M2.1 70 IFBench
Dec 15, 2025 Nemotron 3 Nano 30B-A3B 71.5 IFBench (prompt)
Dec 12, 2025 Olmo 3.1 32B Think 68.1 IFBench
Dec 2, 2025 Nova 2 Omni Preview 68.7 IF-Bench prompt loose acc
Dec 2, 2025 Nova 2 Lite 70.8 IF-Bench prompt loose acc
Dec 2, 2025 Nova 2.0 Pro Preview 80.2 IF-Bench prompt loose acc
Nov 20, 2025 Olmo 3 32B Think 47.6 IFBench
Oct 27, 2025 MiniMax-M2 72 IFBench (AA)
Oct 7, 2025 LFM2-8B-A1B 25.9 IFBench
Jul 10, 2025 LFM2-1.2B 20.7 IFBench

43 system cards report it, from 17 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 23, 2026 Nemotron 3.5 Bolt E4B technical report 64 IFBench, released Q4_K_M imatrix checkpoint (Table 5). BF16 63.79 (Table 3) Capabilities
Oct 19, 2026 Muse Glimmer technical report 77 IFBench, high (Tables 2 and 5) Capabilities
Oct 15, 2026 Nemotron 3.5 Super VL technical report 70.9 IFBench, strict (Table 10) Capabilities
Oct 15, 2026 Nemotron 3.5 Lightning 30B-A3B technical report 72 IFBench (loose), BF16 acc. (Table 22) Capabilities
Oct 15, 2026 Nemotron 3.5 Bolt 74B-A9B technical report 66.2 IFBench (instruction loose), released 24GB checkpoint (Table 7). BF16 68.71 Capabilities
Sep 22, 2026 Qwen3.8-Omni-Flash technical report 81.5 IFBench (Table 2) Capabilities
Aug 5, 2026 K-EXAONE 2.0 technical report 72.6 IFBench (Table 6, Reasoning mode) Capabilities
Jul 30, 2026 Inkling-Small model card 82.2 IFBench (5. Evaluations table) Capabilities
Jul 24, 2026 Gemma 4 E4B technical report 44 IFBench, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 E2B technical report 38 IFBench, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 31B IT technical report 76 IFBench, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 26B A4B technical report 72 IFBench, thinking (Table 5) Capabilities
Jul 22, 2026 Solar Open 2 technical report 80 IFBench (Table 3) Capabilities
Jul 15, 2026 Inkling model card 79.8 IFBench (5. Evaluations table, effort 0.99) Capabilities
Jun 13, 2026 Ling-2.6-1T technical report 57.6 IFBench, Mean@5 (Table 5) Capabilities
Jun 9, 2026 Nemotron 3 Ultra technical report 81.7 IFBench (prompt loose) (Table 10) Capabilities
Jun 2, 2026 MAI-Thinking-1 technical report 69 IF Bench (Table 12, rounded) Capabilities
Jun 2, 2026 MAI-Code-1-Flash model card 75 IF Bench, average (Instruction following table) Capabilities
May 31, 2026 Cosmos3-Nano technical report 32 IFBench, Cosmos3-Nano 8B column, Table 10 Capabilities
May 31, 2026 Cosmos3-Edge technical report 20.8 IFBench, Cosmos3-Edge 2B column, Table 10 Capabilities
May 26, 2026 MiniMax-M2.7 technical report 76 IFBench, AA Index v4.0 protocol (Table 4) Capabilities
May 26, 2026 MiniMax-M2.5 technical report 72 IFBench, AA Index v4.0 protocol (Table 4) Capabilities
Apr 27, 2026 Nemotron 3 Nano Omni 30B-A3B technical report 74.2 IFBench (prompt) (Table 10) Capabilities
Apr 17, 2026 Qwen3.5-Omni-Plus technical report 52.6 IFBench (Table 4) Capabilities
Apr 9, 2026 EXAONE 4.5 33B technical report 62.6 IFBench (Table 3, Reasoning mode) Capabilities
Mar 26, 2026 Intern-S1-Pro technical report 71.2 IFBench, thinking (Table 2) Capabilities
Mar 19, 2026 Nemotron-Cascade 2 30B-A3B technical report 82.9 IFBench (prompt) (Table 1) Capabilities
Mar 11, 2026 Nemotron 3 Super technical report 72.6 IFBench (prompt) (Table 5). Table 8 BF16 prints 72.58 Capabilities
Feb 17, 2026 GLM-5 technical report 72 IFBench, Instruction Following panel (Figure 11) Capabilities
Feb 11, 2026 Step 3.5 Flash technical report 67.4 IFBench (Table 5). PaCoRe 56.8 Capabilities
Jan 14, 2026 STEP3-VL-10B technical report 43.3 IFBench, SeRe (Table 2) Capabilities
Jan 11, 2026 Solar Open 100B technical report 53.7 IFBench (Table 6) Capabilities
Jan 5, 2026 Falcon H1R 7B technical report 53.4 IFBench (Table 6) Capabilities
Jan 5, 2026 K-EXAONE technical report 67.3 IFBench, prompt-loose (Table 3, Reasoning mode). Non-reasoning 40.5 (Table 4) Capabilities
Dec 23, 2025 Nemotron 3 Nano 30B-A3B technical report 71.5 IFBench (prompt) (Table 3) Capabilities
Dec 15, 2025 Olmo 3 32B Think technical report 47.6 IFBench, Final Think 3.0 column (Tables 14 and 24) Capabilities
Dec 15, 2025 Olmo 3.1 32B Think technical report 68.1 IFBench (Tables 1, 14 and 24) Capabilities
Dec 2, 2025 Nova 2 Omni Preview technical report 68.7 IF-Bench prompt loose acc (Table 5) Capabilities
Dec 2, 2025 Nova 2 Lite technical report 70.8 IF-Bench prompt loose acc (Table 1) Capabilities
Dec 2, 2025 Nova 2.0 Pro Preview technical report 80.2 IF-Bench prompt loose acc (Table 1) Capabilities
Nov 28, 2025 LFM2-8B-A1B technical report 25.9 IFBench, LFM2-8B-A1B (Table 7) Capabilities
Nov 28, 2025 LFM2-1.2B technical report 20.7 IFBench, LFM2-1.2B (Table 6) Capabilities
Aug 20, 2025 nvidia-nemotron-nano-9b-v2 technical report 34.6 IFBench (Instr. Following), NVIDIA-Nemotron-Nano-9B-v2 (Figure 1) Capabilities

3 logged changes that move scores

Changes by the maintainers, so a score from before a change may not compare with one from after it. Newest first, each with the maintainers' own source. The list may be incomplete. What we log.

  1. Grader changed July 17th, 2026

    The maintainers fixed six instruction verifiers that incorrectly accepted or rejected responses. Scores can move in either direction without changing the task set. Source

  2. Grader changed January 4th, 2026

    The maintainers fixed verifier crashes, case handling, tokenization, and word-boundary matching. The revised checks can assign different results to identical responses. Source

  3. Scores recomputed November 11th, 2025

    The maintainers revised the leaderboard after fixing IFBench verifiers. Earlier published scores can differ from the revised results. Source

Versions

More in Instructions and chat

All 144 in Instructions and chat
  • IFEval In use Models answer about 500 prompts containing verifiable instructions such as word limits or required keywords. Code checks each instruction for strict and loose accuracy. Gemma 4 31B IT 98.9%
  • MultiChallenge In use Multi-turn conversations testing instruction retention, inference memory, versioned editing and self-coherence. Graded by LLM judge with rubrics. GPT-6.1 Sol 82%
  • Multi-IF In use Models follow verifiable instructions over three-turn conversations in eight languages, extending IFEval. Scored by instruction accuracy at each turn. LFM3-1.7B 89.1%
  • AdvancedIF In use Models respond to over 1,600 prompts with complex, multi-turn, and system-level instructions. A judge model checks each response against expert-written rubrics. MAI-Thinking-1 85%
  • MM-IFEval In use Multimodal models follow image-grounded instructions with exact output constraints. Checks mix rule-based verification and a judge model. GPT-4o 64.6%
  • ComplexConstraints In use Models answer 75 expert-written professional prompts whose constraints depend on each other, fire conditionally or must be inferred. An LLM judge grades 1,559 rubric criteria one at a time, and a task passes only if all criteria pass. GPT-6 Astra 70.8%

Data audit

All 9 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
  • Pass Independent runs Scores from the benchmark’s paper (5 entries), launch posts (69 scores) and system cards (43 scores).
  • Pass Models scored 83 models we track, plus 4 other entries on its leaderboard.
  • Pass Newest score The newest is from October 2026, for LFM3-1.7B.
  • Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30).
  • Note Headroom Saturated. The best is 89.2%, past 85% since October 2026.
Leaderboard import 3 pass
  • Pass Importer We read the board on arxiv.org: 1 row for models we track and 4 other entries at the last read.
  • Pass Last read November 18th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check