Benchmark Almanac

IFEval

Gemma 4 31B IT 98.9% leads since July 2026
Grok-3 mini 95.1% leads since April 2025
Gemma 4 31B IT 98.9% leads since July 2026
In use

Models answer about 500 prompts containing verifiable instructions such as word limits or required keywords. Code checks each instruction for strict and loose accuracy.

Frontier models now score near 90 percent, so it is close to saturation.

Made by
Google
Released
November 2023
Kind
Following precise instructions
Level
Everyday
Scored
Checked automatically
Answers
Open-ended
Tasks
All public
Strongest model tested
Qwen3.7-Max #49 on the capability index
Last in a launch post
October 2026
Last model added
October 2026
Scores refreshed
November 12th, 2026

In the authors’ words

One core capability of Large Language Models (LLMs) is to follow natural language instructions. However, the evaluation of such abilities is not standardized: Human evaluations are expensive, slow, and not objectively reproducible, while LLM-based auto-evaluation is potentially biased or limited by the ability of the evaluator LLM. To overcome these issues, we introduce Instruction-Following Eval (IFEval) for large language models. IFEval is a straightforward and easy-to-reproduce evaluation benchmark. It focuses on a set of "verifiable instructions" such as "write in more than 400 words" and "mention the keyword of AI at least 3 times". We identified 25 types of those verifiable instructions and constructed around 500 prompts, with each prompt containing one or more verifiable instructions. We show evaluation results of two widely available LLMs on the market. Our code and data can be found at https://github.com/google-research/google-research/tree/master/instruction_following_eval

Abstract of Instruction-Following Evaluation for Large Language Models (arXiv:2311.07911, CC0)
135
Models scored in lab cards
101
Launch posts list it, from 34 labs
88
System cards report it, from 31 labs

Gemma 4 31B IT leads with 98.9%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Grok-3 mini leads with 95.1% across all independent runs

Best score per model across Stanford HELM’s leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Gemma 4 31B IT leads with 98.9%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

135 models. Scroll sideways for the rest.

  1. 98.9 Gemma 4 31B IT
  2. 98.5 Gemma 4 26B A4B
  3. 97.4 DiffusionGemma 26B A4B
  4. 96.8 LFM2.5-24B-A2B
  5. 96.7 Gemma 4 E4B
  6. 96.7 LFM3-1.7B
  7. 95.0 Qwen3.5-27B
  8. 94.6 Qwen3.7-Plus
  9. 94.6 Gemma 4 E2B
  10. 94.3 Qwen 3.6 Plus
  11. 94.3 Qwen3.7-Max
  12. 93.8 Olmo 3.1 32B Think
  13. 93.6 Spark-X2.6-4B
  14. 93.4 Qwen3.5-122B-A10B
  15. 93.3 ERNIE 5.0
  16. 93.2 Claude 3.7 Sonnet
  17. 92.6 Qwen3.5 397B-A17B
  18. 92.4 K-EXAONE 2.0
  19. 92.2 Keye-VL-2.0-30B-A3B
  20. 92.1 Llama 3.3 70B Instruct
  21. 91.9 Qwen3.5-35B-A3B
  22. 91.8 LFM2.5-8B-A1B
  23. 91.5 Amazon Nova Premier
  24. 91.5 Qwen3.5-9B
  25. 90.9 Cohere Command A
  26. 90.4 Gemma 3 27B
  27. 90.3 nvidia-nemotron-nano-9b-v2
  28. 90.2 Claude 3.5 Sonnet (October 2024)
  29. 90.2 Gemma 3 4B
  30. 90.0 LongCat-2.0
  31. 89.8 Kimi K2
  32. 89.8 Qwen3.5-4B
  33. 89.7 K-EXAONE
  34. 89.7 Qwen3.5-Omni-Plus
  35. 89.7 TML-Interaction-Small
  36. 89.7 LongCat-Flash-Chat
  37. 89.7 Granite 4.1 30B
  38. 89.6 EXAONE 4.5 33B
  39. 89.5 Llama-3.1-Nemotron-Ultra-253B-v1
  40. 89.4 Mistral Medium 3
  41. 89.4 Falcon-H1-34B
  42. 89.2 Llama-3.3-Nemotron-Super-49B-v1
  43. 89.1 MiniMax-Text-01
  44. 89.1 Apple Foundation Model Server (2025)
  45. 89.0 Olmo 3 32B Think
  46. 88.9 Gemma 3 12B
  47. 88.9 Qwen3-Next-80B-A3B
  48. 88.6 Llama-3.3-Nemotron-Super-49B-v1.5
  49. 88.6 Llama 3.1-405B
  50. 88.6 Hunyuan-TurboS
  51. 88.4 LFM2.5-1.2B-Thinking
  52. 88.2 Qwen3-VL-235B-A22B
  53. 88.0 ERNIE-4.5-300B-A47B
  54. 88.0 Solar Open 100B
  55. 87.9 Nemotron 3 Nano 4B
  56. 87.8 Solar Pro 2
  57. 87.8 Qwen3-235B-A22B-Thinking
  58. 87.5 Granite 4.0 H Small
  59. 87.5 Llama 3.1-70B
  60. 87.4 Seed-Thinking-v1.5
  61. 87.4 GPT-4.1
  62. 87.2 Kimi k1.5
  63. 87.0 Sarvam-M
  64. 86.9 LongCat-Flash-Thinking
  65. 86.8 sonar
  66. 86.7 Intern-S1
  67. 86.5 Qwen3-30B-A3B
  68. 86.3 Qwen2.5-VL-72B
  69. 86.3 GLM-4.5-Air
  70. 86.2 LFM2.5-1.2B-Instruct
  71. 86.1 DeepSeek-V3
  72. 86.1 GLM-4.5
  73. 86.0 Tulu 3 405B
  74. 86.0 Baichuan-M2-32B
  75. 85.9 Claude 3.5 Haiku
  76. 85.8 seed-oss-36b-instruct
  77. 85.6 OLMo 2 32B Instruct
  78. 85.4 Qwen3-14B
  79. 85.2 Ring-1T
  80. 85.0 Qwen3-32B
  81. 85.0 Qwen3-8B
  82. 84.9 Phi-4-Reasoning-plus
  83. 84.8 EXAONE 4.0 32B
  84. 84.8 Sarvam 105B
  85. 84.7 Hunyuan-A13B-Instruct
  86. 84.5 GLM-Z1-32B-0414
  87. 84.3 Qwen2.5-14B-Instruct-1M
  88. 84.3 ERNIE-4.5-21B-A3B-Thinking
  89. 84.1 Qwen2.5-72B
  90. 84.1 GPT-4.1 mini
  91. 83.7 InternVL3.5-241B-A28B
  92. 83.4 Qwen3-235B-A22B
  93. 83.4 Phi-4-reasoning
  94. 83.3 DeepSeek-R1
  95. 82.9 Mistral Small 3
  96. 82.4 LongCat-Flash-Omni
  97. 82.3 LFM2.5-VL-3B
  98. 82.2 STEP3-VL-10B
  99. 81.9 Qwen3-4B
  100. 81.5 INTELLECT-2
  101. 81.5 Hermes 4 405B
  102. 81.0 Qwen2.5-14B
  103. 81.0 Qwen3-Omni-30B-A3B
  104. 80.4 Llama 3.1-8B
  105. 80.2 Gemma 3 1B
  106. 79.5 Qwen2.5-32B
  107. 79.3 InternLM3-8B-Instruct
  108. 78.6 Qwen3.5-2B
  109. 78.5 Aya Vision 32B
  110. 78.2 Nemotron Nano 12B v2 VL
  111. 77.9 Hermes 4.3 36B
  112. 77.6 Qwen2-72B
  113. 77.6 LFM2-8B-A1B
  114. 77.4 granite-4.0-1b
  115. 76.3 Qwen-Turbo
  116. 75.2 Apertus 70B
  117. 74.9 North Micro Vision Instruct
  118. 74.9 LFM2-1.2B
  119. 74.8 Granite 3.3 8B Instruct
  120. 74.5 GPT-4.1 nano
  121. 74.3 Granite 3.2 8B Instruct
  122. 72.5 Qwen3-1.7B
  123. 71.2 Qwen2.5-7B
  124. 70.1 Phi-4 Mini
  125. 70.1 OLMo 2 1B Instruct
  126. 67.7 GLM-4
  127. 67.0 AFM-4.5B
  128. 63.8 DeepSeek-V2 (MoE-236B, May 2024)
  129. 63.0 Phi-4
  130. 61.0 MiMo-7B-RL
  131. 57.6 Falcon-Arabic-7B
  132. 55.5 DeepSeek LLM 67B
  133. 53.5 BitNet b1.58 2B4T
  134. 51.2 Gemma 3 270M
  135. 44.0 Qwen3.5-0.8B
  1. 1 Gemma 4 31B IT 98.9%
  2. 2 Gemma 4 26B A4B 98.5%
  3. 3 DiffusionGemma 26B A4B 97.4%
  4. 4 LFM2.5-24B-A2B 96.8%
  5. 5 Gemma 4 E4B 96.7%
  6. 6 LFM3-1.7B 96.7%
  7. 7 Qwen3.5-27B 95%
  8. 8 Qwen3.7-Plus 94.6%
  9. 9 Gemma 4 E2B 94.6%
  10. 10 Qwen 3.6 Plus 94.3%
  11. 11 Qwen3.7-Max 94.3%
  12. 12 Olmo 3.1 32B Think 93.8%
  13. 13 Spark-X2.6-4B 93.6%
  14. 14 Qwen3.5-122B-A10B 93.4%
  15. 15 ERNIE 5.0 93.3%
  16. 16 Claude 3.7 Sonnet 93.2%
  17. 17 Qwen3.5 397B-A17B 92.6%
  18. 18 K-EXAONE 2.0 92.4%
  19. 19 Keye-VL-2.0-30B-A3B 92.2%
  20. 20 Llama 3.3 70B Instruct 92.1%
  21. 21 Qwen3.5-35B-A3B 91.9%
  22. 22 LFM2.5-8B-A1B 91.8%
  23. 23 Amazon Nova Premier 91.5%
  24. 24 Qwen3.5-9B 91.5%
  25. 25 Cohere Command A 90.9%
  26. 26 Gemma 3 27B 90.4%
  27. 27 nvidia-nemotron-nano-9b-v2 90.3%
  28. 28 Claude 3.5 Sonnet (October 2024) 90.2%
  29. 29 Gemma 3 4B 90.2%
  30. 30 LongCat-2.0 90%
  31. 31 Kimi K2 Jul 2025 89.8%
  32. 32 Qwen3.5-4B 89.8%
  33. 33 K-EXAONE 89.7%
  34. 34 Qwen3.5-Omni-Plus 89.7%
  35. 35 TML-Interaction-Small 89.7%
  36. 36 LongCat-Flash-Chat 89.7%
  37. 37 Granite 4.1 30B 89.7%
  38. 38 EXAONE 4.5 33B 89.6%
  39. 39 Llama-3.1-Nemotron-Ultra-253B-v1 89.5%
  40. 40 Mistral Medium 3 89.4%
  41. 41 Falcon-H1-34B 89.4%
  42. 42 Llama-3.3-Nemotron-Super-49B-v1 89.2%
  43. 43 MiniMax-Text-01 89.1%
  44. 44 Apple Foundation Model Server (2025) 89.1%
  45. 45 Olmo 3 32B Think 89%
  46. 46 Gemma 3 12B 88.9%
  47. 47 Qwen3-Next-80B-A3B 88.9%
  48. 48 Llama-3.3-Nemotron-Super-49B-v1.5 88.6%
  49. 49 Llama 3.1-405B 88.6%
  50. 50 Hunyuan-TurboS 88.6%
  51. 51 LFM2.5-1.2B-Thinking 88.4%
  52. 52 Qwen3-VL-235B-A22B 88.2%
  53. 53 ERNIE-4.5-300B-A47B 88%
  54. 54 Solar Open 100B 88%
  55. 55 Nemotron 3 Nano 4B 87.9%
  56. 56 Solar Pro 2 87.8%
  57. 57 Qwen3-235B-A22B-Thinking Jul 2025 87.8%
  58. 58 Granite 4.0 H Small 87.5%
  59. 59 Llama 3.1-70B 87.5%
  60. 60 Seed-Thinking-v1.5 87.4%
  61. 61 GPT-4.1 87.4%
  62. 62 Kimi k1.5 87.2%
  63. 63 Sarvam-M 87%
  64. 64 LongCat-Flash-Thinking 86.9%
  65. 65 sonar 86.8%
  66. 66 Intern-S1 86.7%
  67. 67 Qwen3-30B-A3B 86.5%
  68. 68 Qwen2.5-VL-72B 86.3%
  69. 69 GLM-4.5-Air 86.3%
  70. 70 LFM2.5-1.2B-Instruct 86.2%
  71. 71 DeepSeek-V3 86.1%
  72. 72 GLM-4.5 86.1%
  73. 73 Tulu 3 405B 86%
  74. 74 Baichuan-M2-32B 86%
  75. 75 Claude 3.5 Haiku 85.9%
  76. 76 seed-oss-36b-instruct 85.8%
  77. 77 OLMo 2 32B Instruct 85.6%
  78. 78 Qwen3-14B 85.4%
  79. 79 Ring-1T 85.2%
  80. 80 Qwen3-32B 85%
  81. 81 Qwen3-8B 85%
  82. 82 Phi-4-Reasoning-plus 84.9%
  83. 83 EXAONE 4.0 32B 84.8%
  84. 84 Sarvam 105B 84.8%
  85. 85 Hunyuan-A13B-Instruct 84.7%
  86. 86 GLM-Z1-32B-0414 84.5%
  87. 87 Qwen2.5-14B-Instruct-1M 84.3%
  88. 88 ERNIE-4.5-21B-A3B-Thinking 84.3%
  89. 89 Qwen2.5-72B 84.1%
  90. 90 GPT-4.1 mini 84.1%
  91. 91 InternVL3.5-241B-A28B 83.7%
  92. 92 Qwen3-235B-A22B 83.4%
  93. 93 Phi-4-reasoning 83.4%
  94. 94 DeepSeek-R1 83.3%
  95. 95 Mistral Small 3 82.9%
  96. 96 LongCat-Flash-Omni 82.4%
  97. 97 LFM2.5-VL-3B 82.3%
  98. 98 STEP3-VL-10B 82.2%
  99. 99 Qwen3-4B 81.9%
  100. 100 INTELLECT-2 81.5%
  101. 101 Hermes 4 405B 81.5%
  102. 102 Qwen2.5-14B 81%
  103. 103 Qwen3-Omni-30B-A3B 81%
  104. 104 Llama 3.1-8B 80.4%
  105. 105 Gemma 3 1B 80.2%
  106. 106 Qwen2.5-32B 79.5%
  107. 107 InternLM3-8B-Instruct 79.3%
  108. 108 Qwen3.5-2B 78.6%
  109. 109 Aya Vision 32B 78.5%
  110. 110 Nemotron Nano 12B v2 VL 78.2%
  111. 111 Hermes 4.3 36B 77.9%
  112. 112 Qwen2-72B 77.6%
  113. 113 LFM2-8B-A1B 77.6%
  114. 114 granite-4.0-1b 77.4%
  115. 115 Qwen-Turbo 76.3%
  116. 116 Apertus 70B 75.2%
  117. 117 North Micro Vision Instruct 74.9%
  118. 118 LFM2-1.2B 74.9%
  119. 119 Granite 3.3 8B Instruct 74.8%
  120. 120 GPT-4.1 nano 74.5%
  121. 121 Granite 3.2 8B Instruct 74.3%
  122. 122 Qwen3-1.7B 72.5%
  123. 123 Qwen2.5-7B 71.2%
  124. 124 Phi-4 Mini 70.1%
  125. 125 OLMo 2 1B Instruct 70.1%
  126. 126 GLM-4 67.7%
  127. 127 AFM-4.5B 67%
  128. 128 DeepSeek-V2 (MoE-236B, May 2024) 63.8%
  129. 129 Phi-4 63%
  130. 130 MiMo-7B-RL 61%
  131. 131 Falcon-Arabic-7B 57.6%
  132. 132 DeepSeek LLM 67B 55.5%
  133. 133 BitNet b1.58 2B4T 53.5%
  134. 134 Gemma 3 270M 51.2%
  135. 135 Qwen3.5-0.8B 44%

Source: Labs’ model cards

60 models. Scroll sideways for the rest.

  1. 95.1 Grok-3 mini
  2. 94.9 Grok 4
  3. 93.5 GPT-5.1
  4. 93.2 GPT-5 nano
  5. 92.8 o4-mini
  6. 92.7 GPT-5 mini
  7. 91.8 Claude Opus 4
  8. 90.8 Llama 4 Maverick
  9. 90.4 GPT-4.1 mini
  10. 89.8 Gemini 2.5 Flash
  11. 89.0 Granite 4.0 H Small
  12. 88.4 Grok 3
  13. 87.7 Mistral Large 2
  14. 87.7 Gemini 3 Pro
  15. 87.5 GPT-5
  16. 87.3 Palmyra X4
  17. 86.9 o3
  18. 85.5 Claude 3.5 Sonnet (October 2024)
  19. 85.0 Kimi K2
  20. 85.0 Claude Sonnet 4.5
  21. 84.9 granite-4.0-micro
  22. 84.3 GPT-4.1 nano
  23. 84.1 Gemini 2.0 Flash
  24. 84.0 Claude Sonnet 4
  25. 84.0 Gemini 2.5 Pro
  26. 83.8 GPT-4.1
  27. 83.7 Gemini 1.5 Pro
  28. 83.6 gpt-oss-120b
  29. 83.5 Qwen3-235B-A22B-Instruct
  30. 83.4 Claude 3.7 Sonnet
  31. 83.2 DeepSeek-V3
  32. 83.1 Gemini 1.5 Flash
  33. 82.4 Gemini 2.0 Flash-Lite
  34. 82.3 Palmyra X5
  35. 82.1 Llama 3.1-70B
  36. 81.8 Llama 4 Scout
  37. 81.7 GPT-4o
  38. 81.6 Qwen3-235B-A22B
  39. 81.5 Amazon Nova Pro
  40. 81.2 GLM-4.5-Air
  41. 81.1 Llama 3.1-405B
  42. 81.0 Gemini 2.5 Flash-Lite
  43. 81.0 Qwen3-Next-80B-A3B
  44. 80.6 Qwen2.5-72B
  45. 80.3 Amazon Nova Premier
  46. 80.1 Claude Haiku 4.5
  47. 79.2 Claude 3.5 Haiku
  48. 78.4 DeepSeek-R1
  49. 78.2 GPT-4o mini
  50. 78.0 OLMo 2 32B Instruct
  51. 77.6 Amazon Nova Lite
  52. 76.0 Amazon Nova Micro
  53. 75.0 Mistral Small 3.1
  54. 74.3 Llama 3.1-8B
  55. 74.1 Qwen2.5-7B
  56. 73.2 gpt-oss-20b
  57. 72.9 Granite 3.3 8B Instruct
  58. 72.4 Mixtral 8x22B
  59. 57.5 Mixtral 8x7B
  60. 56.8 Mistral 7B v0.3
  1. 1 Grok-3 mini strict, share of instructions followed 95.1%
  2. 2 Grok 4 strict, share of instructions followed 94.9%
  3. 3 GPT-5.1 strict, share of instructions followed 93.5%
  4. 4 GPT-5 nano strict, share of instructions followed 93.2%
  5. 5 o4-mini strict, share of instructions followed 92.8%
  6. 6 GPT-5 mini strict, share of instructions followed 92.7%
  7. 7 Claude Opus 4 strict, share of instructions followed 91.8%
  8. 8 Llama 4 Maverick strict, share of instructions followed 90.8%
  9. 9 GPT-4.1 mini strict, share of instructions followed 90.4%
  10. 10 Gemini 2.5 Flash Sep 2025, strict, share of instructions followed 89.8%
  11. 11 Granite 4.0 H Small strict, share of instructions followed 89%
  12. 12 Grok 3 strict, share of instructions followed 88.4%
  13. 13 Mistral Large 2 Nov 2024, strict, share of instructions followed 87.7%
  14. 14 Gemini 3 Pro strict, share of instructions followed 87.7%
  15. 15 GPT-5 strict, share of instructions followed 87.5%
  16. 16 Palmyra X4 strict, share of instructions followed 87.3%
  17. 17 o3 strict, share of instructions followed 86.9%
  18. 18 Claude 3.5 Sonnet (October 2024) strict, share of instructions followed 85.5%
  19. 19 Kimi K2 Jul 2025, strict, share of instructions followed 85%
  20. 20 Claude Sonnet 4.5 strict, share of instructions followed 85%
  21. 21 granite-4.0-micro strict, share of instructions followed 84.9%
  22. 22 GPT-4.1 nano strict, share of instructions followed 84.3%
  23. 23 Gemini 2.0 Flash Feb 2025, strict, share of instructions followed 84.1%
  24. 24 Claude Sonnet 4 strict, share of instructions followed, thinking 10k 84%
  25. 25 Gemini 2.5 Pro Jun 2025, strict, share of instructions followed 84%
  26. 26 GPT-4.1 strict, share of instructions followed 83.8%
  27. 27 Gemini 1.5 Pro Sep 2024, strict, share of instructions followed 83.7%
  28. 28 gpt-oss-120b strict, share of instructions followed 83.6%
  29. 29 Qwen3-235B-A22B-Instruct Jul 2025, strict, share of instructions followed 83.5%
  30. 30 Claude 3.7 Sonnet strict, share of instructions followed 83.4%
  31. 31 DeepSeek-V3 Mar 2025, strict, share of instructions followed 83.2%
  32. 32 Gemini 1.5 Flash Sep 2024, strict, share of instructions followed 83.1%
  33. 33 Gemini 2.0 Flash-Lite strict, share of instructions followed 82.4%
  34. 34 Palmyra X5 strict, share of instructions followed 82.3%
  35. 35 Llama 3.1-70B strict, share of instructions followed 82.1%
  36. 36 Llama 4 Scout strict, share of instructions followed 81.8%
  37. 37 GPT-4o Nov 2024, strict, share of instructions followed 81.7%
  38. 38 Qwen3-235B-A22B strict, share of instructions followed 81.6%
  39. 39 Amazon Nova Pro strict, share of instructions followed 81.5%
  40. 40 GLM-4.5-Air strict, share of instructions followed 81.2%
  41. 41 Llama 3.1-405B strict, share of instructions followed 81.1%
  42. 42 Gemini 2.5 Flash-Lite Sep 2025, strict, share of instructions followed 81%
  43. 43 Qwen3-Next-80B-A3B strict, share of instructions followed 81%
  44. 44 Qwen2.5-72B strict, share of instructions followed 80.6%
  45. 45 Amazon Nova Premier strict, share of instructions followed 80.3%
  46. 46 Claude Haiku 4.5 strict, share of instructions followed 80.1%
  47. 47 Claude 3.5 Haiku strict, share of instructions followed 79.2%
  48. 48 DeepSeek-R1 May 2025, strict, share of instructions followed 78.4%
  49. 49 GPT-4o mini strict, share of instructions followed 78.2%
  50. 50 OLMo 2 32B Instruct strict, share of instructions followed 78%
  51. 51 Amazon Nova Lite strict, share of instructions followed 77.6%
  52. 52 Amazon Nova Micro strict, share of instructions followed 76%
  53. 53 Mistral Small 3.1 strict, share of instructions followed 75%
  54. 54 Llama 3.1-8B strict, share of instructions followed 74.3%
  55. 55 Qwen2.5-7B strict, share of instructions followed 74.1%
  56. 56 gpt-oss-20b strict, share of instructions followed 73.2%
  57. 57 Granite 3.3 8B Instruct strict, share of instructions followed 72.9%
  58. 58 Mixtral 8x22B strict, share of instructions followed 72.4%
  59. 59 Mixtral 8x7B strict, share of instructions followed 57.5%
  60. 60 Mistral 7B v0.3 strict, share of instructions followed 56.8%

Source: Stanford HELM’s leaderboard

176 models. Scroll sideways for the rest.

  1. 98.9 Gemma 4 31B IT
  2. 98.5 Gemma 4 26B A4B
  3. 97.4 DiffusionGemma 26B A4B
  4. 96.8 LFM2.5-24B-A2B
  5. 96.7 Gemma 4 E4B
  6. 96.7 LFM3-1.7B
  7. 95.1 Grok-3 mini
  8. 95.0 Qwen3.5-27B
  9. 94.9 Grok 4
  10. 94.6 Qwen3.7-Plus
  11. 94.6 Gemma 4 E2B
  12. 94.3 Qwen 3.6 Plus
  13. 94.3 Qwen3.7-Max
  14. 93.8 Olmo 3.1 32B Think
  15. 93.6 Spark-X2.6-4B
  16. 93.5 GPT-5.1
  17. 93.4 Qwen3.5-122B-A10B
  18. 93.3 ERNIE 5.0
  19. 93.2 Claude 3.7 Sonnet
  20. 93.2 GPT-5 nano
  21. 92.8 o4-mini
  22. 92.7 GPT-5 mini
  23. 92.6 Qwen3.5 397B-A17B
  24. 92.4 K-EXAONE 2.0
  25. 92.2 Keye-VL-2.0-30B-A3B
  26. 92.1 Llama 3.3 70B Instruct
  27. 91.9 Qwen3.5-35B-A3B
  28. 91.8 LFM2.5-8B-A1B
  29. 91.8 Claude Opus 4
  30. 91.5 Amazon Nova Premier
  31. 91.5 Qwen3.5-9B
  32. 90.9 Cohere Command A
  33. 90.8 Llama 4 Maverick
  34. 90.4 Gemma 3 27B
  35. 90.4 GPT-4.1 mini
  36. 90.3 nvidia-nemotron-nano-9b-v2
  37. 90.2 Claude 3.5 Sonnet (October 2024)
  38. 90.2 Gemma 3 4B
  39. 90.0 LongCat-2.0
  40. 89.8 Kimi K2
  41. 89.8 Gemini 2.5 Flash
  42. 89.8 Qwen3.5-4B
  43. 89.7 K-EXAONE
  44. 89.7 Qwen3.5-Omni-Plus
  45. 89.7 TML-Interaction-Small
  46. 89.7 LongCat-Flash-Chat
  47. 89.7 Granite 4.1 30B
  48. 89.6 EXAONE 4.5 33B
  49. 89.5 Llama-3.1-Nemotron-Ultra-253B-v1
  50. 89.4 Mistral Medium 3
  51. 89.4 Falcon-H1-34B
  52. 89.2 Llama-3.3-Nemotron-Super-49B-v1
  53. 89.1 MiniMax-Text-01
  54. 89.1 Apple Foundation Model Server (2025)
  55. 89.0 Olmo 3 32B Think
  56. 89.0 Granite 4.0 H Small
  57. 88.9 Gemma 3 12B
  58. 88.9 Qwen3-Next-80B-A3B
  59. 88.6 Llama-3.3-Nemotron-Super-49B-v1.5
  60. 88.6 Llama 3.1-405B
  61. 88.6 Hunyuan-TurboS
  62. 88.4 Grok 3
  63. 88.4 LFM2.5-1.2B-Thinking
  64. 88.2 Qwen3-VL-235B-A22B
  65. 88.0 ERNIE-4.5-300B-A47B
  66. 88.0 Solar Open 100B
  67. 87.9 Nemotron 3 Nano 4B
  68. 87.8 Solar Pro 2
  69. 87.8 Qwen3-235B-A22B-Thinking
  70. 87.7 Mistral Large 2
  71. 87.7 Gemini 3 Pro
  72. 87.5 Llama 3.1-70B
  73. 87.5 GPT-5
  74. 87.4 Seed-Thinking-v1.5
  75. 87.4 GPT-4.1
  76. 87.3 Palmyra X4
  77. 87.2 Kimi k1.5
  78. 87.0 Sarvam-M
  79. 86.9 LongCat-Flash-Thinking
  80. 86.9 o3
  81. 86.8 sonar
  82. 86.7 Intern-S1
  83. 86.5 Qwen3-30B-A3B
  84. 86.3 Qwen2.5-VL-72B
  85. 86.3 GLM-4.5-Air
  86. 86.2 LFM2.5-1.2B-Instruct
  87. 86.1 DeepSeek-V3
  88. 86.1 GLM-4.5
  89. 86.0 Tulu 3 405B
  90. 86.0 Baichuan-M2-32B
  91. 85.9 Claude 3.5 Haiku
  92. 85.8 seed-oss-36b-instruct
  93. 85.6 OLMo 2 32B Instruct
  94. 85.4 Qwen3-14B
  95. 85.2 Ring-1T
  96. 85.0 Qwen3-32B
  97. 85.0 Qwen3-8B
  98. 85.0 Claude Sonnet 4.5
  99. 84.9 Phi-4-Reasoning-plus
  100. 84.9 granite-4.0-micro
  101. 84.8 EXAONE 4.0 32B
  102. 84.8 Sarvam 105B
  103. 84.7 Hunyuan-A13B-Instruct
  104. 84.5 GLM-Z1-32B-0414
  105. 84.3 GPT-4.1 nano
  106. 84.3 Qwen2.5-14B-Instruct-1M
  107. 84.3 ERNIE-4.5-21B-A3B-Thinking
  108. 84.1 Qwen2.5-72B
  109. 84.1 Gemini 2.0 Flash
  110. 84.0 Claude Sonnet 4
  111. 84.0 Gemini 2.5 Pro
  112. 83.7 InternVL3.5-241B-A28B
  113. 83.7 Gemini 1.5 Pro
  114. 83.6 gpt-oss-120b
  115. 83.5 Qwen3-235B-A22B-Instruct
  116. 83.4 Qwen3-235B-A22B
  117. 83.4 Phi-4-reasoning
  118. 83.3 DeepSeek-R1
  119. 83.2 DeepSeek-V3
  120. 83.1 Gemini 1.5 Flash
  121. 82.9 Mistral Small 3
  122. 82.4 Gemini 2.0 Flash-Lite
  123. 82.4 LongCat-Flash-Omni
  124. 82.3 LFM2.5-VL-3B
  125. 82.3 Palmyra X5
  126. 82.2 STEP3-VL-10B
  127. 81.9 Qwen3-4B
  128. 81.8 Llama 4 Scout
  129. 81.7 GPT-4o
  130. 81.5 Amazon Nova Pro
  131. 81.5 INTELLECT-2
  132. 81.5 Hermes 4 405B
  133. 81.0 Gemini 2.5 Flash-Lite
  134. 81.0 Qwen2.5-14B
  135. 81.0 Qwen3-Omni-30B-A3B
  136. 80.4 Llama 3.1-8B
  137. 80.2 Gemma 3 1B
  138. 80.1 Claude Haiku 4.5
  139. 79.5 Qwen2.5-32B
  140. 79.3 InternLM3-8B-Instruct
  141. 78.6 Qwen3.5-2B
  142. 78.5 Aya Vision 32B
  143. 78.4 DeepSeek-R1
  144. 78.2 Nemotron Nano 12B v2 VL
  145. 78.2 GPT-4o mini
  146. 77.9 Hermes 4.3 36B
  147. 77.6 Qwen2-72B
  148. 77.6 Amazon Nova Lite
  149. 77.6 LFM2-8B-A1B
  150. 77.4 granite-4.0-1b
  151. 76.3 Qwen-Turbo
  152. 76.0 Amazon Nova Micro
  153. 75.2 Apertus 70B
  154. 75.0 Mistral Small 3.1
  155. 74.9 North Micro Vision Instruct
  156. 74.9 LFM2-1.2B
  157. 74.8 Granite 3.3 8B Instruct
  158. 74.3 Granite 3.2 8B Instruct
  159. 74.1 Qwen2.5-7B
  160. 73.2 gpt-oss-20b
  161. 72.5 Qwen3-1.7B
  162. 72.4 Mixtral 8x22B
  163. 70.1 Phi-4 Mini
  164. 70.1 OLMo 2 1B Instruct
  165. 67.7 GLM-4
  166. 67.0 AFM-4.5B
  167. 63.8 DeepSeek-V2 (MoE-236B, May 2024)
  168. 63.0 Phi-4
  169. 61.0 MiMo-7B-RL
  170. 57.6 Falcon-Arabic-7B
  171. 57.5 Mixtral 8x7B
  172. 56.8 Mistral 7B v0.3
  173. 55.5 DeepSeek LLM 67B
  174. 53.5 BitNet b1.58 2B4T
  175. 51.2 Gemma 3 270M
  176. 44.0 Qwen3.5-0.8B
  1. 1 Gemma 4 31B IT 98.9%
  2. 2 Gemma 4 26B A4B 98.5%
  3. 3 DiffusionGemma 26B A4B 97.4%
  4. 4 LFM2.5-24B-A2B 96.8%
  5. 5 Gemma 4 E4B 96.7%
  6. 6 LFM3-1.7B 96.7%
  7. 7 Grok-3 mini strict, share of instructions followed 95.1%
  8. 8 Qwen3.5-27B 95%
  9. 9 Grok 4 strict, share of instructions followed 94.9%
  10. 10 Qwen3.7-Plus 94.6%
  11. 11 Gemma 4 E2B 94.6%
  12. 12 Qwen 3.6 Plus 94.3%
  13. 13 Qwen3.7-Max 94.3%
  14. 14 Olmo 3.1 32B Think 93.8%
  15. 15 Spark-X2.6-4B 93.6%
  16. 16 GPT-5.1 strict, share of instructions followed 93.5%
  17. 17 Qwen3.5-122B-A10B 93.4%
  18. 18 ERNIE 5.0 93.3%
  19. 19 Claude 3.7 Sonnet 93.2%
  20. 20 GPT-5 nano strict, share of instructions followed 93.2%
  21. 21 o4-mini strict, share of instructions followed 92.8%
  22. 22 GPT-5 mini strict, share of instructions followed 92.7%
  23. 23 Qwen3.5 397B-A17B 92.6%
  24. 24 K-EXAONE 2.0 92.4%
  25. 25 Keye-VL-2.0-30B-A3B 92.2%
  26. 26 Llama 3.3 70B Instruct 92.1%
  27. 27 Qwen3.5-35B-A3B 91.9%
  28. 28 LFM2.5-8B-A1B 91.8%
  29. 29 Claude Opus 4 strict, share of instructions followed 91.8%
  30. 30 Amazon Nova Premier 91.5%
  31. 31 Qwen3.5-9B 91.5%
  32. 32 Cohere Command A 90.9%
  33. 33 Llama 4 Maverick strict, share of instructions followed 90.8%
  34. 34 Gemma 3 27B 90.4%
  35. 35 GPT-4.1 mini strict, share of instructions followed 90.4%
  36. 36 nvidia-nemotron-nano-9b-v2 90.3%
  37. 37 Claude 3.5 Sonnet (October 2024) 90.2%
  38. 38 Gemma 3 4B 90.2%
  39. 39 LongCat-2.0 90%
  40. 40 Kimi K2 Jul 2025 89.8%
  41. 41 Gemini 2.5 Flash Sep 2025, strict, share of instructions followed 89.8%
  42. 42 Qwen3.5-4B 89.8%
  43. 43 K-EXAONE 89.7%
  44. 44 Qwen3.5-Omni-Plus 89.7%
  45. 45 TML-Interaction-Small 89.7%
  46. 46 LongCat-Flash-Chat 89.7%
  47. 47 Granite 4.1 30B 89.7%
  48. 48 EXAONE 4.5 33B 89.6%
  49. 49 Llama-3.1-Nemotron-Ultra-253B-v1 89.5%
  50. 50 Mistral Medium 3 89.4%
  51. 51 Falcon-H1-34B 89.4%
  52. 52 Llama-3.3-Nemotron-Super-49B-v1 89.2%
  53. 53 MiniMax-Text-01 89.1%
  54. 54 Apple Foundation Model Server (2025) 89.1%
  55. 55 Olmo 3 32B Think 89%
  56. 56 Granite 4.0 H Small strict, share of instructions followed 89%
  57. 57 Gemma 3 12B 88.9%
  58. 58 Qwen3-Next-80B-A3B 88.9%
  59. 59 Llama-3.3-Nemotron-Super-49B-v1.5 88.6%
  60. 60 Llama 3.1-405B 88.6%
  61. 61 Hunyuan-TurboS 88.6%
  62. 62 Grok 3 strict, share of instructions followed 88.4%
  63. 63 LFM2.5-1.2B-Thinking 88.4%
  64. 64 Qwen3-VL-235B-A22B 88.2%
  65. 65 ERNIE-4.5-300B-A47B 88%
  66. 66 Solar Open 100B 88%
  67. 67 Nemotron 3 Nano 4B 87.9%
  68. 68 Solar Pro 2 87.8%
  69. 69 Qwen3-235B-A22B-Thinking Jul 2025 87.8%
  70. 70 Mistral Large 2 Nov 2024, strict, share of instructions followed 87.7%
  71. 71 Gemini 3 Pro strict, share of instructions followed 87.7%
  72. 72 Llama 3.1-70B 87.5%
  73. 73 GPT-5 strict, share of instructions followed 87.5%
  74. 74 Seed-Thinking-v1.5 87.4%
  75. 75 GPT-4.1 87.4%
  76. 76 Palmyra X4 strict, share of instructions followed 87.3%
  77. 77 Kimi k1.5 87.2%
  78. 78 Sarvam-M 87%
  79. 79 LongCat-Flash-Thinking 86.9%
  80. 80 o3 strict, share of instructions followed 86.9%
  81. 81 sonar 86.8%
  82. 82 Intern-S1 86.7%
  83. 83 Qwen3-30B-A3B 86.5%
  84. 84 Qwen2.5-VL-72B 86.3%
  85. 85 GLM-4.5-Air 86.3%
  86. 86 LFM2.5-1.2B-Instruct 86.2%
  87. 87 DeepSeek-V3 86.1%
  88. 88 GLM-4.5 86.1%
  89. 89 Tulu 3 405B 86%
  90. 90 Baichuan-M2-32B 86%
  91. 91 Claude 3.5 Haiku 85.9%
  92. 92 seed-oss-36b-instruct 85.8%
  93. 93 OLMo 2 32B Instruct 85.6%
  94. 94 Qwen3-14B 85.4%
  95. 95 Ring-1T 85.2%
  96. 96 Qwen3-32B 85%
  97. 97 Qwen3-8B 85%
  98. 98 Claude Sonnet 4.5 strict, share of instructions followed 85%
  99. 99 Phi-4-Reasoning-plus 84.9%
  100. 100 granite-4.0-micro strict, share of instructions followed 84.9%
  101. 101 EXAONE 4.0 32B 84.8%
  102. 102 Sarvam 105B 84.8%
  103. 103 Hunyuan-A13B-Instruct 84.7%
  104. 104 GLM-Z1-32B-0414 84.5%
  105. 105 GPT-4.1 nano strict, share of instructions followed 84.3%
  106. 106 Qwen2.5-14B-Instruct-1M 84.3%
  107. 107 ERNIE-4.5-21B-A3B-Thinking 84.3%
  108. 108 Qwen2.5-72B 84.1%
  109. 109 Gemini 2.0 Flash Feb 2025, strict, share of instructions followed 84.1%
  110. 110 Claude Sonnet 4 strict, share of instructions followed, thinking 10k 84%
  111. 111 Gemini 2.5 Pro Jun 2025, strict, share of instructions followed 84%
  112. 112 InternVL3.5-241B-A28B 83.7%
  113. 113 Gemini 1.5 Pro Sep 2024, strict, share of instructions followed 83.7%
  114. 114 gpt-oss-120b strict, share of instructions followed 83.6%
  115. 115 Qwen3-235B-A22B-Instruct Jul 2025, strict, share of instructions followed 83.5%
  116. 116 Qwen3-235B-A22B 83.4%
  117. 117 Phi-4-reasoning 83.4%
  118. 118 DeepSeek-R1 83.3%
  119. 119 DeepSeek-V3 Mar 2025, strict, share of instructions followed 83.2%
  120. 120 Gemini 1.5 Flash Sep 2024, strict, share of instructions followed 83.1%
  121. 121 Mistral Small 3 82.9%
  122. 122 Gemini 2.0 Flash-Lite strict, share of instructions followed 82.4%
  123. 123 LongCat-Flash-Omni 82.4%
  124. 124 LFM2.5-VL-3B 82.3%
  125. 125 Palmyra X5 strict, share of instructions followed 82.3%
  126. 126 STEP3-VL-10B 82.2%
  127. 127 Qwen3-4B 81.9%
  128. 128 Llama 4 Scout strict, share of instructions followed 81.8%
  129. 129 GPT-4o Nov 2024, strict, share of instructions followed 81.7%
  130. 130 Amazon Nova Pro strict, share of instructions followed 81.5%
  131. 131 INTELLECT-2 81.5%
  132. 132 Hermes 4 405B 81.5%
  133. 133 Gemini 2.5 Flash-Lite Sep 2025, strict, share of instructions followed 81%
  134. 134 Qwen2.5-14B 81%
  135. 135 Qwen3-Omni-30B-A3B 81%
  136. 136 Llama 3.1-8B 80.4%
  137. 137 Gemma 3 1B 80.2%
  138. 138 Claude Haiku 4.5 strict, share of instructions followed 80.1%
  139. 139 Qwen2.5-32B 79.5%
  140. 140 InternLM3-8B-Instruct 79.3%
  141. 141 Qwen3.5-2B 78.6%
  142. 142 Aya Vision 32B 78.5%
  143. 143 DeepSeek-R1 May 2025, strict, share of instructions followed 78.4%
  144. 144 Nemotron Nano 12B v2 VL 78.2%
  145. 145 GPT-4o mini strict, share of instructions followed 78.2%
  146. 146 Hermes 4.3 36B 77.9%
  147. 147 Qwen2-72B 77.6%
  148. 148 Amazon Nova Lite strict, share of instructions followed 77.6%
  149. 149 LFM2-8B-A1B 77.6%
  150. 150 granite-4.0-1b 77.4%
  151. 151 Qwen-Turbo 76.3%
  152. 152 Amazon Nova Micro strict, share of instructions followed 76%
  153. 153 Apertus 70B 75.2%
  154. 154 Mistral Small 3.1 strict, share of instructions followed 75%
  155. 155 North Micro Vision Instruct 74.9%
  156. 156 LFM2-1.2B 74.9%
  157. 157 Granite 3.3 8B Instruct 74.8%
  158. 158 Granite 3.2 8B Instruct 74.3%
  159. 159 Qwen2.5-7B strict, share of instructions followed 74.1%
  160. 160 gpt-oss-20b strict, share of instructions followed 73.2%
  161. 161 Qwen3-1.7B 72.5%
  162. 162 Mixtral 8x22B strict, share of instructions followed 72.4%
  163. 163 Phi-4 Mini 70.1%
  164. 164 OLMo 2 1B Instruct 70.1%
  165. 165 GLM-4 67.7%
  166. 166 AFM-4.5B 67%
  167. 167 DeepSeek-V2 (MoE-236B, May 2024) 63.8%
  168. 168 Phi-4 63%
  169. 169 MiMo-7B-RL 61%
  170. 170 Falcon-Arabic-7B 57.6%
  171. 171 Mixtral 8x7B strict, share of instructions followed 57.5%
  172. 172 Mistral 7B v0.3 strict, share of instructions followed 56.8%
  173. 173 DeepSeek LLM 67B 55.5%
  174. 174 BitNet b1.58 2B4T 53.5%
  175. 175 Gemma 3 270M 51.2%
  176. 176 Qwen3.5-0.8B 44%

Sources: Labs’ model cards and Stanford HELM’s leaderboard

#ModelScoreSettingDateSource
1 Gemma 4 31B IT 98.9% Jul 2026 Lab model card
2 Gemma 4 26B A4B 98.5% Jul 2026 Lab model card
3 DiffusionGemma 26B A4B 97.4% Jul 2026 Lab model card
4 LFM2.5-24B-A2B 96.8% Oct 2026 Lab model card
5 Gemma 4 E4B 96.7% Jul 2026 Lab model card
6 LFM3-1.7B 96.7% Oct 2026 Lab model card
7 Qwen3.5-27B 95% Feb 2026 Lab model card
8 Qwen3.7-Plus 94.6% Jun 2026 Lab model card
9 Gemma 4 E2B 94.6% Jul 2026 Lab model card
10 Qwen 3.6 Plus 94.3% Apr 2026 Lab model card
11 Qwen3.7-Max 94.3% May 2026 Lab model card
12 Olmo 3.1 32B Think 93.8% Dec 2025 Lab model card
13 Spark-X2.6-4B 93.6% Oct 2026 Lab model card
14 Qwen3.5-122B-A10B 93.4% Feb 2026 Lab model card
15 ERNIE 5.0 93.3% Feb 2026 Lab model card
16 Claude 3.7 Sonnet 93.2% Feb 2025 Lab model card
17 Qwen3.5 397B-A17B 92.6% Feb 2026 Lab model card
18 K-EXAONE 2.0 92.4% Jul 2026 Lab model card
19 Keye-VL-2.0-30B-A3B 92.2% May 2026 Lab model card
20 Llama 3.3 70B Instruct 92.1% Dec 2024 Lab model card
21 Qwen3.5-35B-A3B 91.9% Feb 2026 Lab model card
22 LFM2.5-8B-A1B 91.8% May 2026 Lab model card
23 Amazon Nova Premier 91.5% Apr 2025 Lab model card
24 Qwen3.5-9B 91.5% Mar 2026 Lab model card
25 Cohere Command A 90.9% Apr 2025 Lab model card
26 Gemma 3 27B 90.4% Mar 2025 Lab model card
27 nvidia-nemotron-nano-9b-v2 90.3% Aug 2025 Lab model card
28 Claude 3.5 Sonnet (October 2024) 90.2% Oct 2024 Lab model card
29 Gemma 3 4B 90.2% Mar 2025 Lab model card
30 LongCat-2.0 90% Jul 2026 Lab model card
31 Kimi K2 Jul 2025 89.8% Jul 2025 Lab model card
32 Qwen3.5-4B 89.8% Mar 2026 Lab model card
33 K-EXAONE 89.7% Dec 2025 Lab model card
34 Qwen3.5-Omni-Plus 89.7% Mar 2026 Lab model card
35 TML-Interaction-Small 89.7% May 2026 Lab model card
36 LongCat-Flash-Chat 89.7% Aug 2025 Lab model card
37 Granite 4.1 30B 89.7% Apr 2026 Lab model card
38 EXAONE 4.5 33B 89.6% Apr 2026 Lab model card
39 Llama-3.1-Nemotron-Ultra-253B-v1 89.5% Apr 2025 Lab model card
40 Mistral Medium 3 89.4% May 2025 Lab model card
41 Falcon-H1-34B 89.4% May 2025 Lab model card
42 Llama-3.3-Nemotron-Super-49B-v1 89.2% Mar 2025 Lab model card
43 MiniMax-Text-01 89.1% Jan 2025 Lab model card
44 Apple Foundation Model Server (2025) 89.1% Jul 2025 Lab model card
45 Olmo 3 32B Think 89% Nov 2025 Lab model card
46 Gemma 3 12B 88.9% Mar 2025 Lab model card
47 Qwen3-Next-80B-A3B 88.9% Sep 2025 Lab model card
48 Llama-3.3-Nemotron-Super-49B-v1.5 88.6% Jul 2025 Lab model card
49 Llama 3.1-405B 88.6% Jul 2024 Lab model card
50 Hunyuan-TurboS 88.6% Feb 2025 Lab model card
51 LFM2.5-1.2B-Thinking 88.4% Jan 2026 Lab model card
52 Qwen3-VL-235B-A22B 88.2% Sep 2025 Lab model card
53 ERNIE-4.5-300B-A47B 88% Jun 2025 Lab model card
54 Solar Open 100B 88% Dec 2025 Lab model card
55 Nemotron 3 Nano 4B 87.9% Mar 2026 Lab model card
56 Solar Pro 2 87.8% Jul 2025 Lab model card
57 Qwen3-235B-A22B-Thinking Jul 2025 87.8% Jul 2025 Lab model card
58 Granite 4.0 H Small 87.5% Oct 2025 Lab model card
59 Llama 3.1-70B 87.5% Jul 2024 Lab model card
60 Seed-Thinking-v1.5 87.4% Apr 2025 Lab model card
61 GPT-4.1 87.4% Apr 2025 Lab model card
62 Kimi k1.5 87.2% Jan 2025 Lab model card
63 Sarvam-M 87% May 2025 Lab model card
64 LongCat-Flash-Thinking 86.9% Sep 2025 Lab model card
65 sonar 86.8% Feb 2025 Lab model card
66 Intern-S1 86.7% Jul 2025 Lab model card
67 Qwen3-30B-A3B 86.5% May 2025 Lab model card
68 Qwen2.5-VL-72B 86.3% Feb 2025 Lab model card
69 GLM-4.5-Air 86.3% Aug 2025 Lab model card
70 LFM2.5-1.2B-Instruct 86.2% Jan 2026 Lab model card
71 DeepSeek-V3 86.1% Dec 2024 Lab model card
72 GLM-4.5 86.1% Aug 2025 Lab model card
73 Tulu 3 405B 86% Jan 2025 Lab model card
74 Baichuan-M2-32B 86% Sep 2025 Lab model card
75 Claude 3.5 Haiku 85.9% Oct 2024 Lab model card
76 seed-oss-36b-instruct 85.8% Aug 2025 Lab model card
77 OLMo 2 32B Instruct 85.6% Mar 2025 Lab model card
78 Qwen3-14B 85.4% May 2025 Lab model card
79 Ring-1T 85.2% Oct 2025 Lab model card
80 Qwen3-32B 85% May 2025 Lab model card
81 Qwen3-8B 85% May 2025 Lab model card
82 Phi-4-Reasoning-plus 84.9% Apr 2025 Lab model card
83 EXAONE 4.0 32B 84.8% Jul 2025 Lab model card
84 Sarvam 105B 84.8% Mar 2026 Lab model card
85 Hunyuan-A13B-Instruct 84.7% Jun 2025 Lab model card
86 GLM-Z1-32B-0414 84.5% Apr 2025 Lab model card
87 Qwen2.5-14B-Instruct-1M 84.3% Jan 2025 Lab model card
88 ERNIE-4.5-21B-A3B-Thinking 84.3% Sep 2025 Lab model card
89 Qwen2.5-72B 84.1% Sep 2024 Lab model card
90 GPT-4.1 mini 84.1% Apr 2025 Lab model card
91 InternVL3.5-241B-A28B 83.7% Aug 2025 Lab model card
92 Qwen3-235B-A22B 83.4% Apr 2025 Lab model card
93 Phi-4-reasoning 83.4% Apr 2025 Lab model card
94 DeepSeek-R1 83.3% Jan 2025 Lab model card
95 Mistral Small 3 82.9% Jan 2025 Lab model card
96 LongCat-Flash-Omni 82.4% Oct 2025 Lab model card
97 LFM2.5-VL-3B 82.3% Aug 2026 Lab model card
98 STEP3-VL-10B 82.2% Jan 2026 Lab model card
99 Qwen3-4B 81.9% May 2025 Lab model card
100 INTELLECT-2 81.5% May 2025 Lab model card
101 Hermes 4 405B 81.5% Aug 2025 Lab model card
102 Qwen2.5-14B 81% Dec 2024 Lab model card
103 Qwen3-Omni-30B-A3B 81% Sep 2025 Lab model card
104 Llama 3.1-8B 80.4% Jul 2024 Lab model card
105 Gemma 3 1B 80.2% Mar 2025 Lab model card
106 Qwen2.5-32B 79.5% Dec 2024 Lab model card
107 InternLM3-8B-Instruct 79.3% Jan 2025 Lab model card
108 Qwen3.5-2B 78.6% Mar 2026 Lab model card
109 Aya Vision 32B 78.5% May 2025 Lab model card
110 Nemotron Nano 12B v2 VL 78.2% Nov 2025 Lab model card
111 Hermes 4.3 36B 77.9% Dec 2025 Lab model card
112 Qwen2-72B 77.6% Jun 2024 Lab model card
113 LFM2-8B-A1B 77.6% Oct 2025 Lab model card
114 granite-4.0-1b 77.4% Oct 2025 Lab model card
115 Qwen-Turbo 76.3% Dec 2024 Lab model card
116 Apertus 70B 75.2% Sep 2025 Lab model card
117 North Micro Vision Instruct 74.9% Aug 2026 Lab model card
118 LFM2-1.2B 74.9% Jul 2025 Lab model card
119 Granite 3.3 8B Instruct 74.8% Apr 2025 Lab model card
120 GPT-4.1 nano 74.5% Apr 2025 Lab model card
121 Granite 3.2 8B Instruct 74.3% Feb 2025 Lab model card
122 Qwen3-1.7B 72.5% May 2025 Lab model card
123 Qwen2.5-7B 71.2% Dec 2024 Lab model card
124 Phi-4 Mini 70.1% Mar 2025 Lab model card
125 OLMo 2 1B Instruct 70.1% Apr 2025 Lab model card
126 GLM-4 67.7% Jan 2024 Lab model card
127 AFM-4.5B 67% Jul 2025 Lab model card
128 DeepSeek-V2 (MoE-236B, May 2024) 63.8% May 2024 Lab model card
129 Phi-4 63% Dec 2024 Lab model card
130 MiMo-7B-RL 61% Apr 2025 Lab model card
131 Falcon-Arabic-7B 57.6% Nov 2025 Lab model card
132 DeepSeek LLM 67B 55.5% Jan 2024 Lab model card
133 BitNet b1.58 2B4T 53.5% Apr 2025 Lab model card
134 Gemma 3 270M 51.2% Aug 2025 Lab model card
135 Qwen3.5-0.8B 44% Mar 2026 Lab model card
#ModelScoreSettingDateSource
1 Grok-3 mini 95.1% strict, share of instructions followed Apr 2025 Official leaderboard
2 Grok 4 94.9% strict, share of instructions followed Jul 2025 Official leaderboard
3 GPT-5.1 93.5% strict, share of instructions followed Nov 2025 Official leaderboard
4 GPT-5 nano 93.2% strict, share of instructions followed Aug 2025 Official leaderboard
5 o4-mini 92.8% strict, share of instructions followed Apr 2025 Official leaderboard
6 GPT-5 mini 92.7% strict, share of instructions followed Aug 2025 Official leaderboard
7 Claude Opus 4 91.8% strict, share of instructions followed May 2025 Official leaderboard
8 Llama 4 Maverick 90.8% strict, share of instructions followed Apr 2025 Official leaderboard
9 GPT-4.1 mini 90.4% strict, share of instructions followed Apr 2025 Official leaderboard
10 Gemini 2.5 Flash Sep 2025 89.8% strict, share of instructions followed Sep 2025 Official leaderboard
11 Granite 4.0 H Small 89% strict, share of instructions followed Oct 2025 Official leaderboard
12 Grok 3 88.4% strict, share of instructions followed Apr 2025 Official leaderboard
13 Mistral Large 2 Nov 2024 87.7% strict, share of instructions followed Nov 2024 Official leaderboard
14 Gemini 3 Pro 87.7% strict, share of instructions followed Nov 2025 Official leaderboard
15 GPT-5 87.5% strict, share of instructions followed Aug 2025 Official leaderboard
16 Palmyra X4 87.3% strict, share of instructions followed Oct 2024 Official leaderboard
17 o3 86.9% strict, share of instructions followed Apr 2025 Official leaderboard
18 Claude 3.5 Sonnet (October 2024) 85.5% strict, share of instructions followed Oct 2024 Official leaderboard
19 Kimi K2 Jul 2025 85% strict, share of instructions followed Jul 2025 Official leaderboard
20 Claude Sonnet 4.5 85% strict, share of instructions followed Sep 2025 Official leaderboard
21 granite-4.0-micro 84.9% strict, share of instructions followed Oct 2025 Official leaderboard
22 GPT-4.1 nano 84.3% strict, share of instructions followed Apr 2025 Official leaderboard
23 Gemini 2.0 Flash Feb 2025 84.1% strict, share of instructions followed Feb 2025 Official leaderboard
24 Claude Sonnet 4 84% strict, share of instructions followed, thinking 10k May 2025 Official leaderboard
25 Gemini 2.5 Pro Jun 2025 84% strict, share of instructions followed Jun 2025 Official leaderboard
26 GPT-4.1 83.8% strict, share of instructions followed Apr 2025 Official leaderboard
27 Gemini 1.5 Pro Sep 2024 83.7% strict, share of instructions followed Sep 2024 Official leaderboard
28 gpt-oss-120b 83.6% strict, share of instructions followed Aug 2025 Official leaderboard
29 Qwen3-235B-A22B-Instruct Jul 2025 83.5% strict, share of instructions followed Jul 2025 Official leaderboard
30 Claude 3.7 Sonnet 83.4% strict, share of instructions followed Feb 2025 Official leaderboard
31 DeepSeek-V3 Mar 2025 83.2% strict, share of instructions followed Mar 2025 Official leaderboard
32 Gemini 1.5 Flash Sep 2024 83.1% strict, share of instructions followed Aug 2024 Official leaderboard
33 Gemini 2.0 Flash-Lite 82.4% strict, share of instructions followed Feb 2025 Official leaderboard
34 Palmyra X5 82.3% strict, share of instructions followed Apr 2025 Official leaderboard
35 Llama 3.1-70B 82.1% strict, share of instructions followed Jul 2024 Official leaderboard
36 Llama 4 Scout 81.8% strict, share of instructions followed Apr 2025 Official leaderboard
37 GPT-4o Nov 2024 81.7% strict, share of instructions followed Nov 2024 Official leaderboard
38 Qwen3-235B-A22B 81.6% strict, share of instructions followed Apr 2025 Official leaderboard
39 Amazon Nova Pro 81.5% strict, share of instructions followed Dec 2024 Official leaderboard
40 GLM-4.5-Air 81.2% strict, share of instructions followed Jul 2025 Official leaderboard
41 Llama 3.1-405B 81.1% strict, share of instructions followed Jul 2024 Official leaderboard
42 Gemini 2.5 Flash-Lite Sep 2025 81% strict, share of instructions followed Sep 2025 Official leaderboard
43 Qwen3-Next-80B-A3B 81% strict, share of instructions followed Sep 2025 Official leaderboard
44 Qwen2.5-72B 80.6% strict, share of instructions followed Sep 2024 Official leaderboard
45 Amazon Nova Premier 80.3% strict, share of instructions followed Apr 2025 Official leaderboard
46 Claude Haiku 4.5 80.1% strict, share of instructions followed Oct 2025 Official leaderboard
47 Claude 3.5 Haiku 79.2% strict, share of instructions followed Oct 2024 Official leaderboard
48 DeepSeek-R1 May 2025 78.4% strict, share of instructions followed May 2025 Official leaderboard
49 GPT-4o mini 78.2% strict, share of instructions followed Jul 2024 Official leaderboard
50 OLMo 2 32B Instruct 78% strict, share of instructions followed Mar 2025 Official leaderboard
51 Amazon Nova Lite 77.6% strict, share of instructions followed Dec 2024 Official leaderboard
52 Amazon Nova Micro 76% strict, share of instructions followed Dec 2024 Official leaderboard
53 Mistral Small 3.1 75% strict, share of instructions followed Mar 2025 Official leaderboard
54 Llama 3.1-8B 74.3% strict, share of instructions followed Jul 2024 Official leaderboard
55 Qwen2.5-7B 74.1% strict, share of instructions followed Sep 2024 Official leaderboard
56 gpt-oss-20b 73.2% strict, share of instructions followed Aug 2025 Official leaderboard
57 Granite 3.3 8B Instruct 72.9% strict, share of instructions followed Apr 2025 Official leaderboard
58 Mixtral 8x22B 72.4% strict, share of instructions followed Apr 2024 Official leaderboard
59 Mixtral 8x7B 57.5% strict, share of instructions followed Dec 2023 Official leaderboard
60 Mistral 7B v0.3 56.8% strict, share of instructions followed Sep 2023 Official leaderboard
#ModelScoreSettingDateSource
1 Gemma 4 31B IT 98.9% Jul 2026 Lab model card
2 Gemma 4 26B A4B 98.5% Jul 2026 Lab model card
3 DiffusionGemma 26B A4B 97.4% Jul 2026 Lab model card
4 LFM2.5-24B-A2B 96.8% Oct 2026 Lab model card
5 Gemma 4 E4B 96.7% Jul 2026 Lab model card
6 LFM3-1.7B 96.7% Oct 2026 Lab model card
7 Grok-3 mini 95.1% strict, share of instructions followed Apr 2025 Official leaderboard
8 Qwen3.5-27B 95% Feb 2026 Lab model card
9 Grok 4 94.9% strict, share of instructions followed Jul 2025 Official leaderboard
10 Qwen3.7-Plus 94.6% Jun 2026 Lab model card
11 Gemma 4 E2B 94.6% Jul 2026 Lab model card
12 Qwen 3.6 Plus 94.3% Apr 2026 Lab model card
13 Qwen3.7-Max 94.3% May 2026 Lab model card
14 Olmo 3.1 32B Think 93.8% Dec 2025 Lab model card
15 Spark-X2.6-4B 93.6% Oct 2026 Lab model card
16 GPT-5.1 93.5% strict, share of instructions followed Nov 2025 Official leaderboard
17 Qwen3.5-122B-A10B 93.4% Feb 2026 Lab model card
18 ERNIE 5.0 93.3% Feb 2026 Lab model card
19 Claude 3.7 Sonnet 93.2% Feb 2025 Lab model card
20 GPT-5 nano 93.2% strict, share of instructions followed Aug 2025 Official leaderboard
21 o4-mini 92.8% strict, share of instructions followed Apr 2025 Official leaderboard
22 GPT-5 mini 92.7% strict, share of instructions followed Aug 2025 Official leaderboard
23 Qwen3.5 397B-A17B 92.6% Feb 2026 Lab model card
24 K-EXAONE 2.0 92.4% Jul 2026 Lab model card
25 Keye-VL-2.0-30B-A3B 92.2% May 2026 Lab model card
26 Llama 3.3 70B Instruct 92.1% Dec 2024 Lab model card
27 Qwen3.5-35B-A3B 91.9% Feb 2026 Lab model card
28 LFM2.5-8B-A1B 91.8% May 2026 Lab model card
29 Claude Opus 4 91.8% strict, share of instructions followed May 2025 Official leaderboard
30 Amazon Nova Premier 91.5% Apr 2025 Lab model card
31 Qwen3.5-9B 91.5% Mar 2026 Lab model card
32 Cohere Command A 90.9% Apr 2025 Lab model card
33 Llama 4 Maverick 90.8% strict, share of instructions followed Apr 2025 Official leaderboard
34 Gemma 3 27B 90.4% Mar 2025 Lab model card
35 GPT-4.1 mini 90.4% strict, share of instructions followed Apr 2025 Official leaderboard
36 nvidia-nemotron-nano-9b-v2 90.3% Aug 2025 Lab model card
37 Claude 3.5 Sonnet (October 2024) 90.2% Oct 2024 Lab model card
38 Gemma 3 4B 90.2% Mar 2025 Lab model card
39 LongCat-2.0 90% Jul 2026 Lab model card
40 Kimi K2 Jul 2025 89.8% Jul 2025 Lab model card
41 Gemini 2.5 Flash Sep 2025 89.8% strict, share of instructions followed Sep 2025 Official leaderboard
42 Qwen3.5-4B 89.8% Mar 2026 Lab model card
43 K-EXAONE 89.7% Dec 2025 Lab model card
44 Qwen3.5-Omni-Plus 89.7% Mar 2026 Lab model card
45 TML-Interaction-Small 89.7% May 2026 Lab model card
46 LongCat-Flash-Chat 89.7% Aug 2025 Lab model card
47 Granite 4.1 30B 89.7% Apr 2026 Lab model card
48 EXAONE 4.5 33B 89.6% Apr 2026 Lab model card
49 Llama-3.1-Nemotron-Ultra-253B-v1 89.5% Apr 2025 Lab model card
50 Mistral Medium 3 89.4% May 2025 Lab model card
51 Falcon-H1-34B 89.4% May 2025 Lab model card
52 Llama-3.3-Nemotron-Super-49B-v1 89.2% Mar 2025 Lab model card
53 MiniMax-Text-01 89.1% Jan 2025 Lab model card
54 Apple Foundation Model Server (2025) 89.1% Jul 2025 Lab model card
55 Olmo 3 32B Think 89% Nov 2025 Lab model card
56 Granite 4.0 H Small 89% strict, share of instructions followed Oct 2025 Official leaderboard
57 Gemma 3 12B 88.9% Mar 2025 Lab model card
58 Qwen3-Next-80B-A3B 88.9% Sep 2025 Lab model card
59 Llama-3.3-Nemotron-Super-49B-v1.5 88.6% Jul 2025 Lab model card
60 Llama 3.1-405B 88.6% Jul 2024 Lab model card
61 Hunyuan-TurboS 88.6% Feb 2025 Lab model card
62 Grok 3 88.4% strict, share of instructions followed Apr 2025 Official leaderboard
63 LFM2.5-1.2B-Thinking 88.4% Jan 2026 Lab model card
64 Qwen3-VL-235B-A22B 88.2% Sep 2025 Lab model card
65 ERNIE-4.5-300B-A47B 88% Jun 2025 Lab model card
66 Solar Open 100B 88% Dec 2025 Lab model card
67 Nemotron 3 Nano 4B 87.9% Mar 2026 Lab model card
68 Solar Pro 2 87.8% Jul 2025 Lab model card
69 Qwen3-235B-A22B-Thinking Jul 2025 87.8% Jul 2025 Lab model card
70 Mistral Large 2 Nov 2024 87.7% strict, share of instructions followed Nov 2024 Official leaderboard
71 Gemini 3 Pro 87.7% strict, share of instructions followed Nov 2025 Official leaderboard
72 Llama 3.1-70B 87.5% Jul 2024 Lab model card
73 GPT-5 87.5% strict, share of instructions followed Aug 2025 Official leaderboard
74 Seed-Thinking-v1.5 87.4% Apr 2025 Lab model card
75 GPT-4.1 87.4% Apr 2025 Lab model card
76 Palmyra X4 87.3% strict, share of instructions followed Oct 2024 Official leaderboard
77 Kimi k1.5 87.2% Jan 2025 Lab model card
78 Sarvam-M 87% May 2025 Lab model card
79 LongCat-Flash-Thinking 86.9% Sep 2025 Lab model card
80 o3 86.9% strict, share of instructions followed Apr 2025 Official leaderboard
81 sonar 86.8% Feb 2025 Lab model card
82 Intern-S1 86.7% Jul 2025 Lab model card
83 Qwen3-30B-A3B 86.5% May 2025 Lab model card
84 Qwen2.5-VL-72B 86.3% Feb 2025 Lab model card
85 GLM-4.5-Air 86.3% Aug 2025 Lab model card
86 LFM2.5-1.2B-Instruct 86.2% Jan 2026 Lab model card
87 DeepSeek-V3 86.1% Dec 2024 Lab model card
88 GLM-4.5 86.1% Aug 2025 Lab model card
89 Tulu 3 405B 86% Jan 2025 Lab model card
90 Baichuan-M2-32B 86% Sep 2025 Lab model card
91 Claude 3.5 Haiku 85.9% Oct 2024 Lab model card
92 seed-oss-36b-instruct 85.8% Aug 2025 Lab model card
93 OLMo 2 32B Instruct 85.6% Mar 2025 Lab model card
94 Qwen3-14B 85.4% May 2025 Lab model card
95 Ring-1T 85.2% Oct 2025 Lab model card
96 Qwen3-32B 85% May 2025 Lab model card
97 Qwen3-8B 85% May 2025 Lab model card
98 Claude Sonnet 4.5 85% strict, share of instructions followed Sep 2025 Official leaderboard
99 Phi-4-Reasoning-plus 84.9% Apr 2025 Lab model card
100 granite-4.0-micro 84.9% strict, share of instructions followed Oct 2025 Official leaderboard
101 EXAONE 4.0 32B 84.8% Jul 2025 Lab model card
102 Sarvam 105B 84.8% Mar 2026 Lab model card
103 Hunyuan-A13B-Instruct 84.7% Jun 2025 Lab model card
104 GLM-Z1-32B-0414 84.5% Apr 2025 Lab model card
105 GPT-4.1 nano 84.3% strict, share of instructions followed Apr 2025 Official leaderboard
106 Qwen2.5-14B-Instruct-1M 84.3% Jan 2025 Lab model card
107 ERNIE-4.5-21B-A3B-Thinking 84.3% Sep 2025 Lab model card
108 Qwen2.5-72B 84.1% Sep 2024 Lab model card
109 Gemini 2.0 Flash Feb 2025 84.1% strict, share of instructions followed Feb 2025 Official leaderboard
110 Claude Sonnet 4 84% strict, share of instructions followed, thinking 10k May 2025 Official leaderboard
111 Gemini 2.5 Pro Jun 2025 84% strict, share of instructions followed Jun 2025 Official leaderboard
112 InternVL3.5-241B-A28B 83.7% Aug 2025 Lab model card
113 Gemini 1.5 Pro Sep 2024 83.7% strict, share of instructions followed Sep 2024 Official leaderboard
114 gpt-oss-120b 83.6% strict, share of instructions followed Aug 2025 Official leaderboard
115 Qwen3-235B-A22B-Instruct Jul 2025 83.5% strict, share of instructions followed Jul 2025 Official leaderboard
116 Qwen3-235B-A22B 83.4% Apr 2025 Lab model card
117 Phi-4-reasoning 83.4% Apr 2025 Lab model card
118 DeepSeek-R1 83.3% Jan 2025 Lab model card
119 DeepSeek-V3 Mar 2025 83.2% strict, share of instructions followed Mar 2025 Official leaderboard
120 Gemini 1.5 Flash Sep 2024 83.1% strict, share of instructions followed Aug 2024 Official leaderboard
121 Mistral Small 3 82.9% Jan 2025 Lab model card
122 Gemini 2.0 Flash-Lite 82.4% strict, share of instructions followed Feb 2025 Official leaderboard
123 LongCat-Flash-Omni 82.4% Oct 2025 Lab model card
124 LFM2.5-VL-3B 82.3% Aug 2026 Lab model card
125 Palmyra X5 82.3% strict, share of instructions followed Apr 2025 Official leaderboard
126 STEP3-VL-10B 82.2% Jan 2026 Lab model card
127 Qwen3-4B 81.9% May 2025 Lab model card
128 Llama 4 Scout 81.8% strict, share of instructions followed Apr 2025 Official leaderboard
129 GPT-4o Nov 2024 81.7% strict, share of instructions followed Nov 2024 Official leaderboard
130 Amazon Nova Pro 81.5% strict, share of instructions followed Dec 2024 Official leaderboard
131 INTELLECT-2 81.5% May 2025 Lab model card
132 Hermes 4 405B 81.5% Aug 2025 Lab model card
133 Gemini 2.5 Flash-Lite Sep 2025 81% strict, share of instructions followed Sep 2025 Official leaderboard
134 Qwen2.5-14B 81% Dec 2024 Lab model card
135 Qwen3-Omni-30B-A3B 81% Sep 2025 Lab model card
136 Llama 3.1-8B 80.4% Jul 2024 Lab model card
137 Gemma 3 1B 80.2% Mar 2025 Lab model card
138 Claude Haiku 4.5 80.1% strict, share of instructions followed Oct 2025 Official leaderboard
139 Qwen2.5-32B 79.5% Dec 2024 Lab model card
140 InternLM3-8B-Instruct 79.3% Jan 2025 Lab model card
141 Qwen3.5-2B 78.6% Mar 2026 Lab model card
142 Aya Vision 32B 78.5% May 2025 Lab model card
143 DeepSeek-R1 May 2025 78.4% strict, share of instructions followed May 2025 Official leaderboard
144 Nemotron Nano 12B v2 VL 78.2% Nov 2025 Lab model card
145 GPT-4o mini 78.2% strict, share of instructions followed Jul 2024 Official leaderboard
146 Hermes 4.3 36B 77.9% Dec 2025 Lab model card
147 Qwen2-72B 77.6% Jun 2024 Lab model card
148 Amazon Nova Lite 77.6% strict, share of instructions followed Dec 2024 Official leaderboard
149 LFM2-8B-A1B 77.6% Oct 2025 Lab model card
150 granite-4.0-1b 77.4% Oct 2025 Lab model card
151 Qwen-Turbo 76.3% Dec 2024 Lab model card
152 Amazon Nova Micro 76% strict, share of instructions followed Dec 2024 Official leaderboard
153 Apertus 70B 75.2% Sep 2025 Lab model card
154 Mistral Small 3.1 75% strict, share of instructions followed Mar 2025 Official leaderboard
155 North Micro Vision Instruct 74.9% Aug 2026 Lab model card
156 LFM2-1.2B 74.9% Jul 2025 Lab model card
157 Granite 3.3 8B Instruct 74.8% Apr 2025 Lab model card
158 Granite 3.2 8B Instruct 74.3% Feb 2025 Lab model card
159 Qwen2.5-7B 74.1% strict, share of instructions followed Sep 2024 Official leaderboard
160 gpt-oss-20b 73.2% strict, share of instructions followed Aug 2025 Official leaderboard
161 Qwen3-1.7B 72.5% May 2025 Lab model card
162 Mixtral 8x22B 72.4% strict, share of instructions followed Apr 2024 Official leaderboard
163 Phi-4 Mini 70.1% Mar 2025 Lab model card
164 OLMo 2 1B Instruct 70.1% Apr 2025 Lab model card
165 GLM-4 67.7% Jan 2024 Lab model card
166 AFM-4.5B 67% Jul 2025 Lab model card
167 DeepSeek-V2 (MoE-236B, May 2024) 63.8% May 2024 Lab model card
168 Phi-4 63% Dec 2024 Lab model card
169 MiMo-7B-RL 61% Apr 2025 Lab model card
170 Falcon-Arabic-7B 57.6% Nov 2025 Lab model card
171 Mixtral 8x7B 57.5% strict, share of instructions followed Dec 2023 Official leaderboard
172 Mistral 7B v0.3 56.8% strict, share of instructions followed Sep 2023 Official leaderboard
173 DeepSeek LLM 67B 55.5% Jan 2024 Lab model card
174 BitNet b1.58 2B4T 53.5% Apr 2025 Lab model card
175 Gemma 3 270M 51.2% Aug 2025 Lab model card
176 Qwen3.5-0.8B 44% Mar 2026 Lab model card

Stanford HELM’s leaderboard has 66 entries, 60 of them models we track. Switch the scores above to rank them together.

10 models have held number one in labs’ own numbers since its release in November 2023 11 models have held number one across all independent runs since its release in November 2023 12 models have held number one with claims included since its release in November 2023

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for Stanford HELM’s leaderboard and what labs claim in their cards.

Mistral AI OpenAI Meta Google DeepMind Writer xAI DeepSeek Z.ai Qwen Anthropic Ai2
Stanford HELM leaderboard Passed 85% in Oct 2024, by Palmyra X4
Grok-3 mini 95.1%
Lab claims Passed 85% in Jul 2024, by Llama 3.1-405B
Gemma 4 31B IT 98.9%

Sources: Stanford HELM’s leaderboard and labs’ model cards

Meta’s own number for Llama 3.1-405B passed 85% in July 2024

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On Stanford HELM’s leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) Gemma 4 31B IT Grok-3 mini Claude 3.5 Sonnet (October 2024) GPT-4o mini Mixtral 8x22B GLM-4 Mixtral 8x7B 0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) Gemma 4 31B IT Llama 3.3 70B Instruct

Sources: Stanford HELM’s leaderboard and labs’ model cards

6 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

DeepSeek
Z.ai
Qwen
Meta
Anthropic
Microsoft
Shanghai AI Lab
MiniMax
Moonshot AI
Mistral AI
Ai2
Perplexity
IBM
Tencent
Google DeepMind
Cohere
NVIDIA
ByteDance Seed
OpenAI
Xiaomi Corp
Amazon
Prime Intellect
Technology Innovation Institute
Sarvam AI
Baidu
Upstage
Liquid AI
LG AI Research
Arcee AI
Nous Research
Meituan
Thinking Machines
Kuaishou
iFlytek
DateLaunch postScoreAs the post writes it
Oct 27, 2026 LFM3-1.7B 96.7 IFEval
Oct 22, 2026 Spark-X2.6-4B 93.6 IFEval
Oct 13, 2026 LFM2.5-24B-A2B 96.8 IFEval
Aug 12, 2026 LFM2.5-VL-3B 82.3 IFEval
Aug 12, 2026 North Micro Vision Instruct 74.9 IFEval
Jul 31, 2026 K-EXAONE 2.0 92.4 IFEval
Jul 5, 2026 LongCat-2.0 90 IFEval
Jun 1, 2026 Qwen3.7-Plus 94.6 IFEval
May 28, 2026 LFM2.5-8B-A1B 91.8 IFEval
May 25, 2026 Keye-VL-2.0-30B-A3B 92.2 IFEval
May 20, 2026 Qwen3.7-Max 94.3 IFEval
May 11, 2026 TML-Interaction-Small 89.7 IFEval accuracy, text
Apr 29, 2026 Granite 4.1 30B 89.7 IFEval Avg
Apr 9, 2026 EXAONE 4.5 33B 89.6 IFEval
Apr 2, 2026 Qwen 3.6 Plus 94.3 IFEvalstrict prompt
Mar 30, 2026 Qwen3.5-Omni-Plus 89.7 IFEval
Mar 16, 2026 Nemotron 3 Nano 4B 87.9 IFEVAL-Prompt (reasoning on); Instruction 92
Mar 6, 2026 Sarvam 105B 84.8 IF Eval
Mar 2, 2026 Qwen3.5-0.8B 44 IFEval (Thinking)
Mar 2, 2026 Qwen3.5-2B 78.6 IFEval (Thinking)
Mar 2, 2026 Qwen3.5-4B 89.8 IFEval
Mar 2, 2026 Qwen3.5-9B 91.5 IFEval
Feb 24, 2026 Qwen3.5-35B-A3B 91.9 IFEval
Feb 24, 2026 Qwen3.5-27B 95 IFEval
Feb 24, 2026 Qwen3.5-122B-A10B 93.4 IFEval
Feb 16, 2026 Qwen3.5 397B-A17B 92.6 IFEval
Feb 6, 2026 ERNIE 5.0 93.3 IFEval
Jan 20, 2026 LFM2.5-1.2B-Thinking 88.4 IFEval (± 0.35)
Jan 5, 2026 LFM2.5-1.2B-Instruct 86.2 IFEval
Dec 31, 2025 Solar Open 100B 88 IFEval
Dec 31, 2025 K-EXAONE 89.7 IFEval
Dec 12, 2025 Olmo 3.1 32B Think 93.8 IFEval
Dec 3, 2025 Hermes 4.3 36B 77.9 IFEval
Nov 20, 2025 Olmo 3 32B Think 89 IFEval
Oct 31, 2025 LongCat-Flash-Omni 82.4 IFEval(acc), text
Oct 28, 2025 granite-4.0-1b 77.4 IFEval Average
Oct 7, 2025 LFM2-8B-A1B 77.6 IFEval
Oct 2, 2025 Granite 4.0 H Small 87.5 IFEval Average
Sep 23, 2025 Qwen3-VL-235B-A22B 88.2 IFEval
Sep 22, 2025 Qwen3-Omni-30B-A3B 81 IFEval
Sep 21, 2025 LongCat-Flash-Thinking 86.9 IFEval(strict prompt)
Sep 11, 2025 Qwen3-Next-80B-A3B 88.9 IFEval
Sep 9, 2025 ERNIE-4.5-21B-A3B-Thinking 84.3 IFEval (Prompt Strict Accuracy)
Aug 29, 2025 LongCat-Flash-Chat 89.7 IFEval(acc)
Aug 26, 2025 Hermes 4 405B 81.5 IFEval (Loose) (reasoning)
Aug 25, 2025 InternVL3.5-241B-A28B 83.7 IFEval
Aug 20, 2025 seed-oss-36b-instruct 85.8 IFEval
Aug 18, 2025 nvidia-nemotron-nano-9b-v2 90.3 IFEval (Instruction Strict)
Aug 14, 2025 Gemma 3 270M 51.2 IF Eval 0-shot (IT)
Jul 29, 2025 AFM-4.5B 67 IFEval Inst Strict
Jul 25, 2025 Llama-3.3-Nemotron-Super-49B-v1.5 88.6 IFEval (Strict:Instruction)
Jul 25, 2025 Qwen3-235B-A22B-Thinking 87.8 IFEval
Jul 24, 2025 Intern-S1 86.7 IFEval
Jul 15, 2025 EXAONE 4.0 32B 84.8 IFEval 84.8 non-reasoning (83.7 reasoning)
Jul 11, 2025 Kimi K2 89.8 IFEval, Prompt Strict
Jul 10, 2025 LFM2-1.2B 74.9 IFEval
Jul 10, 2025 Solar Pro 2 87.8 IFEval
Jun 30, 2025 ERNIE-4.5-300B-A47B 88 IFEval
Jun 27, 2025 Hunyuan-A13B-Instruct 84.7 IF-Eval
May 23, 2025 Sarvam-M 87 IFEval (0.87)
May 20, 2025 Falcon-H1-34B 89.4 IFEval
May 11, 2025 INTELLECT-2 81.5 IFEval
May 7, 2025 Mistral Medium 3 89.4 IfEval 0-shot
Apr 30, 2025 Phi-4-reasoning 83.4 IFEval Strict
Apr 30, 2025 Amazon Nova Premier 91.5 IFEval
Apr 30, 2025 Phi-4-Reasoning-plus 84.9 IFEval Strict
Apr 29, 2025 OLMo 2 1B Instruct 70.1 IFEval
Apr 29, 2025 MiMo-7B-RL 61 IF-Eval (Prompt Strict)
Apr 29, 2025 Qwen3-235B-A22B 83.4 IFEval strict prompt
Apr 16, 2025 Granite 3.3 8B Instruct 74.8 IFEval
Apr 15, 2025 BitNet b1.58 2B4T 53.5 IFEval
Apr 14, 2025 GPT-4.1 nano 74.5 IFEval
Apr 14, 2025 GPT-4.1 mini 84.1 IFEval
Apr 14, 2025 GLM-Z1-32B-0414 84.5 IFEval
Apr 14, 2025 GPT-4.1 87.4 IFEval
Apr 10, 2025 Seed-Thinking-v1.5 87.4 IFEval
Apr 7, 2025 Llama-3.1-Nemotron-Ultra-253B-v1 89.5 IFEval Strict:Instruction (reasoning on)
Mar 18, 2025 Llama-3.3-Nemotron-Super-49B-v1 89.2 IFEval Strict:Instruction (reasoning off)
Mar 13, 2025 Cohere Command A IFEval (chart only)
Mar 13, 2025 OLMo 2 32B Instruct 85.6 IFEval
Mar 12, 2025 Gemma 3 27B 90.4 IFEval 0-shot (IT)
Mar 6, 2025 QwQ-32B IFEval
Feb 27, 2025 Hunyuan-TurboS 88.6 IF-Eval
Feb 26, 2025 Granite 3.2 8B Instruct 74.3 IFEval
Feb 24, 2025 Claude 3.7 Sonnet 93.2 Instruction-following IFEval (64K extended thinking)
Feb 11, 2025 sonar 86.8 IFEval
Jan 30, 2025 Tulu 3 405B 86 IFEval (prompt loose)
Jan 30, 2025 Mistral Small 3 82.9 ifeval
Jan 20, 2025 Kimi k1.5 87.2 IF-Eval (Prompt Strict), short-CoT
Jan 20, 2025 DeepSeek-R1 83.3 IF-Eval (Prompt Strict)
Jan 15, 2025 MiniMax-Text-01 89.1 IFEval (avg)
Jan 13, 2025 InternLM3-8B-Instruct 79.3 IFEval(Prompt-Strict)
Dec 26, 2024 DeepSeek-V3 86.1 IF-Eval (Prompt Strict)
Dec 12, 2024 Phi-4 63 IFEval
Dec 6, 2024 Llama 3.3 70B Instruct 92.1 IFEval
Oct 22, 2024 Claude 3.5 Sonnet (October 2024) 90.2 IFEval
Sep 19, 2024 Qwen2.5-72B 84.1 IFeval strict-prompt
Jul 23, 2024 Llama 3.1-405B 88.6 IFEval
Jun 7, 2024 Qwen2-72B 77.6 IFEval (Prompt Strict-Acc.)
Jan 16, 2024 GLM-4 67.7 IFEval Prompt级别、英文 (prompt-level, English)
Nov 29, 2023 DeepSeek LLM 67B Instruction Following Evaluation (prompt-level loose)

88 system cards report it, from 31 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Aug 5, 2026 K-EXAONE 2.0 technical report 92.4 IFEval (Table 6, Reasoning mode) Capabilities
Jul 31, 2026 DiffusionGemma 26B A4B technical report 97.4 IFEval, TD, 94.5 no-think (Table 3) Capabilities
Jul 24, 2026 Gemma 4 E4B technical report 96.7 IFEval, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 E2B technical report 94.6 IFEval, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 31B IT technical report 98.9 IFEval, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 26B A4B technical report 98.5 IFEval, thinking (Table 5) Capabilities
Jun 9, 2026 Keye-VL-2.0-30B-A3B technical report 92.2 IFEval (Figure 5, printed value) Capabilities
Apr 17, 2026 Qwen3.5-Omni-Plus technical report 89.7 IFEval (Table 4) Capabilities
Apr 9, 2026 EXAONE 4.5 33B technical report 89.6 IFEval (Table 3, Reasoning mode) Capabilities
Feb 4, 2026 ERNIE 5.0 technical report 93.3 IFEval (Table 2) Capabilities
Jan 14, 2026 STEP3-VL-10B technical report 82.2 IFEval, SeRe (Table 2) Capabilities
Jan 11, 2026 Solar Open 100B technical report 88 IFEval (Table 6) Capabilities
Jan 5, 2026 K-EXAONE technical report 89.7 IFEval, prompt-strict (Table 3, Reasoning mode). Non-reasoning 85.5 (Table 4) Capabilities
Dec 15, 2025 Olmo 3 32B Think technical report 89 IFEval, Final Think 3.0 column (Tables 14 and 24) Capabilities
Dec 15, 2025 Olmo 3.1 32B Think technical report 93.8 IFEval (Tables 1, 14 and 24) Capabilities
Nov 28, 2025 LFM2-8B-A1B technical report 77.6 IFEval, LFM2-8B-A1B (Table 7) Capabilities
Nov 28, 2025 LFM2-1.2B technical report 74.9 IFEval, LFM2-1.2B (Table 6) Capabilities
Nov 26, 2025 Qwen3-VL-235B-A22B technical report 88.2 IFEval (Table 6) Capabilities
Nov 8, 2025 Falcon-Arabic-7B technical report 57.6 IFEval 0-shot, English (Table 8) Capabilities
Nov 6, 2025 Nemotron Nano 12B v2 VL technical report 78.2 IFEval_prompt_strict, reasoning on, SFT stage 4 (Table 6). Instruction strict 84.7 Capabilities
Oct 31, 2025 LongCat-Flash-Omni technical report 82.4 IFEval (acc) (Table 13) Capabilities
Oct 21, 2025 Ring-1T technical report 85.2 IFEval (Table 1) Capabilities
Oct 8, 2025 OLMo 2 1B Instruct technical report 70.1 IFE, IFEval (Table 7, Table 23) Capabilities
Sep 23, 2025 LongCat-Flash-Thinking technical report 86.9 IFEval(strict prompt) (Table 2) Capabilities
Sep 22, 2025 Qwen3-Omni-30B-A3B technical report 81 IFEval, Instruct (Table 4). Thinking 85.1 (Table 5) Capabilities
Sep 17, 2025 Apertus 70B technical report 75.2 IFEval, prompt-level loose (Table 19) Capabilities
Sep 2, 2025 Baichuan-M2-32B technical report 86 IFEval (Table 1) Capabilities
Sep 1, 2025 LongCat-Flash-Chat technical report 89.7 IFEval (acc) (Table 3) Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 83.7 IFEval (Table 2) Capabilities
Aug 25, 2025 Hermes 4 405B technical report 81.5 IFEval (Loose), reasoning mode (Table 3). Non-reasoning 84.9 Capabilities
Aug 21, 2025 Intern-S1 technical report 86.7 IFEval (Table 2) Capabilities
Aug 8, 2025 GLM-4.5 technical report 86.1 IFEval (Prompt Strict) (Table 6) Capabilities
Aug 8, 2025 GLM-4.5-Air technical report 86.3 IFEval (Prompt Strict) (Table 6) Capabilities
Jul 30, 2025 Falcon-H1-34B technical report 89.4 IFEval, instruction and prompt average accuracy (Table 24) Capabilities
Jul 21, 2025 Kimi K2 technical report 89.8 IFEval (Prompt Strict) (Table 3) Capabilities
Jul 17, 2025 Apple Foundation Model Server (2025) technical report 89.1 IFEval (instruct), AFM Server at 16 bits (Table 3). AFM Server Opt at 3.6 bits per weight 90.2 Capabilities
Jul 15, 2025 EXAONE 4.0 32B technical report 83.7 IFEval (Table 3, Reasoning). Non-reasoning 84.8 (Table 4) Capabilities
Jun 29, 2025 ERNIE-4.5-300B-A47B technical report 88 IFEval (Table 5) Capabilities
Jun 26, 2025 Hunyuan-A13B-Instruct technical report 84.7 IF-Eval, slow-thinking (Table 3). Fast-thinking 84.4 (Table 4) Capabilities
May 14, 2025 Qwen3-8B technical report 85 IFEval strict prompt, thinking mode (Table 17). Non-thinking 83.0 (Table 18) Capabilities
May 14, 2025 Qwen3-4B technical report 81.9 IFEval strict prompt, thinking mode (Table 17). Non-thinking 81.2 (Table 18) Capabilities
May 14, 2025 Qwen3-32B technical report 85 IFEval strict prompt, thinking mode (Table 13). Non-thinking 83.2 (Table 14) Capabilities
May 14, 2025 Qwen3-30B-A3B technical report 86.5 IFEval strict prompt, thinking mode (Table 15). Non-thinking 83.7 (Table 16) Capabilities
May 14, 2025 Qwen3-235B-A22B technical report 83.4 IFEval strict prompt, thinking mode (Table 11). Non-thinking 83.2 (Table 12) Capabilities
May 14, 2025 Qwen3-14B technical report 85.4 IFEval strict prompt, thinking mode (Table 15). Non-thinking 84.8 (Table 16) Capabilities
May 14, 2025 Qwen3-1.7B technical report 72.5 IFEval strict prompt, thinking mode (Table 19). Non-thinking 68.2 (Table 20) Capabilities
May 13, 2025 Aya Vision 32B technical report 78.5 IFEval, English only (Table 5) Capabilities
May 12, 2025 MiMo-7B-RL technical report 61 IF-Eval (Prompt Strict), average of 8 runs (Table 4) Capabilities
May 12, 2025 INTELLECT-2 technical report 81.5 IFEval, evalchemy defaults, target length 10,000 tokens (Table 1) Capabilities
May 2, 2025 Llama-3.3-Nemotron-Super-49B-v1 technical report 89.2 IFEval (Strict-Instruction), reasoning on, reasoning off 89.0 (Table 4) Capabilities
May 2, 2025 Llama-3.1-Nemotron-Ultra-253B-v1 technical report 88.9 IFEval (Strict-Instruction), reasoning on, reasoning off 89.5 (Table 5) Capabilities
Apr 30, 2025 Phi-4-reasoning technical report 83.4 IFEval Strict (Table 2) Capabilities
Apr 30, 2025 Phi-4-Reasoning-plus technical report 84.9 IFEval Strict (Table 2) Capabilities
Apr 30, 2025 Amazon Nova Premier technical report 91.5 IFEval, instruction-level loose accuracy, +/-2.4 (Table 1) Capabilities
Apr 16, 2025 BitNet b1.58 2B4T technical report 53.5 IFEval (0-shot; Instruct-Strict) (Table 1, Table 2) Capabilities
Apr 10, 2025 Seed-Thinking-v1.5 technical report 87.4 IFEval (Table 2) Capabilities
Apr 1, 2025 Cohere Command A technical report 90.9 IFEval, mean of prompt-level and instruction-level strict accuracy (Table 3) Capabilities
Mar 25, 2025 Gemma 3 4B technical report 90.2 IFEval, IT, 0-shot (Table 18) Capabilities
Mar 25, 2025 Gemma 3 27B technical report 90.4 IFEval, IT, 0-shot (Table 18) Capabilities
Mar 25, 2025 Gemma 3 1B technical report 80.2 IFEval, IT, 0-shot (Table 18) Capabilities
Mar 25, 2025 Gemma 3 12B technical report 88.9 IFEval, IT, 0-shot (Table 18) Capabilities
Mar 3, 2025 Phi-4 Mini technical report 70.1 IFEval (0-shot) (Table 7) Capabilities
Feb 19, 2025 Qwen2.5-VL-72B technical report 86.3 IFEval (Table 4) Capabilities
Jan 26, 2025 Qwen2.5-14B-Instruct-1M technical report 84.3 IFEval, strict prompt-level accuracy (Table 6) Capabilities
Jan 20, 2025 Kimi k1.5 technical report 87.2 IF-Eval (Prompt Strict), short-CoT (Table 3) Capabilities
Jan 20, 2025 DeepSeek-R1 technical report 83.3 IF-Eval (Prompt Strict), DeepSeek-R1 (Table 8, same as final column of Table 3) Capabilities
Jan 14, 2025 MiniMax-Text-01 technical report 89.1 IFEval (avg) (Table 8) Capabilities
Dec 31, 2024 OLMo 2 32B Instruct technical report 85.6 IFEval, prompt loose (Tables 7 and 16) Capabilities
Dec 26, 2024 DeepSeek-V3 technical report 86.1 IF-Eval (Prompt Strict), DeepSeek-V3 (Table 6) Capabilities
Dec 19, 2024 Qwen2.5-7B technical report 71.2 IFEval, Qwen2.5-7B-Instruct (Table 8) Capabilities
Dec 19, 2024 Qwen2.5-72B technical report 84.1 IFEval, Qwen2.5-72B-Instruct (Table 6) Capabilities
Dec 19, 2024 Qwen2.5-32B technical report 79.5 IFEval, Qwen2.5-32B-Instruct (Table 7) Capabilities
Dec 19, 2024 Qwen2.5-14B technical report 81 IFEval, Qwen2.5-14B-Instruct (Table 7) Capabilities
Dec 19, 2024 Qwen-Turbo technical report 76.3 IFEval, Qwen2.5-Turbo (Table 7) Capabilities
Dec 12, 2024 Phi-4 technical report 63 IFEval (Table 1) Capabilities
Dec 6, 2024 Llama 3.3 70B Instruct model card 92.1 IFEval (Instruction tuned models table) Capabilities
Nov 23, 2024 Llama 3.1-8B technical report 80.4 IFEval, post-trained (Table 2) Capabilities
Nov 23, 2024 Llama 3.1-70B technical report 87.5 IFEval, post-trained (Table 2) Capabilities
Nov 23, 2024 Llama 3.1-405B technical report 88.6 IFEval, post-trained (Table 2) Capabilities
Nov 22, 2024 Tulu 3 405B technical report 86 IFEval (loose prompt), RLVR (Table 4) Capabilities
Oct 22, 2024 Claude 3.5 Sonnet (October 2024) model card 90.2 IFEval (Table 7) Capabilities
Oct 22, 2024 Claude 3.5 Haiku model card 85.9 IFEval (Table 8) Capabilities
Jul 23, 2024 Llama 3.1-8B model card 80.4 IFEval, instruction tuned (Instruction tuned models table) Capabilities
Jul 23, 2024 Llama 3.1-70B model card 87.5 IFEval, instruction tuned (Instruction tuned models table) Capabilities
Jul 23, 2024 Llama 3.1-405B model card 88.6 IFEval, instruction tuned (Instruction tuned models table) Capabilities
Jul 15, 2024 Qwen2-72B technical report 77.6 IFEval strict-prompt, Qwen2-72B-Instruct (Table 6) Capabilities
May 7, 2024 DeepSeek-V2 (MoE-236B, May 2024) technical report 63.8 IFEval 0-shot, DeepSeek-V2 Chat (RL) (Table 3) Capabilities
Jan 5, 2024 DeepSeek LLM 67B technical report 55.5 IFEval prompt-level loose, held-out (Table 9) Capabilities

The lab's number comes out ahead in 15 of 19 pairs that differ

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. Every benchmark.

Model Lab says Board says Gap
Amazon Nova Premier 91.5% IFEval 80.3% Leaderboard, strict, share of instructions followed +11 pts
GPT-4.1 nano 74.5% IFEval 84.3% Leaderboard, strict, share of instructions followed −9.8 pts
Claude 3.7 Sonnet 93.2% Instruction-following IFEval (64K extended thinking) 83.4% Leaderboard, strict, share of instructions followed +9.8 pts
Qwen3-Next-80B-A3B 88.9% IFEval 81% Leaderboard, strict, share of instructions followed +7.9 pts
OLMo 2 32B Instruct 85.6% IFEval 78% Leaderboard, strict, share of instructions followed +7.6 pts
Llama 3.1-405B 88.6% IFEval 81.1% Leaderboard, strict, share of instructions followed +7.5 pts
Claude 3.5 Haiku 85.9% IFEval (Table 8) 79.2% Leaderboard, strict, share of instructions followed +6.7 pts
GPT-4.1 mini 84.1% IFEval 90.4% Leaderboard, strict, share of instructions followed −6.3 pts
Llama 3.1-8B 80.4% IFEval, instruction tuned (Instruction tuned models table) 74.3% Leaderboard, strict, share of instructions followed +6.1 pts
Llama 3.1-70B 87.5% IFEval, instruction tuned (Instruction tuned models table) 82.1% Leaderboard, strict, share of instructions followed +5.4 pts
GLM-4.5-Air 86.3% IFEval (Prompt Strict) (Table 6) 81.2% Leaderboard, strict, share of instructions followed +5.1 pts
Kimi K2 Jul 2025 89.8% IFEval, Prompt Strict 85% Leaderboard, strict, share of instructions followed +4.8 pts
Claude 3.5 Sonnet (October 2024) 90.2% IFEval 85.5% Leaderboard, strict, share of instructions followed +4.7 pts
GPT-4.1 87.4% IFEval 83.8% Leaderboard, strict, share of instructions followed +3.6 pts
Qwen2.5-72B 84.1% IFeval strict-prompt 80.6% Leaderboard, strict, share of instructions followed +3.5 pts
Qwen2.5-7B 71.2% IFEval, Qwen2.5-7B-Instruct (Table 8) 74.1% Leaderboard, strict, share of instructions followed −2.9 pts
Granite 3.3 8B Instruct 74.8% IFEval 72.9% Leaderboard, strict, share of instructions followed +1.9 pts
Qwen3-235B-A22B 83.4% IFEval strict prompt 81.6% Leaderboard, strict, share of instructions followed +1.8 pts
Granite 4.0 H Small 87.5% IFEval Average 89% Leaderboard, strict, share of instructions followed −1.4 pts

Versions

Replaced by IFBench

More in Instructions and chat

All 81 in Instructions and chat
  • IFBench In use 58 new out-of-domain verifiable output constraints. Scored by programmatic constraint checks (strict or loose). Nemotron-Cascade 2 30B-A3B 82.9%
  • MultiChallenge In use Multi-turn conversations testing instruction retention, inference memory, versioned editing and self-coherence. Graded by LLM judge with rubrics. GPT-6.1 Sol 82%
  • Multi-IF In use Models follow verifiable instructions over three-turn conversations in eight languages, extending IFEval. Scored by instruction accuracy at each turn. LFM3-1.7B 89.1%
  • AdvancedIF In use Models respond to over 1,600 prompts with complex, multi-turn, and system-level instructions. A judge model checks each response against expert-written rubrics. MAI-Thinking-1 85%
  • IFStruct In use Models generate JSON or YAML that must follow a requested schema and phrasing constraints, without constrained decoding. A response passes only if every structural constraint holds. Gemma 4 31B IT 95.9%
  • HANDBOOK.md In use Agents do routine company work with mock email, chat, calendar, issue and commerce tools while following a 20 to 124 page handbook. Programmatic criteria check required and forbidden actions, and a trial passes only if all pass. Claude Opus 5.5 52.3%