Benchmark Almanac

IFStruct

Gemma 4 31B IT 95.9% leads since June 2026
Gemma 4 31B IT 95.9% leads since June 2026
In use

Models generate JSON or YAML that must follow a requested schema and phrasing constraints, without constrained decoding. A response passes only if every structural constraint holds.

2,000 frozen test prompts. The makers say it saturates near 100% for frontier models and targets small models.

Made by
Liquid AI
Released
June 2026
Kind
Following precise instructions
Level
Everyday
Scored
Checked automatically
Answers
Open-ended
Tasks
All public
Strongest model tested
Gemma 4 31B IT #100 on the capability index
Last in a launch post
October 2026
Last model added
October 2026
Scores refreshed
November 12th, 2026
15
Entries on its own leaderboard, 10 models we track
2
Launch posts list it, from 1 lab

Gemma 4 31B IT leads with 95.9% among models released by April 2026

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

Gemma 4 31B IT leads with 95.9%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

The newest model with a score here came out in April 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

15 entries. Scroll sideways for the rest.

  1. 95.9 Gemma 4 31B IT
  2. 92.0 gpt-oss-20b
  3. 86.8 Nemotron 3 Nano 30B-A3B
  4. 79.8 Qwen3-8B
  5. 76.7 Gemma 4 E4B
  6. 68.5 Granite 4.1 8B
  7. 64.8 Gemma 4 E2B
  8. 44.9 LFM2.5-350M + IFStruct RL
  9. 38.8 granite-4.0-h-tiny
  10. 36.3 Qwen3.5-4B
  11. 33.1 Qwen3.5-2B
  12. 21.1 LFM2.5-350M
  13. 15.5 Qwen3.5-0.8B
  14. 1.70 granite-4.0-h-350m
  15. 0.50 gemma-3-270m-it
  1. 1 Gemma 4 31B IT 95.9%
  2. 2 gpt-oss-20b 92%
  3. 3 Nemotron 3 Nano 30B-A3B 86.8%
  4. 4 Qwen3-8B 79.8%
  5. 5 Gemma 4 E4B 76.7%
  6. 6 Granite 4.1 8B 68.5%
  7. 7 Gemma 4 E2B 64.8%
  8. 8 LFM2.5-350M + IFStruct RL 44.9%
  9. 9 granite-4.0-h-tiny 38.8%
  10. 10 Qwen3.5-4B 36.3%
  11. 11 Qwen3.5-2B 33.1%
  12. 12 LFM2.5-350M 21.1%
  13. 13 Qwen3.5-0.8B 15.5%
  14. 14 granite-4.0-h-350m 1.70%
  15. 15 gemma-3-270m-it 0.50%

Source: Official IFStruct leaderboard

Sources: Official IFStruct leaderboard and labs’ model cards

#EntryScoreSettingDateSource
1 Gemma 4 31B IT 95.9% Apr 2026 Official leaderboard
2 gpt-oss-20b 92% Aug 2025 Official leaderboard
3 Nemotron 3 Nano 30B-A3B 86.8% Dec 2025 Official leaderboard
4 Qwen3-8B 79.8% Apr 2025 Official leaderboard
5 Gemma 4 E4B 76.7% Mar 2026 Official leaderboard
6 Granite 4.1 8B 68.5% Apr 2026 Official leaderboard
7 Gemma 4 E2B 64.8% Mar 2026 Official leaderboard
8 LFM2.5-350M + IFStruct RL 44.9% Official leaderboard
9 granite-4.0-h-tiny 38.8% Official leaderboard
10 Qwen3.5-4B 36.3% Feb 2026 Official leaderboard
11 Qwen3.5-2B 33.1% Feb 2026 Official leaderboard
12 LFM2.5-350M 21.1% Official leaderboard
13 Qwen3.5-0.8B 15.5% Feb 2026 Official leaderboard
14 granite-4.0-h-350m 1.70% Official leaderboard
15 gemma-3-270m-it 0.50% Official leaderboard
#ModelScoreSettingDateSource
1 Gemma 4 31B IT 95.9% Apr 2026 Official leaderboard
2 LFM3-1.7B 93.7% Oct 2026 Lab model card
3 gpt-oss-20b 92% Aug 2025 Official leaderboard
4 Nemotron 3 Nano 30B-A3B 86.8% Dec 2025 Official leaderboard
5 LFM2.5-2.6B 85.5% Aug 2026 Lab model card
6 Qwen3-8B 79.8% Apr 2025 Official leaderboard
7 Gemma 4 E4B 76.7% Mar 2026 Official leaderboard
8 Granite 4.1 8B 68.5% Apr 2026 Official leaderboard
9 Gemma 4 E2B 64.8% Mar 2026 Official leaderboard
10 Qwen3.5-4B 36.3% Feb 2026 Official leaderboard
11 Qwen3.5-2B 33.1% Feb 2026 Official leaderboard
12 Qwen3.5-0.8B 15.5% Feb 2026 Official leaderboard

Labs report scores for 2 models in their cards. Switch the scores above to rank them together.

Number one over time Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind
Own leaderboard Passed 85% in Jun 2026, by Gemma 4 31B IT
Gemma 4 31B IT 95.9%

Source: Official IFStruct leaderboard

Gpt-oss-20b on the official leaderboard passed 85% in August 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Gemma 4 31B IT gpt-oss-20b Qwen3-8B 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Gemma 4 31B IT Qwen3-8B

Sources: Official IFStruct leaderboard and labs’ model cards

One lab put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Liquid AI
DateLaunch postScoreAs the post writes it
Oct 27, 2026 LFM3-1.7B 93.7 IFStruct
Aug 4, 2026 LFM2.5-2.6B 85.5 IFStruct

More in Instructions and chat

All 81 in Instructions and chat
  • IFEval In use Models answer about 500 prompts containing verifiable instructions such as word limits or required keywords. Code checks each instruction for strict and loose accuracy. Gemma 4 31B IT 98.9%
  • IFBench In use 58 new out-of-domain verifiable output constraints. Scored by programmatic constraint checks (strict or loose). Nemotron-Cascade 2 30B-A3B 82.9%
  • MultiChallenge In use Multi-turn conversations testing instruction retention, inference memory, versioned editing and self-coherence. Graded by LLM judge with rubrics. GPT-6.1 Sol 82%
  • Multi-IF In use Models follow verifiable instructions over three-turn conversations in eight languages, extending IFEval. Scored by instruction accuracy at each turn. LFM3-1.7B 89.1%
  • AdvancedIF In use Models respond to over 1,600 prompts with complex, multi-turn, and system-level instructions. A judge model checks each response against expert-written rubrics. MAI-Thinking-1 85%
  • HANDBOOK.md In use Agents do routine company work with mock email, chat, calendar, issue and commerce tools while following a 20 to 124 page handbook. Programmatic criteria check required and forbidden actions, and a trial passes only if all pass. Claude Opus 5.5 52.3%