ComplexConstraints
The official leaderboard hasn’t scored a model released after Inkling 2 Small (November 2026). Labs report their own numbers for one newer model: Inkling 2 Mini at 33.4%.
Models answer 75 expert-written professional prompts whose constraints depend on each other, fire conditionally or must be inferred. An LLM judge grades 1,559 rubric criteria one at a time, and a task passes only if all criteria pass.
Surge helped build AdvancedIF and made this to test entangled constraints that AdvancedIF lacks. The repo is an Inspect harness. Its README points to a Hugging Face dataset that returned 401 on November 1st, 2026.
- Made by
- Surge AI
- Released
- June 2026
- Kind
- Following precise instructions
- Level
- University
- Scored
- Graded by a model
- Answers
- Open-ended
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Single turn
- Items from
- Written by experts
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
- GPT-6 Astra #4 on the capability index
- Last in a launch post
- November 2026
- Last model added
- November 2026
- Scores refreshed
- November 18th, 2026
In the authors’ words
Evaluation protocols can lag behind LLM capabilities. Programmatically verified benchmarks cover narrow surface constraints, whereas real-world instruction following and agentic workflows require judging semantic, contextual, and policy-dependent behavior. We study expert-curated rubric-based evaluation as a unified mechanism for measurement and reinforcement-learning rewards across two settings: complex instruction following and enterprise agentic tasks. We identify rubric-design choices that affect reward quality, including maximum viable atomicity, intent-aware criterion design, and LLM-judge calibration. We introduce ComplexConstraints, an expert-curated instruction-following suite comprising a public 75-prompt benchmark with 1,559 rubric criteria and a disjoint 1,000-prompt training set, with 10-40 atomic criteria per prompt. Empirically, rubric rewards improve training in both fixed task datasets, such as ComplexConstraints, and stateful RL environments, such as CoreCraft. Training a 4B model on ComplexConstraints improves mean criterion pass rate by +15.5 pp on a held-out split, bringing it within 0.5 pp of the untrained baseline of a roughly 60x larger Qwen3 model, and the gains transfer to external benchmarks the model never saw during training: +8.4 pp on AdvancedIF and +10.1 pp on MultiChallenge. In CoreCraft, rubric-reward RL likewise transfers to out-of-distribution benchmarks (+4.5 pp BFCL, +7.4 pp tau^2-Bench, +6.8 pp Toolathlon). These results show that expert-authored rubrics provide effective evaluation targets and scalable reward signals for improving LLM instruction following and agentic behavior.
GPT-6 Astra leads with 70.8%
Best score per model from the benchmark's own leaderboard, read on November 18th, 2026. Where a model ran with several settings or harnesses, we keep its best.
GPT-6 Astra leads with 70.8%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
- 70.8 GPT-6 Astra
- 70.3
Gemini 4 Argon
- 70.1 GPT-6.1 Sol
- 67.9
Muse Wonder
- 67.2 Claude Sonnet 5.5
- 64.8
Muse Spark 1.3
- 64.4 Claude Opus 5.5
- 63.2
Gemini 3.8 Flash
- 60.8 Grok 4.7
- 60.4 Claude Fable 5.1
- 54.4 GLM-5.3
- 51.1 Kimi K3
- 49.2
Tencent Hy4 preview
- 47.6
Inkling 2 Small
- 46.3
Qwen 3.8 Max
- 42.4
Mistral Large 4
- 1 GPT-6 Astra max 70.8%
- 2
Gemini 4 Argon high 70.3%
- 3 GPT-6.1 Sol max 70.1%
- 4
Muse Wonder max 67.9%
- 5 Claude Sonnet 5.5 adaptive/max 67.2%
- 6
Muse Spark 1.3 max 64.8%
- 7 Claude Opus 5.5 adaptive/max 64.4%
- 8
Gemini 3.8 Flash high 63.2%
- 9 Grok 4.7 xhigh 60.8%
- 10 Claude Fable 5.1 adaptive/max 60.4%
- 11 GLM-5.3 max 54.4%
- 12 Kimi K3 max 51.1%
- 13
Tencent Hy4 preview high 49.2%
- 14
Inkling 2 Small xhigh 47.6%
- 15
Qwen 3.8 Max xhigh 46.3%
- 16
Mistral Large 4 high 42.4%
Source: Official ComplexConstraints leaderboard
- 70.8 GPT-6 Astra
- 70.3
Gemini 4 Argon
- 70.1 GPT-6.1 Sol
- 67.9
Muse Wonder
- 67.2 Claude Sonnet 5.5
- 64.8
Muse Spark 1.3
- 64.4 Claude Opus 5.5
- 63.2
Gemini 3.8 Flash
- 60.8 Grok 4.7
- 60.4 Claude Fable 5.1
- 54.4 GLM-5.3
- 51.1 Kimi K3
- 49.2
Tencent Hy4 preview
- 47.6
Inkling 2 Small
- 46.3
Qwen 3.8 Max
- 42.4
Mistral Large 4
- 33.4
Inkling 2 Mini
- 1 GPT-6 Astra max 70.8%
- 2
Gemini 4 Argon high 70.3%
- 3 GPT-6.1 Sol max 70.1%
- 4
Muse Wonder max 67.9%
- 5 Claude Sonnet 5.5 adaptive/max 67.2%
- 6
Muse Spark 1.3 max 64.8%
- 7 Claude Opus 5.5 adaptive/max 64.4%
- 8
Gemini 3.8 Flash high 63.2%
- 9 Grok 4.7 xhigh 60.8%
- 10 Claude Fable 5.1 adaptive/max 60.4%
- 11 GLM-5.3 max 54.4%
- 12 Kimi K3 max 51.1%
- 13
Tencent Hy4 preview high 49.2%
- 14
Inkling 2 Small xhigh 47.6%
- 15
Qwen 3.8 Max xhigh 46.3%
- 16
Mistral Large 4 high 42.4%
- 17
Inkling 2 Mini 33.4%
Sources: Official ComplexConstraints leaderboard and labs’ model cards
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 70.8% | max | Sep 2026 | Official leaderboard |
| 2 | | 70.3% | high | Sep 2026 | Official leaderboard |
| 3 | GPT-6.1 Sol | 70.1% | max | Sep 2026 | Official leaderboard |
| 4 | | 67.9% | max | Oct 2026 | Official leaderboard |
| 5 | Claude Sonnet 5.5 | 67.2% | adaptive/max | Sep 2026 | Official leaderboard |
| 6 | | 64.8% | max | Sep 2026 | Official leaderboard |
| 7 | Claude Opus 5.5 | 64.4% | adaptive/max | Sep 2026 | Official leaderboard |
| 8 | | 63.2% | high | Sep 2026 | Official leaderboard |
| 9 | Grok 4.7 | 60.8% | xhigh | Sep 2026 | Official leaderboard |
| 10 | Claude Fable 5.1 | 60.4% | adaptive/max | Sep 2026 | Official leaderboard |
| 11 | GLM-5.3 | 54.4% | max | Aug 2026 | Official leaderboard |
| 12 | Kimi K3 | 51.1% | max | Jul 2026 | Official leaderboard |
| 13 | | 49.2% | high | Aug 2026 | Official leaderboard |
| 14 | | 47.6% | xhigh | Nov 2026 | Official leaderboard |
| 15 | | 46.3% | xhigh | Aug 2026 | Official leaderboard |
| 16 | | 42.4% | high | Oct 2026 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 70.8% | max | Sep 2026 | Official leaderboard |
| 2 | 70.3% | high | Sep 2026 | Official leaderboard | |
| 3 | GPT-6.1 Sol | 70.1% | max | Sep 2026 | Official leaderboard |
| 4 | 67.9% | max | Oct 2026 | Official leaderboard | |
| 5 | Claude Sonnet 5.5 | 67.2% | adaptive/max | Sep 2026 | Official leaderboard |
| 6 | 64.8% | max | Sep 2026 | Official leaderboard | |
| 7 | Claude Opus 5.5 | 64.4% | adaptive/max | Sep 2026 | Official leaderboard |
| 8 | 63.2% | high | Sep 2026 | Official leaderboard | |
| 9 | Grok 4.7 | 60.8% | xhigh | Sep 2026 | Official leaderboard |
| 10 | Claude Fable 5.1 | 60.4% | adaptive/max | Sep 2026 | Official leaderboard |
| 11 | GLM-5.3 | 54.4% | max | Aug 2026 | Official leaderboard |
| 12 | Kimi K3 | 51.1% | max | Jul 2026 | Official leaderboard |
| 13 | 49.2% | high | Aug 2026 | Official leaderboard | |
| 14 | 47.6% | xhigh | Nov 2026 | Official leaderboard | |
| 15 | 46.3% | xhigh | Aug 2026 | Official leaderboard | |
| 16 | 42.4% | high | Oct 2026 | Official leaderboard | |
| 17 | 33.4% | Nov 2026 | Lab model card |
Labs report scores for 2 models in their cards. Switch the scores above to rank them together.
5 models have held number one on the official leaderboard since its release in June 2026 5 models have held number one with claims included since its release in June 2026
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Sources: Official ComplexConstraints leaderboard and labs’ model cards
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official ComplexConstraints leaderboard and labs’ model cards
One lab put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Nov 20, 2026 | 33.4 | ComplexConstraints v1.1 (openhands v1.45.0) | |
| Nov 5, 2026 | 44.5 | ComplexConstraints v1.0 |
One system card reports it
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Nov 20, 2026 | | 33.4 | ComplexConstraints v1.1 (openhands v1.45.0) | Capabilities |
One lab claim next to the leaderboard
Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. Every benchmark.
| Model | Lab says | Board says | Gap |
|---|---|---|---|
| | 44.5% ComplexConstraints v1.0 | 47.6% Leaderboard, xhigh | −3.1 pts |
More in Instructions and chat
- IFEval In use Models answer about 500 prompts containing verifiable instructions such as word limits or required keywords. Code checks each instruction for strict and loose accuracy.
Gemma 4 31B IT 98.9%
- IFBench In use 58 new out-of-domain verifiable output constraints. Scored by programmatic constraint checks (strict or loose). Flint 89.2%
- MultiChallenge In use Multi-turn conversations testing instruction retention, inference memory, versioned editing and self-coherence. Graded by LLM judge with rubrics. GPT-6.1 Sol 82%
- Multi-IF In use Models follow verifiable instructions over three-turn conversations in eight languages, extending IFEval. Scored by instruction accuracy at each turn. LFM3-1.7B 89.1%
- AdvancedIF In use Models respond to over 1,600 prompts with complex, multi-turn, and system-level instructions. A judge model checks each response against expert-written rubrics.
MAI-Thinking-1 85%
- MM-IFEval In use Multimodal models follow image-grounded instructions with exact output constraints. Checks mix rule-based verification and a judge model. GPT-4o 64.6%
Data audit
All 12 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 5 pass
- Pass Independent runs Scores from the official leaderboard (16 entries), launch posts (2 scores) and system cards (1 score).
- Pass Models scored 17 models we track.
- Pass Newest score The newest is from November 2026, for Inkling 2 Mini.
- Pass Frontier models 6 of the top 10 have a score, led by GPT-6 Astra (#4).
- Pass Headroom The best is 70.8%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on surgehq.ai: 16 rows for models we track at the last read.
- Pass Last read November 18th, 2026.
- Pass Read worked The last read worked.
Consistency 2 pass
- Pass Lab claims The one pair agrees within 10 points.
- Pass Known issues No published audit found problems.