Benchmark Almanac

PlotPoints Round 4: Willingness and Judgment

Claude Fable 5.1 0.8 leads since September 2026
Not in model cards

Models respond to scripted roleplay scenes. Two classifiers check whether models respect a stated limit and whether they refuse allowed requests.

The maker’s leaderboard ranks 55 models. The score subtracts the over-refusal rate from the share of hard limits respected on the first request.

Released
September 2026
Kind
Deception and alignment
Level
Everyday
Scored
Graded by a model
Answers
Open-ended
Tasks
All public
Measures
Propensity
Language
English
In and out
Text → Text
Interaction
Multi-turn
Run by
Anyone
Strongest model tested
GPT-6 Astra #4 on the capability index
Last model added
November 2026
Scores refreshed
November 20th, 2026
51
Entries on its own leaderboard, 39 models we track

Claude Fable 5.1 leads with 0.8 Claude Fable 5.1 leads with 0.8

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

  1. 1 Claude Fable 5.1 0.8
  2. 2 MiMo 2.6 Flash 0.8
  3. 3 Claude Opus 4.6 0.7
  4. 4 Claude Opus 4.7 0.7
  5. 5 Claude Opus 5.5 0.6
  6. 6 Claude Sonnet 5 0.6
  7. 7 Claude Opus 5 0.5
  8. 8 Claude Opus 4.8 0.5
  9. 9 Tencent Hy4 0.5
  10. 10 GLM-5.1 0.5
  11. 11 Kimi K2.6 0.4
  12. 12 Grok 4.7 0.4
  13. 13 Qwen3.7-Max 0.4
  14. 14 Ember-1 0.4
  15. 15 GLM-5.3-Flash FlashX 0.4
  16. 16 Muse Spark 1.3 0.4
  17. 17 Gemini 3.8 Flash 0.4
  18. 18 Grok 4.3 0.3
  19. 19 MiMo 2.6 Pro 0.3
  20. 20 MiniMax-M3 0.3
  21. 21 Gemini 3.7 Flash 0.3
  22. 22 Gemini 3.5 Flash 0.3
  23. 23 DeepSeek-V3 Mar 2025 0.3
  24. 24 Qwen 3.8 Max prime 0.3
  25. 25 Qwen3.8-Omni-Flash 0.3
  26. 26 GPT-4.1 0.3
  27. 27 Qwen3.6 27B 0.2
  28. 28 MiniMax-M2.7 0.2
  29. 29 DeepSeek V4.1 Flash 0.2
  30. 30 DeepSeek-V4-Flash 0.2
  31. 31 Qwen3.8 Flash 0.2
  32. 32 DeepSeek-V4-Pro 0.2
  33. 33 GLM-5.3 prime 0.2
  34. 34 Claude Sonnet 4.6 0.1
  35. 35 Euryale 70B 0.1
  36. 36 Gemma 4 31B IT 0.0
  37. 37 MiMo 2.5 Pro 0.0
  38. 38 Qwen 3.6 35B-A3B 0.0
  39. 39 Lunaris 8B -0.0
  40. 40 Magnum v4 72B -0.0
  41. 41 GPT-5.5 -0.1
  42. 42 UnslopNemo 12B -0.1
  43. 43 Cydonia 24B -0.1
  44. 44 Skyfall 36B -0.2
  45. 45 Command A+ -0.3
  46. 46 Aion 3.5 -0.3
  47. 47 Hemmingway 1 -0.3
  48. 48 GPT-6 Sol -0.4
  49. 49 GPT-6 Astra -0.4
  50. 50 GPT-6 Luna -0.4
  51. 51 Dolphin 24B Venice -0.5

Source: Official PlotPoints Round 4: Willingness and Judgment leaderboard

Source: Official PlotPoints Round 4: Willingness and Judgment leaderboard

#EntryScoreSettingDateSource
1 Claude Fable 5.1 0.8 Sep 2026 Official leaderboard
2 MiMo 2.6 Flash 0.8 Official leaderboard
3 Claude Opus 4.6 0.7 Feb 2026 Official leaderboard
4 Claude Opus 4.7 0.7 Apr 2026 Official leaderboard
5 Claude Opus 5.5 0.6 Sep 2026 Official leaderboard
6 Claude Sonnet 5 0.6 Jun 2026 Official leaderboard
7 Claude Opus 5 0.5 Jul 2026 Official leaderboard
8 Claude Opus 4.8 0.5 May 2026 Official leaderboard
9 Tencent Hy4 0.5 Nov 2026 Official leaderboard
10 GLM-5.1 0.5 Apr 2026 Official leaderboard
11 Kimi K2.6 0.4 Apr 2026 Official leaderboard
12 Grok 4.7 0.4 Sep 2026 Official leaderboard
13 Qwen3.7-Max 0.4 May 2026 Official leaderboard
14 Ember-1 0.4 Sep 2026 Official leaderboard
15 GLM-5.3-Flash 0.4 FlashX Aug 2026 Official leaderboard
16 Muse Spark 1.3 0.4 Sep 2026 Official leaderboard
17 Gemini 3.8 Flash 0.4 Sep 2026 Official leaderboard
18 Grok 4.3 0.3 Apr 2026 Official leaderboard
19 MiMo 2.6 Pro 0.3 Official leaderboard
20 MiniMax-M3 0.3 Jun 2026 Official leaderboard
21 Gemini 3.7 Flash 0.3 Aug 2026 Official leaderboard
22 Gemini 3.5 Flash 0.3 May 2026 Official leaderboard
23 DeepSeek-V3 Mar 2025 0.3 Mar 2025 Official leaderboard
24 Qwen 3.8 Max 0.3 prime Aug 2026 Official leaderboard
25 Qwen3.8-Omni-Flash 0.3 Sep 2026 Official leaderboard
26 GPT-4.1 0.3 Apr 2025 Official leaderboard
27 Qwen3.6 27B 0.2 Apr 2026 Official leaderboard
28 MiniMax-M2.7 0.2 Mar 2026 Official leaderboard
29 DeepSeek V4.1 Flash 0.2 Sep 2026 Official leaderboard
30 DeepSeek-V4-Flash 0.2 Apr 2026 Official leaderboard
31 Qwen3.8 Flash 0.2 Aug 2026 Official leaderboard
32 DeepSeek-V4-Pro 0.2 Apr 2026 Official leaderboard
33 GLM-5.3 0.2 prime Aug 2026 Official leaderboard
34 Claude Sonnet 4.6 0.1 Feb 2026 Official leaderboard
35 Euryale 70B 0.1 Official leaderboard
36 Gemma 4 31B IT 0.0 Apr 2026 Official leaderboard
37 MiMo 2.5 Pro 0.0 Official leaderboard
38 Qwen 3.6 35B-A3B 0.0 Apr 2026 Official leaderboard
39 Lunaris 8B -0.0 Official leaderboard
40 Magnum v4 72B -0.0 Official leaderboard
41 GPT-5.5 -0.1 Apr 2026 Official leaderboard
42 UnslopNemo 12B -0.1 Official leaderboard
43 Cydonia 24B -0.1 Official leaderboard
44 Skyfall 36B -0.2 Official leaderboard
45 Command A+ -0.3 May 2026 Official leaderboard
46 Aion 3.5 -0.3 Official leaderboard
47 Hemmingway 1 -0.3 Official leaderboard
48 GPT-6 Sol -0.4 Sep 2026 Official leaderboard
49 GPT-6 Astra -0.4 Sep 2026 Official leaderboard
50 GPT-6 Luna -0.4 Sep 2026 Official leaderboard
51 Dolphin 24B Venice -0.5 Official leaderboard
#ModelScoreSettingDateSource
1 Claude Fable 5.1 0.8 Sep 2026 Official leaderboard
2 Claude Opus 4.6 0.7 Feb 2026 Official leaderboard
3 Claude Opus 4.7 0.7 Apr 2026 Official leaderboard
4 Claude Opus 5.5 0.6 Sep 2026 Official leaderboard
5 Claude Sonnet 5 0.6 Jun 2026 Official leaderboard
6 Claude Opus 5 0.5 Jul 2026 Official leaderboard
7 Claude Opus 4.8 0.5 May 2026 Official leaderboard
8 Tencent Hy4 0.5 Nov 2026 Official leaderboard
9 GLM-5.1 0.5 Apr 2026 Official leaderboard
10 Kimi K2.6 0.4 Apr 2026 Official leaderboard
11 Grok 4.7 0.4 Sep 2026 Official leaderboard
12 Qwen3.7-Max 0.4 May 2026 Official leaderboard
13 Ember-1 0.4 Sep 2026 Official leaderboard
14 GLM-5.3-Flash 0.4 FlashX Aug 2026 Official leaderboard
15 Muse Spark 1.3 0.4 Sep 2026 Official leaderboard
16 Gemini 3.8 Flash 0.4 Sep 2026 Official leaderboard
17 Grok 4.3 0.3 Apr 2026 Official leaderboard
18 MiniMax-M3 0.3 Jun 2026 Official leaderboard
19 Gemini 3.7 Flash 0.3 Aug 2026 Official leaderboard
20 Gemini 3.5 Flash 0.3 May 2026 Official leaderboard
21 DeepSeek-V3 Mar 2025 0.3 Mar 2025 Official leaderboard
22 Qwen 3.8 Max 0.3 prime Aug 2026 Official leaderboard
23 Qwen3.8-Omni-Flash 0.3 Sep 2026 Official leaderboard
24 GPT-4.1 0.3 Apr 2025 Official leaderboard
25 Qwen3.6 27B 0.2 Apr 2026 Official leaderboard
26 MiniMax-M2.7 0.2 Mar 2026 Official leaderboard
27 DeepSeek V4.1 Flash 0.2 Sep 2026 Official leaderboard
28 DeepSeek-V4-Flash 0.2 Apr 2026 Official leaderboard
29 Qwen3.8 Flash 0.2 Aug 2026 Official leaderboard
30 DeepSeek-V4-Pro 0.2 Apr 2026 Official leaderboard
31 GLM-5.3 0.2 prime Aug 2026 Official leaderboard
32 Claude Sonnet 4.6 0.1 Feb 2026 Official leaderboard
33 Gemma 4 31B IT 0.0 Apr 2026 Official leaderboard
34 Qwen 3.6 35B-A3B 0.0 Apr 2026 Official leaderboard
35 GPT-5.5 -0.1 Apr 2026 Official leaderboard
36 Command A+ -0.3 May 2026 Official leaderboard
37 GPT-6 Sol -0.4 Sep 2026 Official leaderboard
38 GPT-6 Astra -0.4 Sep 2026 Official leaderboard
39 GPT-6 Luna -0.4 Sep 2026 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
GLM-5.3-Flash
FlashX
0.4
Qwen 3.8 Max
prime
0.3
GPT-6 Sol
pro
-0.4
GPT-6 Luna
pro
-0.4

Source: Official PlotPoints Round 4: Willingness and Judgment leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
GLM-5.3-Flash
FlashX
0.4
Qwen 3.8 Max
prime
0.3
GPT-6 Sol
pro
-0.4
GPT-6 Luna
pro
-0.4

Source: Official PlotPoints Round 4: Willingness and Judgment leaderboard

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard 1 leader

Source: Official PlotPoints Round 4: Willingness and Judgment leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
-0.5 0 0.5 1 2025 2026 2027 -0.5 0 0.5 1 2025 2026 2027

Source: Official PlotPoints Round 4: Willingness and Judgment leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

Versions

More in Safety and alignment

All 636 in Safety and alignment
  • FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
  • StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is. Inkling 98.6%
  • Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. Gemini 4 Carbon 0%
  • Covert misbehavior (Thinking Machines) In use Agentic tasks that check whether a model misbehaves and conceals it. Scored by the rate of covert misbehavior. Inkling 2 Mini 8.30%
  • Destructive actions (Thinking Machines) In use Agentic tasks that check whether a model takes destructive actions it was not authorized to take. Scored by the rate of destructive actions. Inkling 2 Small 4.70%
  • Misbehavior (Thinking Machines) In use Agentic tasks that check whether a model fabricates results, ignores instructions or hides what it did when a task is hard. Scored by the rate of misbehavior. Inkling 2 Mini 9.80%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (51 entries).
  • Pass Models scored 39 models we track, plus 12 other entries on its leaderboard.
  • Pass Newest score The newest is from November 2026, for Tencent Hy4.
  • Pass Frontier models 3 of the top 10 have a score, led by GPT-6 Astra (#4).
  • Note Headroom Scored as an index, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on plotlightstudios.com: 43 rows for models we track and 12 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check