Benchmark Almanac

Lech Mazur Creative Writing

Claude Fable 5.1 3.75 leads since September 2026
Not in model cards

Models write short stories that must use 10 assigned elements such as characters, objects, and motivations. Judge models compare matched stories pairwise.

Earlier versions used 0 to 10 rubric ratings, which the current pairwise ranking replaces.

Made by
Lech Mazur
Released
January 2025
Kind
Creative writing
Level
University
Scored
Graded by a model
Answers
Open-ended
Tasks
All public
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
55
Entries on its own leaderboard, 50 models we track

Claude Fable 5.1 leads with 3.75

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

55 entries. Scroll sideways for the rest.

  1. 3.75 Claude Fable 5.1
  2. 3.69 Claude Opus 5.5
  3. 3.69 Claude Opus 5
  4. 3.42 GPT-6 Astra
  5. 2.95 GLM-5.3
  6. 2.81 Claude Fable 5
  7. 2.72 Kimi K3.1 Pro
  8. 2.42 GPT-5.6 Sol
  9. 2.39 GPT-5.5
  10. 2.35 Kimi K3
  11. 1.92 GPT-5.4
  12. 1.90 Muse Wonder
  13. 1.72 Claude Opus 4.7
  14. 1.61 Claude Sonnet 4.6
  15. 1.52 Xiaomi MiMo V2.6 Pro
  16. 1.08 Claude Opus 4.6
  17. 0.72 Gemini 4 Argon
  18. 0.72 Claude Opus 4.8
  19. 0.71 Muse Spark 1.1
  20. 0.50 Muse Spark 1.3
  21. 0.44 DeepSeek-V4-Pro
  22. 0.41 GLM-5.2
  23. 0.35 Grok 4.7
  24. 0.29 GPT-5.2
  25. 0.12 Qwen 3.8 Max
  26. 0.08 Gemini 3.8 Flash
  27. 0.03 Kimi K2.6
  28. -0.10 Muse Spark 1.2
  29. -0.14 MiniMax-M3
  30. -0.43 Mistral Medium 3.1
  31. -0.55 DeepSeek V4.1 Flash
  32. -0.70 Qwen 3.8 27B
  33. -0.72 Xiaomi MiMo V2.5 Pro
  34. -0.73 Qwen3-Max
  35. -0.76 Gemini 3.7 Flash
  36. -0.98 Qwen 3.6 Max (Preview)
  37. -1.08 GLM-5.1
  38. -1.13 Kimi K2.5
  39. -1.27 Xiaomi MiMo V2 Pro
  40. -1.30 Baidu Ernie 5.1
  41. -1.68 Mistral Large 4
  42. -1.81 Mistral Large 3
  43. -1.91 Gemma 4 31B IT
  44. -2.00 ByteDance Seed2.0 Pro
  45. -2.03 Gemini 3.5 Flash
  46. -2.21 Gemini 3.1 Pro
  47. -2.33 Qwen 3.6 Plus
  48. -2.56 Mistral Medium 3.5
  49. -2.71 Qwen3.7-Max
  50. -2.92 DeepSeek-V3.2
  51. -2.93 Grok 4.6
  52. -3.17 gpt-oss-120b
  53. -3.81 MiniMax-M2.7
  54. -4.30 Grok 4.3
  55. -5.13 Grok 4.5
  1. 1 Claude Fable 5.1 high 3.75
  2. 2 Claude Opus 5.5 high 3.69
  3. 3 Claude Opus 5 xhigh 3.69
  4. 4 GPT-6 Astra high 3.42
  5. 5 GLM-5.3 max 2.95
  6. 6 Claude Fable 5 high 2.81
  7. 7 Kimi K3.1 Pro high 2.72
  8. 8 GPT-5.6 Sol xhigh 2.42
  9. 9 GPT-5.5 xhigh 2.39
  10. 10 Kimi K3 2.35
  11. 11 GPT-5.4 medium 1.92
  12. 12 Muse Wonder high 1.90
  13. 13 Claude Opus 4.7 high 1.72
  14. 14 Claude Sonnet 4.6 thinking 16K 1.61
  15. 15 Xiaomi MiMo V2.6 Pro thinking 1.52
  16. 16 Claude Opus 4.6 thinking 16K 1.08
  17. 17 Gemini 4 Argon high 0.72
  18. 18 Claude Opus 4.8 xhigh 0.72
  19. 19 Muse Spark 1.1 high 0.71
  20. 20 Muse Spark 1.3 high 0.50
  21. 21 DeepSeek-V4-Pro high 0.44
  22. 22 GLM-5.2 max 0.41
  23. 23 Grok 4.7 high 0.35
  24. 24 GPT-5.2 medium 0.29
  25. 25 Qwen 3.8 Max 0.12
  26. 26 Gemini 3.8 Flash high 0.08
  27. 27 Kimi K2.6 0.03
  28. 28 Muse Spark 1.2 high -0.10
  29. 29 MiniMax-M3 -0.14
  30. 30 Mistral Medium 3.1 -0.43
  31. 31 DeepSeek V4.1 Flash high -0.55
  32. 32 Qwen 3.8 27B -0.70
  33. 33 Xiaomi MiMo V2.5 Pro -0.72
  34. 34 Qwen3-Max -0.73
  35. 35 Gemini 3.7 Flash high -0.76
  36. 36 Qwen 3.6 Max (Preview) -0.98
  37. 37 GLM-5.1 -1.08
  38. 38 Kimi K2.5 Thinking -1.13
  39. 39 Xiaomi MiMo V2 Pro -1.27
  40. 40 Baidu Ernie 5.1 -1.30
  41. 41 Mistral Large 4 high -1.68
  42. 42 Mistral Large 3 -1.81
  43. 43 Gemma 4 31B IT Reasoning -1.91
  44. 44 ByteDance Seed2.0 Pro -2.00
  45. 45 Gemini 3.5 Flash -2.03
  46. 46 Gemini 3.1 Pro -2.21
  47. 47 Qwen 3.6 Plus -2.33
  48. 48 Mistral Medium 3.5 -2.56
  49. 49 Qwen3.7-Max -2.71
  50. 50 DeepSeek-V3.2 -2.92
  51. 51 Grok 4.6 high -2.93
  52. 52 gpt-oss-120b -3.17
  53. 53 MiniMax-M2.7 -3.81
  54. 54 Grok 4.3 -4.30
  55. 55 Grok 4.5 high -5.13

Source: Official Lech Mazur Creative Writing leaderboard

#EntryScoreSettingDateSource
1 Claude Fable 5.1 3.75 high Sep 2026 Official leaderboard
2 Claude Opus 5.5 3.69 high Sep 2026 Official leaderboard
3 Claude Opus 5 3.69 xhigh Jul 2026 Official leaderboard
4 GPT-6 Astra 3.42 high Sep 2026 Official leaderboard
5 GLM-5.3 2.95 max Aug 2026 Official leaderboard
6 Claude Fable 5 2.81 high Jun 2026 Official leaderboard
7 Kimi K3.1 Pro 2.72 high Oct 2026 Official leaderboard
8 GPT-5.6 Sol 2.42 xhigh Jul 2026 Official leaderboard
9 GPT-5.5 2.39 xhigh Apr 2026 Official leaderboard
10 Kimi K3 2.35 Jul 2026 Official leaderboard
11 GPT-5.4 1.92 medium Mar 2026 Official leaderboard
12 Muse Wonder 1.90 high Oct 2026 Official leaderboard
13 Claude Opus 4.7 1.72 high Apr 2026 Official leaderboard
14 Claude Sonnet 4.6 1.61 thinking 16K Feb 2026 Official leaderboard
15 Xiaomi MiMo V2.6 Pro 1.52 thinking Official leaderboard
16 Claude Opus 4.6 1.08 thinking 16K Feb 2026 Official leaderboard
17 Gemini 4 Argon 0.72 high Sep 2026 Official leaderboard
18 Claude Opus 4.8 0.72 xhigh May 2026 Official leaderboard
19 Muse Spark 1.1 0.71 high Jul 2026 Official leaderboard
20 Muse Spark 1.3 0.50 high Sep 2026 Official leaderboard
21 DeepSeek-V4-Pro 0.44 high Apr 2026 Official leaderboard
22 GLM-5.2 0.41 max Jun 2026 Official leaderboard
23 Grok 4.7 0.35 high Sep 2026 Official leaderboard
24 GPT-5.2 0.29 medium Dec 2025 Official leaderboard
25 Qwen 3.8 Max 0.12 Aug 2026 Official leaderboard
26 Gemini 3.8 Flash 0.08 high Sep 2026 Official leaderboard
27 Kimi K2.6 0.03 Apr 2026 Official leaderboard
28 Muse Spark 1.2 -0.10 high Aug 2026 Official leaderboard
29 MiniMax-M3 -0.14 Jun 2026 Official leaderboard
30 Mistral Medium 3.1 -0.43 Aug 2025 Official leaderboard
31 DeepSeek V4.1 Flash -0.55 high Sep 2026 Official leaderboard
32 Qwen 3.8 27B -0.70 Aug 2026 Official leaderboard
33 Xiaomi MiMo V2.5 Pro -0.72 Official leaderboard
34 Qwen3-Max -0.73 Sep 2025 Official leaderboard
35 Gemini 3.7 Flash -0.76 high Aug 2026 Official leaderboard
36 Qwen 3.6 Max (Preview) -0.98 Apr 2026 Official leaderboard
37 GLM-5.1 -1.08 Apr 2026 Official leaderboard
38 Kimi K2.5 -1.13 Thinking Jan 2026 Official leaderboard
39 Xiaomi MiMo V2 Pro -1.27 Official leaderboard
40 Baidu Ernie 5.1 -1.30 Official leaderboard
41 Mistral Large 4 -1.68 high Oct 2026 Official leaderboard
42 Mistral Large 3 -1.81 Dec 2025 Official leaderboard
43 Gemma 4 31B IT -1.91 Reasoning Apr 2026 Official leaderboard
44 ByteDance Seed2.0 Pro -2.00 Official leaderboard
45 Gemini 3.5 Flash -2.03 May 2026 Official leaderboard
46 Gemini 3.1 Pro -2.21 Feb 2026 Official leaderboard
47 Qwen 3.6 Plus -2.33 Mar 2026 Official leaderboard
48 Mistral Medium 3.5 -2.56 Apr 2026 Official leaderboard
49 Qwen3.7-Max -2.71 May 2026 Official leaderboard
50 DeepSeek-V3.2 -2.92 Dec 2025 Official leaderboard
51 Grok 4.6 -2.93 high Aug 2026 Official leaderboard
52 gpt-oss-120b -3.17 Aug 2025 Official leaderboard
53 MiniMax-M2.7 -3.81 Mar 2026 Official leaderboard
54 Grok 4.3 -4.30 Apr 2026 Official leaderboard
55 Grok 4.5 -5.13 high Jul 2026 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Opus 5
high xhigh
3.69
GPT-5.6 Sol
high
2.42
GPT-5.4
medium
1.92
Claude Opus 4.8
high xhigh
0.72
DeepSeek-V4-Pro
high
0.44

Source: Official Lech Mazur Creative Writing leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5.1 high 3.75
Claude Opus 5.5 high 3.69
Claude Opus 5 xhigh 3.69
GPT-6 Astra high 3.42
Claude Opus 5 high 3.32
GLM-5.3 max 2.95
Claude Fable 5 high 2.81
Kimi K3.1 Pro high 2.72
GPT-5.6 Sol xhigh 2.42
GPT-5.5 xhigh 2.39
Kimi K3 2.35
GPT-5.6 Sol high 2.24
GPT-5.4 medium 1.92
GPT-5.4 xhigh 1.92
Muse Wonder high 1.90
Claude Opus 4.7 high 1.72
Claude Sonnet 4.6 thinking 16K 1.61
Claude Opus 4.6 thinking 16K 1.08
Gemini 4 Argon high 0.72
Claude Opus 4.8 xhigh 0.72
Muse Spark 1.1 high 0.71
Muse Spark 1.3 high 0.50
DeepSeek-V4-Pro high 0.44
GLM-5.2 max 0.41
Grok 4.7 high 0.35
GPT-5.2 medium 0.29
Claude Opus 4.8 high 0.20
Qwen 3.8 Max 0.12
Gemini 3.8 Flash high 0.08
Kimi K2.6 0.03
Muse Spark 1.2 high -0.10
MiniMax-M3 -0.14
Mistral Medium 3.1 -0.43
DeepSeek V4.1 Flash high -0.55
DeepSeek-V4-Pro -0.59
Qwen 3.8 27B -0.70
Qwen3-Max -0.73
Gemini 3.7 Flash high -0.76
Qwen 3.6 Max (Preview) -0.98
GLM-5.1 -1.08
Kimi K2.5 Thinking -1.13
Mistral Large 4 high -1.68
Mistral Large 3 -1.81
Gemma 4 31B IT Reasoning -1.91
Gemini 3.5 Flash -2.03
Gemini 3.1 Pro -2.21
Qwen 3.6 Plus -2.33
Mistral Medium 3.5 -2.56
Qwen3.7-Max -2.71
DeepSeek-V3.2 -2.92
Grok 4.6 high -2.93
gpt-oss-120b -3.17
MiniMax-M2.7 -3.81
Grok 4.3 -4.30
Grok 4.5 high -5.13

10 models have held number one on the official leaderboard since its release in January 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Mistral AI Anthropic
Own leaderboard 10 leaders
Claude Fable 5.1 3.75

Source: Official Lech Mazur Creative Writing leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
-6.00 -4.00 -2.00 0.00 2.00 4.00 2026 2027 -6.00 -4.00 -2.00 0.00 2.00 4.00 2026 2027

Source: Official Lech Mazur Creative Writing leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Writing and taste

All 30 in Writing and taste
  • Design Bench (Moonshot) In use Models make graphic designs from direct prompts or few-shot reference images. An automated grader rates the results as Elo scores. Kimi K3.1 Pro 957
  • WritingBench Fading Models complete 1,000 real-world writing requests across 6 domains and 100 subdomains. A judge model scores each reply on five query-specific criteria. GPT-5 83.9
  • Creative Writing (MiniMax in-house) Only in system cards Models write creative pieces for real Hailuo AI user requests. Automatic assessments score the writing. MiniMax-Text-01 81.3%
  • DeepSeek Chinese functional writing evaluation Only in system cards Models answer 3,170 everyday Chinese writing prompts such as reports, emails and speeches. Pairwise judging against a baseline model gives a win rate. DeepSeek-V4-Pro 62.6%
  • Slide generation human evaluation (Z.ai) Only in system cards People compare HTML slide decks from two models on content, layout and visual quality. Metric is overall win rate. GLM-5 67.5%
  • Task-dependent output homogenization evaluation Only in system cards Models answer prompts several times. A judge scores functional diversity where it is wanted and consistency where one answer is right. Muse Spark 0.21