Benchmark Almanac

LisanBench

Muse Wonder Full 37152.2 leads since October 2026
Not in model cards

Models build the longest chain of distinct English words from a given start word, each differing from the last by one letter. The score sums valid chain lengths over all start words.

Three trials per start word, averaged. A difficulty-weighted score gives common words less weight. The site also tracks reasoning tokens and full-run cost. Tokenization affects letter-level tasks like this one.

Released
May 2025
Kind
Puzzles and chess
Level
Everyday
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
137
Entries on its own leaderboard, 133 models we track

Muse Wonder Full leads with 37152.2

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

137 entries. Scroll sideways for the rest.

  1. 37152.2 Muse Wonder Full
  2. 27973.3 Muse Wonder
  3. 24029.0 Claude Opus 5.5
  4. 21107.3 DeepSeek V4.1 Flash 1017
  5. 15428.3 Claude Opus 5
  6. 15100.3 Claude Fable 5.1
  7. 14400.0 Claude Opus 4.7
  8. 14206.3 Claude Opus 4.6
  9. 13787.0 Claude Sonnet 5.5
  10. 13680.3 Claude Fable 5
  11. 12897.0 Claude Haiku 5.5
  12. 12060.0 Claude Sonnet 4.6
  13. 10520.7 Kimi K3
  14. 9844.7 GPT-5.5
  15. 9463.0 Claude Opus 4.8
  16. 8736.0 GPT-6.1 Sol
  17. 8222.0 GPT-6 Sol
  18. 7560.3 Gemini 4 Argon
  19. 7392.7 GPT-6 Astra
  20. 6445.0 Gemini 3.1 Pro
  21. 5724.0 Claude Sonnet 5
  22. 5513.0 Claude Opus 4.5
  23. 5334.7 GPT-5.4
  24. 5072.7 DeepSeek-V3.2-Speciale
  25. 4699.0 GPT-5.6 Sol
  26. 4677.0 Gemini 3 Pro
  27. 4180.7 GPT-5.6 Luna
  28. 4123.0 Gemini 3.5 Flash
  29. 4042.2 Grok 4
  30. 3863.7 GPT-6 Luna
  31. 3856.3 Grok 4.20
  32. 3592.3 GPT-5.2
  33. 3308.7 o3
  34. 3302.7 GPT-5.6 Terra
  35. 3256.3 DeepSeek-V4-Pro
  36. 2789.0 DeepSeek-V4-Flash
  37. 2749.6 Grok 4 Fast
  38. 2665.3 Claude Sonnet 4.5
  39. 2409.7 Step 3.5 Flash
  40. 2348.3 DeepSeek-V3.2
  41. 2247.0 GPT-5
  42. 2236.0 Claude Sonnet 4
  43. 1996.0 Gemini 3 Flash
  44. 1851.7 Kimi K2.5
  45. 1834.7 GLM-5.2
  46. 1817.3 Grok 4.1 Fast
  47. 1753.0 Seed2.0 Pro
  48. 1561.7 GPT-5 mini
  49. 1426.0 GPT-5.4 Nano
  50. 1366.3 o3-mini
  51. 1293.3 Qwen3.5 397B-A17B
  52. 1217.7 GPT-5 nano
  53. 1143.7 gpt-oss-120b
  54. 1140.7 GPT-5.4 Mini
  55. 1087.3 Seed1.8
  56. 986.8 GLM-5
  57. 973.0 Seed2.0 Lite (0215)
  58. 852.5 MiniMax-M2.5
  59. 849.3 Grok-3 mini
  60. 803.7 Qwen3-235B-A22B-Thinking
  61. 800.0 o4-mini
  62. 739.0 Grok 3
  63. 666.7 Seed2.0 Mini
  64. 598.0 Claude Opus 4
  65. 569.0 gpt-oss-20b
  66. 556.0 Gemini 2.5 Pro
  67. 498.3 Claude 3.7 Sonnet
  68. 492.0 Gemini 1.5 Pro
  69. 459.7 Claude 3.5 Sonnet (October 2024)
  70. 447.3 Qwen3.5-35B-A3B
  71. 400.3 GLM-4.5
  72. 394.7 Claude Opus 4.1
  73. 392.7 Qwen3.5-122B-A10B
  74. 372.3 DeepSeek-V3
  75. 365.3 Claude 3.5 Sonnet
  76. 362.3 Gemini 2.5 Flash
  77. 347.3 Kimi K2
  78. 333.7 Olmo 3 32B Think
  79. 302.7 GPT-4o
  80. 300.0 GPT-4 Turbo
  81. 295.3 o1-mini
  82. 288.7 DeepSeek-R1
  83. 263.3 Qwen3-4B
  84. 249.0 Claude 3 Opus
  85. 238.7 GLM-4.5-Air
  86. 218.0 Gemini 1.5 Flash
  87. 213.7 Qwen3-Coder-480B-A35B
  88. 200.0 Llama 3.1-405B
  89. 197.7 Gemini 2.0 Flash
  90. 197.0 GLM-4.7
  91. 194.7 Polaris Alpha
  92. 194.7 MiniMax-M1-80k
  93. 189.7 GPT-3.5 Turbo
  94. 180.3 Devstral Medium
  95. 176.3 ERNIE-4.5-300B-A47B
  96. 175.3 Gemma 3 27B
  97. 172.0 GPT-4.1
  98. 165.3 Horizon Beta
  99. 157.3 Gemini 2.0 Flash-Lite
  100. 157.0 GLM-4.6
  101. 151.3 Llama 3.1-70B
  102. 148.0 Sherlock Dash Alpha
  103. 146.0 Amazon Nova Pro
  104. 144.0 Claude 3.5 Haiku
  105. 138.0 Mistral Large 2
  106. 137.0 Claude Haiku 4.5
  107. 135.7 Claude 3 Haiku
  108. 133.3 Qwen3-1.7B
  109. 133.3 Qwen3-235B-A22B-Instruct
  110. 131.0 Llama 4 Maverick
  111. 130.7 Mistral Medium 3
  112. 126.3 Llama 4 Scout
  113. 126.3 GPT-4.1 mini
  114. 122.3 Gemini 2.5 Flash-Lite
  115. 114.0 Gemma 3 12B
  116. 104.3 MiMo-V2-Flash
  117. 89.7 GPT-4o mini
  118. 84.0 Amazon Nova Lite
  119. 77.0 Gemini 1.5 Flash 8B
  120. 76.0 Amazon Nova Micro
  121. 74.7 Qwen3-30B-A3B-Instruct
  122. 68.0 Qwen3-32B
  123. 58.3 Qwen3-14B
  124. 54.0 GPT-4.1 nano
  125. 52.7 Mistral Small 3.2
  126. 51.3 Qwen3-8B
  127. 50.0 Gemma 3 4B
  128. 48.7 Mistral NeMo
  129. 44.0 Codestral 2508
  130. 41.7 Ministral 3 14B
  131. 40.0 Devstral Small 1.0
  132. 32.3 Ministral 3 8B Instruct
  133. 31.7 Qwen3-0.6B
  134. 23.3 Ministral 3 3B Instruct
  135. 12.7 Llama 3.1-8B
  136. 8.00 Llama 3.2 3B
  137. 3.67 Llama 3.2 1B
  1. 1 Muse Wonder Full max 37152.2
  2. 2 Muse Wonder max 27973.3
  3. 3 Claude Opus 5.5 xhigh 24029.0
  4. 4 DeepSeek V4.1 Flash 1017 max 21107.3
  5. 5 Claude Opus 5 high 15428.3
  6. 6 Claude Fable 5.1 xhigh 15100.3
  7. 7 Claude Opus 4.7 xhigh 14400.0
  8. 8 Claude Opus 4.6 16k 14206.3
  9. 9 Claude Sonnet 5.5 xhigh 13787.0
  10. 10 Claude Fable 5 medium 13680.3
  11. 11 Claude Haiku 5.5 xhigh 12897.0
  12. 12 Claude Sonnet 4.6 16k 12060.0
  13. 13 Kimi K3 max 10520.7
  14. 14 GPT-5.5 medium 9844.7
  15. 15 Claude Opus 4.8 high 9463.0
  16. 16 GPT-6.1 Sol high 8736.0
  17. 17 GPT-6 Sol high 8222.0
  18. 18 Gemini 4 Argon high 7560.3
  19. 19 GPT-6 Astra medium 7392.7
  20. 20 Gemini 3.1 Pro high 6445.0
  21. 21 Claude Sonnet 5 high 5724.0
  22. 22 Claude Opus 4.5 16k 5513.0
  23. 23 GPT-5.4 medium 5334.7
  24. 24 DeepSeek-V3.2-Speciale thinking 5072.7
  25. 25 GPT-5.6 Sol medium 4699.0
  26. 26 Gemini 3 Pro high 4677.0
  27. 27 GPT-5.6 Luna xhigh 4180.7
  28. 28 Gemini 3.5 Flash high 4123.0
  29. 29 Grok 4 medium 4042.2
  30. 30 GPT-6 Luna high 3863.7
  31. 31 Grok 4.20 thinking 3856.3
  32. 32 GPT-5.2 medium 3592.3
  33. 33 o3 medium 3308.7
  34. 34 GPT-5.6 Terra medium 3302.7
  35. 35 DeepSeek-V4-Pro high 3256.3
  36. 36 DeepSeek-V4-Flash high 2789.0
  37. 37 Grok 4 Fast thinking 2749.6
  38. 38 Claude Sonnet 4.5 16k 2665.3
  39. 39 Step 3.5 Flash thinking 2409.7
  40. 40 DeepSeek-V3.2 thinking 2348.3
  41. 41 GPT-5 medium 2247.0
  42. 42 Claude Sonnet 4 16k 2236.0
  43. 43 Gemini 3 Flash high 1996.0
  44. 44 Kimi K2.5 thinking 1851.7
  45. 45 GLM-5.2 high 1834.7
  46. 46 Grok 4.1 Fast thinking 1817.3
  47. 47 Seed2.0 Pro thinking 1753.0
  48. 48 GPT-5 mini medium 1561.7
  49. 49 GPT-5.4 Nano medium 1426.0
  50. 50 o3-mini medium 1366.3
  51. 51 Qwen3.5 397B-A17B thinking 1293.3
  52. 52 GPT-5 nano medium 1217.7
  53. 53 gpt-oss-120b medium 1143.7
  54. 54 GPT-5.4 Mini xhigh 1140.7
  55. 55 Seed1.8 thinking 1087.3
  56. 56 GLM-5 thinking 986.8
  57. 57 Seed2.0 Lite (0215) thinking 973.0
  58. 58 MiniMax-M2.5 thinking 852.5
  59. 59 Grok-3 mini medium 849.3
  60. 60 Qwen3-235B-A22B-Thinking Jul 2025, thinking 803.7
  61. 61 o4-mini medium 800.0
  62. 62 Grok 3 739.0
  63. 63 Seed2.0 Mini thinking 666.7
  64. 64 Claude Opus 4 598.0
  65. 65 gpt-oss-20b medium 569.0
  66. 66 Gemini 2.5 Pro Jun 2025, 16k 556.0
  67. 67 Claude 3.7 Sonnet 498.3
  68. 68 Gemini 1.5 Pro Feb 2024 492.0
  69. 69 Claude 3.5 Sonnet (October 2024) 459.7
  70. 70 Qwen3.5-35B-A3B thinking 447.3
  71. 71 GLM-4.5 thinking 400.3
  72. 72 Claude Opus 4.1 394.7
  73. 73 Qwen3.5-122B-A10B thinking 392.7
  74. 74 DeepSeek-V3 372.3
  75. 75 Claude 3.5 Sonnet 365.3
  76. 76 Gemini 2.5 Flash Jun 2025, 16k 362.3
  77. 77 Kimi K2 Jul 2025 347.3
  78. 78 Olmo 3 32B Think thinking 333.7
  79. 79 GPT-4o Mar 2025 302.7
  80. 80 GPT-4 Turbo Apr 2024 300.0
  81. 81 o1-mini medium 295.3
  82. 82 DeepSeek-R1 May 2025, thinking 288.7
  83. 83 Qwen3-4B 16k 263.3
  84. 84 Claude 3 Opus 249.0
  85. 85 GLM-4.5-Air thinking 238.7
  86. 86 Gemini 1.5 Flash May 2024 218.0
  87. 87 Qwen3-Coder-480B-A35B 213.7
  88. 88 Llama 3.1-405B 200.0
  89. 89 Gemini 2.0 Flash Feb 2025 197.7
  90. 90 GLM-4.7 thinking 197.0
  91. 91 Polaris Alpha 194.7
  92. 92 MiniMax-M1-80k thinking 194.7
  93. 93 GPT-3.5 Turbo Jan 2024 189.7
  94. 94 Devstral Medium 180.3
  95. 95 ERNIE-4.5-300B-A47B 176.3
  96. 96 Gemma 3 27B 175.3
  97. 97 GPT-4.1 172.0
  98. 98 Horizon Beta 165.3
  99. 99 Gemini 2.0 Flash-Lite 157.3
  100. 100 GLM-4.6 thinking 157.0
  101. 101 Llama 3.1-70B 151.3
  102. 102 Sherlock Dash Alpha 148.0
  103. 103 Amazon Nova Pro 146.0
  104. 104 Claude 3.5 Haiku 144.0
  105. 105 Mistral Large 2 Nov 2024 138.0
  106. 106 Claude Haiku 4.5 137.0
  107. 107 Claude 3 Haiku 135.7
  108. 108 Qwen3-1.7B 16k 133.3
  109. 109 Qwen3-235B-A22B-Instruct Jul 2025 133.3
  110. 110 Llama 4 Maverick 131.0
  111. 111 Mistral Medium 3 130.7
  112. 112 Llama 4 Scout 126.3
  113. 113 GPT-4.1 mini 126.3
  114. 114 Gemini 2.5 Flash-Lite Jun 2025, 16k 122.3
  115. 115 Gemma 3 12B 114.0
  116. 116 MiMo-V2-Flash thinking 104.3
  117. 117 GPT-4o mini 89.7
  118. 118 Amazon Nova Lite 84.0
  119. 119 Gemini 1.5 Flash 8B 77.0
  120. 120 Amazon Nova Micro 76.0
  121. 121 Qwen3-30B-A3B-Instruct Jul 2025 74.7
  122. 122 Qwen3-32B 68.0
  123. 123 Qwen3-14B 58.3
  124. 124 GPT-4.1 nano 54.0
  125. 125 Mistral Small 3.2 52.7
  126. 126 Qwen3-8B 51.3
  127. 127 Gemma 3 4B 50.0
  128. 128 Mistral NeMo 48.7
  129. 129 Codestral 2508 44.0
  130. 130 Ministral 3 14B 41.7
  131. 131 Devstral Small 1.0 40.0
  132. 132 Ministral 3 8B Instruct 32.3
  133. 133 Qwen3-0.6B 16k 31.7
  134. 134 Ministral 3 3B Instruct 23.3
  135. 135 Llama 3.1-8B 12.7
  136. 136 Llama 3.2 3B 8.00
  137. 137 Llama 3.2 1B 3.67

Source: Official LisanBench leaderboard

#EntryScoreSettingDateSource
1 Muse Wonder Full 37152.2 max Oct 2026 Official leaderboard
2 Muse Wonder 27973.3 max Oct 2026 Official leaderboard
3 Claude Opus 5.5 24029.0 xhigh Sep 2026 Official leaderboard
4 DeepSeek V4.1 Flash 1017 21107.3 max Oct 2026 Official leaderboard
5 Claude Opus 5 15428.3 high Jul 2026 Official leaderboard
6 Claude Fable 5.1 15100.3 xhigh Sep 2026 Official leaderboard
7 Claude Opus 4.7 14400.0 xhigh Apr 2026 Official leaderboard
8 Claude Opus 4.6 14206.3 16k Feb 2026 Official leaderboard
9 Claude Sonnet 5.5 13787.0 xhigh Sep 2026 Official leaderboard
10 Claude Fable 5 13680.3 medium Jun 2026 Official leaderboard
11 Claude Haiku 5.5 12897.0 xhigh Oct 2026 Official leaderboard
12 Claude Sonnet 4.6 12060.0 16k Feb 2026 Official leaderboard
13 Kimi K3 10520.7 max Jul 2026 Official leaderboard
14 GPT-5.5 9844.7 medium Apr 2026 Official leaderboard
15 Claude Opus 4.8 9463.0 high May 2026 Official leaderboard
16 GPT-6.1 Sol 8736.0 high Sep 2026 Official leaderboard
17 GPT-6 Sol 8222.0 high Sep 2026 Official leaderboard
18 Gemini 4 Argon 7560.3 high Sep 2026 Official leaderboard
19 GPT-6 Astra 7392.7 medium Sep 2026 Official leaderboard
20 Gemini 3.1 Pro 6445.0 high Feb 2026 Official leaderboard
21 Claude Sonnet 5 5724.0 high Jun 2026 Official leaderboard
22 Claude Opus 4.5 5513.0 16k Nov 2025 Official leaderboard
23 GPT-5.4 5334.7 medium Mar 2026 Official leaderboard
24 DeepSeek-V3.2-Speciale 5072.7 thinking Dec 2025 Official leaderboard
25 GPT-5.6 Sol 4699.0 medium Jul 2026 Official leaderboard
26 Gemini 3 Pro 4677.0 high Nov 2025 Official leaderboard
27 GPT-5.6 Luna 4180.7 xhigh Jul 2026 Official leaderboard
28 Gemini 3.5 Flash 4123.0 high May 2026 Official leaderboard
29 Grok 4 4042.2 medium Jul 2025 Official leaderboard
30 GPT-6 Luna 3863.7 high Sep 2026 Official leaderboard
31 Grok 4.20 3856.3 thinking Feb 2026 Official leaderboard
32 GPT-5.2 3592.3 medium Dec 2025 Official leaderboard
33 o3 3308.7 medium Apr 2025 Official leaderboard
34 GPT-5.6 Terra 3302.7 medium Jul 2026 Official leaderboard
35 DeepSeek-V4-Pro 3256.3 high Apr 2026 Official leaderboard
36 DeepSeek-V4-Flash 2789.0 high Apr 2026 Official leaderboard
37 Grok 4 Fast 2749.6 thinking Sep 2025 Official leaderboard
38 Claude Sonnet 4.5 2665.3 16k Sep 2025 Official leaderboard
39 Step 3.5 Flash 2409.7 thinking Feb 2026 Official leaderboard
40 DeepSeek-V3.2 2348.3 thinking Dec 2025 Official leaderboard
41 GPT-5 2247.0 medium Aug 2025 Official leaderboard
42 Claude Sonnet 4 2236.0 16k May 2025 Official leaderboard
43 Gemini 3 Flash 1996.0 high Dec 2025 Official leaderboard
44 Kimi K2.5 1851.7 thinking Jan 2026 Official leaderboard
45 GLM-5.2 1834.7 high Jun 2026 Official leaderboard
46 Grok 4.1 Fast 1817.3 thinking Nov 2025 Official leaderboard
47 Seed2.0 Pro 1753.0 thinking Feb 2026 Official leaderboard
48 GPT-5 mini 1561.7 medium Aug 2025 Official leaderboard
49 GPT-5.4 Nano 1426.0 medium Mar 2026 Official leaderboard
50 o3-mini 1366.3 medium Jan 2025 Official leaderboard
51 Qwen3.5 397B-A17B 1293.3 thinking Feb 2026 Official leaderboard
52 GPT-5 nano 1217.7 medium Aug 2025 Official leaderboard
53 gpt-oss-120b 1143.7 medium Aug 2025 Official leaderboard
54 GPT-5.4 Mini 1140.7 xhigh Mar 2026 Official leaderboard
55 Seed1.8 1087.3 thinking Dec 2025 Official leaderboard
56 GLM-5 986.8 thinking Feb 2026 Official leaderboard
57 Seed2.0 Lite (0215) 973.0 thinking Feb 2026 Official leaderboard
58 MiniMax-M2.5 852.5 thinking Feb 2026 Official leaderboard
59 Grok-3 mini 849.3 medium Apr 2025 Official leaderboard
60 Qwen3-235B-A22B-Thinking Jul 2025 803.7 thinking Jul 2025 Official leaderboard
61 o4-mini 800.0 medium Apr 2025 Official leaderboard
62 Grok 3 739.0 Apr 2025 Official leaderboard
63 Seed2.0 Mini 666.7 thinking Feb 2026 Official leaderboard
64 Claude Opus 4 598.0 May 2025 Official leaderboard
65 gpt-oss-20b 569.0 medium Aug 2025 Official leaderboard
66 Gemini 2.5 Pro Jun 2025 556.0 16k Jun 2025 Official leaderboard
67 Claude 3.7 Sonnet 498.3 Feb 2025 Official leaderboard
68 Gemini 1.5 Pro Feb 2024 492.0 Feb 2024 Official leaderboard
69 Claude 3.5 Sonnet (October 2024) 459.7 Oct 2024 Official leaderboard
70 Qwen3.5-35B-A3B 447.3 thinking Feb 2026 Official leaderboard
71 GLM-4.5 400.3 thinking Aug 2025 Official leaderboard
72 Claude Opus 4.1 394.7 Aug 2025 Official leaderboard
73 Qwen3.5-122B-A10B 392.7 thinking Feb 2026 Official leaderboard
74 DeepSeek-V3 372.3 Dec 2024 Official leaderboard
75 Claude 3.5 Sonnet 365.3 Jun 2024 Official leaderboard
76 Gemini 2.5 Flash Jun 2025 362.3 16k Jun 2025 Official leaderboard
77 Kimi K2 Jul 2025 347.3 Jul 2025 Official leaderboard
78 Olmo 3 32B Think 333.7 thinking Nov 2025 Official leaderboard
79 GPT-4o Mar 2025 302.7 Mar 2025 Official leaderboard
80 GPT-4 Turbo Apr 2024 300.0 Apr 2024 Official leaderboard
81 o1-mini 295.3 medium Sep 2024 Official leaderboard
82 DeepSeek-R1 May 2025 288.7 thinking May 2025 Official leaderboard
83 Qwen3-4B 263.3 16k Apr 2025 Official leaderboard
84 Claude 3 Opus 249.0 Feb 2024 Official leaderboard
85 GLM-4.5-Air 238.7 thinking Jul 2025 Official leaderboard
86 Gemini 1.5 Flash May 2024 218.0 May 2024 Official leaderboard
87 Qwen3-Coder-480B-A35B 213.7 Jul 2025 Official leaderboard
88 Llama 3.1-405B 200.0 Jul 2024 Official leaderboard
89 Gemini 2.0 Flash Feb 2025 197.7 Feb 2025 Official leaderboard
90 GLM-4.7 197.0 thinking Dec 2025 Official leaderboard
91 Polaris Alpha 194.7 Official leaderboard
92 MiniMax-M1-80k 194.7 thinking Jun 2025 Official leaderboard
93 GPT-3.5 Turbo Jan 2024 189.7 Jan 2024 Official leaderboard
94 Devstral Medium 180.3 Jul 2025 Official leaderboard
95 ERNIE-4.5-300B-A47B 176.3 Jun 2025 Official leaderboard
96 Gemma 3 27B 175.3 Mar 2025 Official leaderboard
97 GPT-4.1 172.0 Apr 2025 Official leaderboard
98 Horizon Beta 165.3 Official leaderboard
99 Gemini 2.0 Flash-Lite 157.3 Feb 2025 Official leaderboard
100 GLM-4.6 157.0 thinking Sep 2025 Official leaderboard
101 Llama 3.1-70B 151.3 Jul 2024 Official leaderboard
102 Sherlock Dash Alpha 148.0 Official leaderboard
103 Amazon Nova Pro 146.0 Dec 2024 Official leaderboard
104 Claude 3.5 Haiku 144.0 Oct 2024 Official leaderboard
105 Mistral Large 2 Nov 2024 138.0 Nov 2024 Official leaderboard
106 Claude Haiku 4.5 137.0 Oct 2025 Official leaderboard
107 Claude 3 Haiku 135.7 Mar 2024 Official leaderboard
108 Qwen3-1.7B 133.3 16k Apr 2025 Official leaderboard
109 Qwen3-235B-A22B-Instruct Jul 2025 133.3 Jul 2025 Official leaderboard
110 Llama 4 Maverick 131.0 Apr 2025 Official leaderboard
111 Mistral Medium 3 130.7 May 2025 Official leaderboard
112 Llama 4 Scout 126.3 Apr 2025 Official leaderboard
113 GPT-4.1 mini 126.3 Apr 2025 Official leaderboard
114 Gemini 2.5 Flash-Lite Jun 2025 122.3 16k Jun 2025 Official leaderboard
115 Gemma 3 12B 114.0 Mar 2025 Official leaderboard
116 MiMo-V2-Flash 104.3 thinking Dec 2025 Official leaderboard
117 GPT-4o mini 89.7 Jul 2024 Official leaderboard
118 Amazon Nova Lite 84.0 Dec 2024 Official leaderboard
119 Gemini 1.5 Flash 8B 77.0 Oct 2024 Official leaderboard
120 Amazon Nova Micro 76.0 Dec 2024 Official leaderboard
121 Qwen3-30B-A3B-Instruct Jul 2025 74.7 Jul 2025 Official leaderboard
122 Qwen3-32B 68.0 Apr 2025 Official leaderboard
123 Qwen3-14B 58.3 Apr 2025 Official leaderboard
124 GPT-4.1 nano 54.0 Apr 2025 Official leaderboard
125 Mistral Small 3.2 52.7 Jun 2025 Official leaderboard
126 Qwen3-8B 51.3 Apr 2025 Official leaderboard
127 Gemma 3 4B 50.0 Mar 2025 Official leaderboard
128 Mistral NeMo 48.7 Jul 2024 Official leaderboard
129 Codestral 2508 44.0 Official leaderboard
130 Ministral 3 14B 41.7 Dec 2025 Official leaderboard
131 Devstral Small 1.0 40.0 May 2025 Official leaderboard
132 Ministral 3 8B Instruct 32.3 Dec 2025 Official leaderboard
133 Qwen3-0.6B 31.7 16k Apr 2025 Official leaderboard
134 Ministral 3 3B Instruct 23.3 Dec 2025 Official leaderboard
135 Llama 3.1-8B 12.7 Jul 2024 Official leaderboard
136 Llama 3.2 3B 8.00 Sep 2024 Official leaderboard
137 Llama 3.2 1B 3.67 Sep 2024 Official leaderboard

Reasoning Tokens vs Path Length

Path Length against the average reasoning tokens per trial, on a log scale; runs without reasoning sit at zero. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

0 10K 20K 30K 40K 101001,00010K100K Average reasoning tokens per trial (tokens), log scale, lower is better Path Length o1-mini (medium): Path Length 295, average reasoning tokens per trial 1,839 tokens o3-mini (medium): Path Length 1,366, average reasoning tokens per trial 7,625 tokens Grok-3 mini (medium): Path Length 849, average reasoning tokens per trial 3,451 tokens o3 (medium): Path Length 3,309, average reasoning tokens per trial 25K tokens o4-mini (medium): Path Length 800, average reasoning tokens per trial 7,145 tokens Qwen3-0.6B (16k): Path Length 31.7, average reasoning tokens per trial 4,777 tokens Qwen3-1.7B (16k): Path Length 133, average reasoning tokens per trial 7,405 tokens Qwen3-4B (16k): Path Length 263, average reasoning tokens per trial 8,386 tokens Claude Sonnet 4 (16k): Path Length 2,236, average reasoning tokens per trial 16K tokens DeepSeek-R1 (thinking): Path Length 289, average reasoning tokens per trial 4,948 tokens Gemini 2.5 Pro (16k): Path Length 556, average reasoning tokens per trial 13K tokens MiniMax-M1-80k (thinking): Path Length 195, average reasoning tokens per trial 20K tokens Gemini 2.5 Flash (16k): Path Length 362, average reasoning tokens per trial 20K tokens Grok 4 (medium): Path Length 4,042, average reasoning tokens per trial 13K tokens GLM-4.5-Air (thinking): Path Length 239, average reasoning tokens per trial 12K tokens Qwen3-235B-A22B-Thinking (thinking): Path Length 804, average reasoning tokens per trial 14K tokens GLM-4.5 (thinking): Path Length 400, average reasoning tokens per trial 11K tokens gpt-oss-120b (medium): Path Length 1,144, average reasoning tokens per trial 8,592 tokens gpt-oss-20b (medium): Path Length 569, average reasoning tokens per trial 10K tokens GPT-5 (medium): Path Length 2,247, average reasoning tokens per trial 30K tokens GPT-5 mini (medium): Path Length 1,562, average reasoning tokens per trial 7,908 tokens GPT-5 nano (medium): Path Length 1,218, average reasoning tokens per trial 18K tokens Grok 4 Fast (thinking): Path Length 2,750, average reasoning tokens per trial 11K tokens Claude Sonnet 4.5 (16k): Path Length 2,665, average reasoning tokens per trial 10K tokens Gemini 3 Pro (high): Path Length 4,677, average reasoning tokens per trial 11K tokens Grok 4.1 Fast (thinking): Path Length 1,817, average reasoning tokens per trial 16K tokens Olmo 3 32B Think (thinking): Path Length 334, average reasoning tokens per trial 17K tokens Claude Opus 4.5 (16k): Path Length 5,513, average reasoning tokens per trial 15K tokens DeepSeek-V3.2 (thinking): Path Length 2,348, average reasoning tokens per trial 23K tokens DeepSeek-V3.2-Speciale (thinking): Path Length 5,073, average reasoning tokens per trial 49K tokens GPT-5.2 (medium): Path Length 3,592, average reasoning tokens per trial 27K tokens Gemini 3 Flash (high): Path Length 1,996, average reasoning tokens per trial 35K tokens Seed1.8 (thinking): Path Length 1,087, average reasoning tokens per trial 7,525 tokens Kimi K2.5 (thinking): Path Length 1,852, average reasoning tokens per trial 18K tokens Step 3.5 Flash (thinking): Path Length 2,410, average reasoning tokens per trial 76K tokens Claude Opus 4.6 (16k): Path Length 14K, average reasoning tokens per trial 14K tokens GLM-5 (thinking): Path Length 987, average reasoning tokens per trial 17K tokens MiniMax-M2.5 (thinking): Path Length 853, average reasoning tokens per trial 14K tokens Qwen3.5 397B-A17B (thinking): Path Length 1,293, average reasoning tokens per trial 25K tokens Seed2.0 Lite (0215) (thinking): Path Length 973, average reasoning tokens per trial 5,494 tokens Seed2.0 Mini (thinking): Path Length 667, average reasoning tokens per trial 10K tokens Seed2.0 Pro (thinking): Path Length 1,753, average reasoning tokens per trial 5,909 tokens Claude Sonnet 4.6 (16k): Path Length 12K, average reasoning tokens per trial 13K tokens Grok 4.20 (thinking): Path Length 3,856, average reasoning tokens per trial 9,169 tokens Gemini 3.1 Pro (high): Path Length 6,445, average reasoning tokens per trial 30K tokens Gemini 3.1 Pro (low): Path Length 3,514, average reasoning tokens per trial 2,760 tokens Qwen3.5-122B-A10B (thinking): Path Length 393, average reasoning tokens per trial 17K tokens Qwen3.5-35B-A3B (thinking): Path Length 447, average reasoning tokens per trial 13K tokens GPT-5.4 (medium): Path Length 5,335, average reasoning tokens per trial 23K tokens GPT-5.4 Mini (medium): Path Length 1,062, average reasoning tokens per trial 29K tokens GPT-5.4 Mini (minimal): Path Length 733, average reasoning tokens per trial 3,265 tokens GPT-5.4 Mini (low): Path Length 653, average reasoning tokens per trial 3,333 tokens GPT-5.4 Mini (high): Path Length 1,040, average reasoning tokens per trial 54K tokens GPT-5.4 Mini (xhigh): Path Length 1,141, average reasoning tokens per trial 55K tokens GPT-5.4 Nano (medium): Path Length 1,426, average reasoning tokens per trial 13K tokens Claude Opus 4.7 (xhigh): Path Length 14K, average reasoning tokens per trial 30K tokens GPT-5.5 (medium): Path Length 9,845, average reasoning tokens per trial 12K tokens DeepSeek-V4-Flash (high): Path Length 2,789, average reasoning tokens per trial 34K tokens DeepSeek-V4-Pro (high): Path Length 3,256, average reasoning tokens per trial 36K tokens Gemini 3.5 Flash (high): Path Length 4,123, average reasoning tokens per trial 41K tokens Claude Opus 4.8 (high): Path Length 9,463, average reasoning tokens per trial 9,465 tokens Claude Fable 5 (medium): Path Length 14K, average reasoning tokens per trial 12K tokens GLM-5.2 (high): Path Length 1,835, average reasoning tokens per trial 31K tokens Claude Sonnet 5 (high): Path Length 5,724, average reasoning tokens per trial 7,669 tokens Claude Sonnet 5 (medium): Path Length 4,488, average reasoning tokens per trial 3,290 tokens GPT-5.6 Luna (medium): Path Length 2,105, average reasoning tokens per trial 4,699 tokens GPT-5.6 Luna (low): Path Length 1,543, average reasoning tokens per trial 1,684 tokens GPT-5.6 Luna (high): Path Length 3,101, average reasoning tokens per trial 26K tokens GPT-5.6 Luna (xhigh): Path Length 4,181, average reasoning tokens per trial 58K tokens GPT-5.6 Sol (medium): Path Length 4,699, average reasoning tokens per trial 8,508 tokens GPT-5.6 Terra (medium): Path Length 3,303, average reasoning tokens per trial 6,769 tokens Kimi K3 (max): Path Length 11K, average reasoning tokens per trial 28K tokens Claude Opus 5 (medium): Path Length 8,113, average reasoning tokens per trial 4,262 tokens Claude Opus 5 (high): Path Length 15K, average reasoning tokens per trial 19K tokens Claude Fable 5.1 (medium): Path Length 6,797, average reasoning tokens per trial 1,493 tokens Claude Fable 5.1 (high): Path Length 8,171, average reasoning tokens per trial 3,154 tokens Claude Fable 5.1 (xhigh): Path Length 15K, average reasoning tokens per trial 56K tokens GPT-6 Astra (medium): Path Length 7,393, average reasoning tokens per trial 20K tokens GPT-6 Astra (low): Path Length 5,108, average reasoning tokens per trial 2,623 tokens GPT-6 Luna (medium): Path Length 3,172, average reasoning tokens per trial 5,719 tokens GPT-6 Luna (low): Path Length 1,640, average reasoning tokens per trial 1,230 tokens GPT-6 Luna (high): Path Length 3,864, average reasoning tokens per trial 15K tokens GPT-6 Sol (medium): Path Length 5,675, average reasoning tokens per trial 8,798 tokens GPT-6 Sol (low): Path Length 3,664, average reasoning tokens per trial 2,265 tokens GPT-6 Sol (high): Path Length 8,222, average reasoning tokens per trial 18K tokens Claude Sonnet 5.5 (high): Path Length 6,368, average reasoning tokens per trial 1,356 tokens GPT-6.1 Sol (medium): Path Length 6,006, average reasoning tokens per trial 12K tokens GPT-6.1 Sol (low): Path Length 3,876, average reasoning tokens per trial 1,966 tokens GPT-6.1 Sol (high): Path Length 8,736, average reasoning tokens per trial 37K tokens Gemini 4 Argon (high): Path Length 7,560, average reasoning tokens per trial 83K tokens Claude Haiku 5.5 (high): Path Length 7,608, average reasoning tokens per trial 5,855 tokens Claude Haiku 5.5 (xhigh): Path Length 13K, average reasoning tokens per trial 24K tokens Claude Haiku 5.5 (low): Path Length 867, average reasoning tokens per trial 169 tokens Claude Haiku 5.5 (medium): Path Length 4,888, average reasoning tokens per trial 2,364 tokens Muse Wonder (minimal): Path Length 4,764, average reasoning tokens per trial 2,151 tokens Muse Wonder (low): Path Length 8,833, average reasoning tokens per trial 6,547 tokens Muse Wonder (medium): Path Length 14K, average reasoning tokens per trial 12K tokens Muse Wonder (high): Path Length 18K, average reasoning tokens per trial 20K tokens Muse Wonder (xhigh): Path Length 23K, average reasoning tokens per trial 33K tokens Muse Wonder (max): Path Length 28K, average reasoning tokens per trial 54K tokens Muse Wonder Full (minimal): Path Length 6,005, average reasoning tokens per trial 2,175 tokens Muse Wonder Full (low): Path Length 13K, average reasoning tokens per trial 6,311 tokens Muse Wonder Full (medium): Path Length 18K, average reasoning tokens per trial 12K tokens DeepSeek V4.1 Flash 1017 (low): Path Length 12K, average reasoning tokens per trial 33K tokens DeepSeek V4.1 Flash 1017 (high): Path Length 15K, average reasoning tokens per trial 51K tokens DeepSeek V4.1 Flash 1017 (max): Path Length 21K, average reasoning tokens per trial 80K tokens Claude Fable 5.1 (low): Path Length 2,100, average reasoning tokens per trial 59.8 tokens Claude Opus 5.5 (high): Path Length 8,660, average reasoning tokens per trial 1,331 tokens Claude Opus 5.5 (xhigh): Path Length 24K, average reasoning tokens per trial 12K tokens Claude Opus 5.5 (medium): Path Length 7,150, average reasoning tokens per trial 932 tokens Claude Opus 5.5 (low): Path Length 3,142, average reasoning tokens per trial 144 tokens Claude Sonnet 5.5 (xhigh): Path Length 14K, average reasoning tokens per trial 6,171 tokens Muse Wonder Full (high): Path Length 25K, average reasoning tokens per trial 21K tokens Muse Wonder Full (xhigh): Path Length 32K, average reasoning tokens per trial 38K tokens Muse Wonder Full (max): Path Length 37K, average reasoning tokens per trial 46K tokens Muse Wonder Full (max)Muse Wonder Full (xhigh)Muse Wonder Full (high)Claude Opus 5.5 (xhigh)Claude Sonnet 5.5 (xhigh)Claude Opus 5.5 (high)Claude Opus 5.5 (medium)Claude Opus 5.5 (low)Claude Fable 5.1 (low) 0 10K 20K 30K 40K 101001,00010K100K Average reasoning tokens per trial (tokens), log scale, lower is better Path Length o1-mini (medium): Path Length 295, average reasoning tokens per trial 1,839 tokens o3-mini (medium): Path Length 1,366, average reasoning tokens per trial 7,625 tokens Grok-3 mini (medium): Path Length 849, average reasoning tokens per trial 3,451 tokens o3 (medium): Path Length 3,309, average reasoning tokens per trial 25K tokens o4-mini (medium): Path Length 800, average reasoning tokens per trial 7,145 tokens Qwen3-0.6B (16k): Path Length 31.7, average reasoning tokens per trial 4,777 tokens Qwen3-1.7B (16k): Path Length 133, average reasoning tokens per trial 7,405 tokens Qwen3-4B (16k): Path Length 263, average reasoning tokens per trial 8,386 tokens Claude Sonnet 4 (16k): Path Length 2,236, average reasoning tokens per trial 16K tokens DeepSeek-R1 (thinking): Path Length 289, average reasoning tokens per trial 4,948 tokens Gemini 2.5 Pro (16k): Path Length 556, average reasoning tokens per trial 13K tokens MiniMax-M1-80k (thinking): Path Length 195, average reasoning tokens per trial 20K tokens Gemini 2.5 Flash (16k): Path Length 362, average reasoning tokens per trial 20K tokens Grok 4 (medium): Path Length 4,042, average reasoning tokens per trial 13K tokens GLM-4.5-Air (thinking): Path Length 239, average reasoning tokens per trial 12K tokens Qwen3-235B-A22B-Thinking (thinking): Path Length 804, average reasoning tokens per trial 14K tokens GLM-4.5 (thinking): Path Length 400, average reasoning tokens per trial 11K tokens gpt-oss-120b (medium): Path Length 1,144, average reasoning tokens per trial 8,592 tokens gpt-oss-20b (medium): Path Length 569, average reasoning tokens per trial 10K tokens GPT-5 (medium): Path Length 2,247, average reasoning tokens per trial 30K tokens GPT-5 mini (medium): Path Length 1,562, average reasoning tokens per trial 7,908 tokens GPT-5 nano (medium): Path Length 1,218, average reasoning tokens per trial 18K tokens Grok 4 Fast (thinking): Path Length 2,750, average reasoning tokens per trial 11K tokens Claude Sonnet 4.5 (16k): Path Length 2,665, average reasoning tokens per trial 10K tokens Gemini 3 Pro (high): Path Length 4,677, average reasoning tokens per trial 11K tokens Grok 4.1 Fast (thinking): Path Length 1,817, average reasoning tokens per trial 16K tokens Olmo 3 32B Think (thinking): Path Length 334, average reasoning tokens per trial 17K tokens Claude Opus 4.5 (16k): Path Length 5,513, average reasoning tokens per trial 15K tokens DeepSeek-V3.2 (thinking): Path Length 2,348, average reasoning tokens per trial 23K tokens DeepSeek-V3.2-Speciale (thinking): Path Length 5,073, average reasoning tokens per trial 49K tokens GPT-5.2 (medium): Path Length 3,592, average reasoning tokens per trial 27K tokens Gemini 3 Flash (high): Path Length 1,996, average reasoning tokens per trial 35K tokens Seed1.8 (thinking): Path Length 1,087, average reasoning tokens per trial 7,525 tokens Kimi K2.5 (thinking): Path Length 1,852, average reasoning tokens per trial 18K tokens Step 3.5 Flash (thinking): Path Length 2,410, average reasoning tokens per trial 76K tokens Claude Opus 4.6 (16k): Path Length 14K, average reasoning tokens per trial 14K tokens GLM-5 (thinking): Path Length 987, average reasoning tokens per trial 17K tokens MiniMax-M2.5 (thinking): Path Length 853, average reasoning tokens per trial 14K tokens Qwen3.5 397B-A17B (thinking): Path Length 1,293, average reasoning tokens per trial 25K tokens Seed2.0 Lite (0215) (thinking): Path Length 973, average reasoning tokens per trial 5,494 tokens Seed2.0 Mini (thinking): Path Length 667, average reasoning tokens per trial 10K tokens Seed2.0 Pro (thinking): Path Length 1,753, average reasoning tokens per trial 5,909 tokens Claude Sonnet 4.6 (16k): Path Length 12K, average reasoning tokens per trial 13K tokens Grok 4.20 (thinking): Path Length 3,856, average reasoning tokens per trial 9,169 tokens Gemini 3.1 Pro (high): Path Length 6,445, average reasoning tokens per trial 30K tokens Gemini 3.1 Pro (low): Path Length 3,514, average reasoning tokens per trial 2,760 tokens Qwen3.5-122B-A10B (thinking): Path Length 393, average reasoning tokens per trial 17K tokens Qwen3.5-35B-A3B (thinking): Path Length 447, average reasoning tokens per trial 13K tokens GPT-5.4 (medium): Path Length 5,335, average reasoning tokens per trial 23K tokens GPT-5.4 Mini (medium): Path Length 1,062, average reasoning tokens per trial 29K tokens GPT-5.4 Mini (minimal): Path Length 733, average reasoning tokens per trial 3,265 tokens GPT-5.4 Mini (low): Path Length 653, average reasoning tokens per trial 3,333 tokens GPT-5.4 Mini (high): Path Length 1,040, average reasoning tokens per trial 54K tokens GPT-5.4 Mini (xhigh): Path Length 1,141, average reasoning tokens per trial 55K tokens GPT-5.4 Nano (medium): Path Length 1,426, average reasoning tokens per trial 13K tokens Claude Opus 4.7 (xhigh): Path Length 14K, average reasoning tokens per trial 30K tokens GPT-5.5 (medium): Path Length 9,845, average reasoning tokens per trial 12K tokens DeepSeek-V4-Flash (high): Path Length 2,789, average reasoning tokens per trial 34K tokens DeepSeek-V4-Pro (high): Path Length 3,256, average reasoning tokens per trial 36K tokens Gemini 3.5 Flash (high): Path Length 4,123, average reasoning tokens per trial 41K tokens Claude Opus 4.8 (high): Path Length 9,463, average reasoning tokens per trial 9,465 tokens Claude Fable 5 (medium): Path Length 14K, average reasoning tokens per trial 12K tokens GLM-5.2 (high): Path Length 1,835, average reasoning tokens per trial 31K tokens Claude Sonnet 5 (high): Path Length 5,724, average reasoning tokens per trial 7,669 tokens Claude Sonnet 5 (medium): Path Length 4,488, average reasoning tokens per trial 3,290 tokens GPT-5.6 Luna (medium): Path Length 2,105, average reasoning tokens per trial 4,699 tokens GPT-5.6 Luna (low): Path Length 1,543, average reasoning tokens per trial 1,684 tokens GPT-5.6 Luna (high): Path Length 3,101, average reasoning tokens per trial 26K tokens GPT-5.6 Luna (xhigh): Path Length 4,181, average reasoning tokens per trial 58K tokens GPT-5.6 Sol (medium): Path Length 4,699, average reasoning tokens per trial 8,508 tokens GPT-5.6 Terra (medium): Path Length 3,303, average reasoning tokens per trial 6,769 tokens Kimi K3 (max): Path Length 11K, average reasoning tokens per trial 28K tokens Claude Opus 5 (medium): Path Length 8,113, average reasoning tokens per trial 4,262 tokens Claude Opus 5 (high): Path Length 15K, average reasoning tokens per trial 19K tokens Claude Fable 5.1 (medium): Path Length 6,797, average reasoning tokens per trial 1,493 tokens Claude Fable 5.1 (high): Path Length 8,171, average reasoning tokens per trial 3,154 tokens Claude Fable 5.1 (xhigh): Path Length 15K, average reasoning tokens per trial 56K tokens GPT-6 Astra (medium): Path Length 7,393, average reasoning tokens per trial 20K tokens GPT-6 Astra (low): Path Length 5,108, average reasoning tokens per trial 2,623 tokens GPT-6 Luna (medium): Path Length 3,172, average reasoning tokens per trial 5,719 tokens GPT-6 Luna (low): Path Length 1,640, average reasoning tokens per trial 1,230 tokens GPT-6 Luna (high): Path Length 3,864, average reasoning tokens per trial 15K tokens GPT-6 Sol (medium): Path Length 5,675, average reasoning tokens per trial 8,798 tokens GPT-6 Sol (low): Path Length 3,664, average reasoning tokens per trial 2,265 tokens GPT-6 Sol (high): Path Length 8,222, average reasoning tokens per trial 18K tokens Claude Sonnet 5.5 (high): Path Length 6,368, average reasoning tokens per trial 1,356 tokens GPT-6.1 Sol (medium): Path Length 6,006, average reasoning tokens per trial 12K tokens GPT-6.1 Sol (low): Path Length 3,876, average reasoning tokens per trial 1,966 tokens GPT-6.1 Sol (high): Path Length 8,736, average reasoning tokens per trial 37K tokens Gemini 4 Argon (high): Path Length 7,560, average reasoning tokens per trial 83K tokens Claude Haiku 5.5 (high): Path Length 7,608, average reasoning tokens per trial 5,855 tokens Claude Haiku 5.5 (xhigh): Path Length 13K, average reasoning tokens per trial 24K tokens Claude Haiku 5.5 (low): Path Length 867, average reasoning tokens per trial 169 tokens Claude Haiku 5.5 (medium): Path Length 4,888, average reasoning tokens per trial 2,364 tokens Muse Wonder (minimal): Path Length 4,764, average reasoning tokens per trial 2,151 tokens Muse Wonder (low): Path Length 8,833, average reasoning tokens per trial 6,547 tokens Muse Wonder (medium): Path Length 14K, average reasoning tokens per trial 12K tokens Muse Wonder (high): Path Length 18K, average reasoning tokens per trial 20K tokens Muse Wonder (xhigh): Path Length 23K, average reasoning tokens per trial 33K tokens Muse Wonder (max): Path Length 28K, average reasoning tokens per trial 54K tokens Muse Wonder Full (minimal): Path Length 6,005, average reasoning tokens per trial 2,175 tokens Muse Wonder Full (low): Path Length 13K, average reasoning tokens per trial 6,311 tokens Muse Wonder Full (medium): Path Length 18K, average reasoning tokens per trial 12K tokens DeepSeek V4.1 Flash 1017 (low): Path Length 12K, average reasoning tokens per trial 33K tokens DeepSeek V4.1 Flash 1017 (high): Path Length 15K, average reasoning tokens per trial 51K tokens DeepSeek V4.1 Flash 1017 (max): Path Length 21K, average reasoning tokens per trial 80K tokens Claude Fable 5.1 (low): Path Length 2,100, average reasoning tokens per trial 59.8 tokens Claude Opus 5.5 (high): Path Length 8,660, average reasoning tokens per trial 1,331 tokens Claude Opus 5.5 (xhigh): Path Length 24K, average reasoning tokens per trial 12K tokens Claude Opus 5.5 (medium): Path Length 7,150, average reasoning tokens per trial 932 tokens Claude Opus 5.5 (low): Path Length 3,142, average reasoning tokens per trial 144 tokens Claude Sonnet 5.5 (xhigh): Path Length 14K, average reasoning tokens per trial 6,171 tokens Muse Wonder Full (high): Path Length 25K, average reasoning tokens per trial 21K tokens Muse Wonder Full (xhigh): Path Length 32K, average reasoning tokens per trial 38K tokens Muse Wonder Full (max): Path Length 37K, average reasoning tokens per trial 46K tokens Muse Wonder Full (max)Muse Wonder Full (xhigh)Muse Wonder Full (high)Claude Opus 5.5 (xhigh)Claude Sonnet 5.5 (xhigh)Claude Opus 5.5 (high)Claude Opus 5.5 (medium)Claude Opus 5.5 (low)Claude Fable 5.1 (low)
ModelSettingPath LengthAverage reasoning tokens per trial
Muse Wonder Full max 37,152 46,061 tokens
Muse Wonder Full xhigh 32,190 37,696 tokens
Muse Wonder max 27,973 53,973 tokens
Muse Wonder Full high 24,760 21,012 tokens
Claude Opus 5.5 xhigh 24,029 11,535 tokens
Muse Wonder xhigh 22,589 32,632 tokens
DeepSeek V4.1 Flash 1017 max 21,107 79,786 tokens
Muse Wonder Full medium 18,245 12,283 tokens
Muse Wonder high 18,145 19,832 tokens
Claude Opus 5 high 15,428 19,383 tokens
Claude Fable 5.1 xhigh 15,100 56,105 tokens
DeepSeek V4.1 Flash 1017 high 14,750 51,391 tokens
Claude Opus 4.7 xhigh 14,400 29,529 tokens
Claude Opus 4.6 16k 14,206 13,576 tokens
Muse Wonder medium 13,807 11,720 tokens
Claude Sonnet 5.5 xhigh 13,787 6,171 tokens
Claude Fable 5 medium 13,680 11,984 tokens
Muse Wonder Full low 13,338 6,311 tokens
Claude Haiku 5.5 xhigh 12,897 23,939 tokens
Claude Sonnet 4.6 16k 12,060 12,621 tokens
DeepSeek V4.1 Flash 1017 low 11,638 32,680 tokens
Kimi K3 max 10,521 27,549 tokens
GPT-5.5 medium 9,845 12,013 tokens
Claude Opus 4.8 high 9,463 9,465 tokens
Muse Wonder low 8,833 6,547 tokens
GPT-6.1 Sol high 8,736 37,385 tokens
Claude Opus 5.5 high 8,660 1,331 tokens
GPT-6 Sol high 8,222 17,542 tokens
Claude Fable 5.1 high 8,171 3,154 tokens
Claude Opus 5 medium 8,113 4,262 tokens
Claude Haiku 5.5 high 7,608 5,855 tokens
Gemini 4 Argon high 7,560 83,270 tokens
GPT-6 Astra medium 7,393 20,228 tokens
Claude Opus 5.5 medium 7,150 931.5 tokens
Claude Fable 5.1 medium 6,797 1,493 tokens
Gemini 3.1 Pro high 6,445 30,345 tokens
Claude Sonnet 5.5 high 6,368 1,356 tokens
GPT-6.1 Sol medium 6,006 11,919 tokens
Muse Wonder Full minimal 6,005 2,175 tokens
Claude Sonnet 5 high 5,724 7,669 tokens
GPT-6 Sol medium 5,675 8,798 tokens
Claude Opus 4.5 16k 5,513 14,529 tokens
GPT-5.4 medium 5,335 23,152 tokens
GPT-6 Astra low 5,108 2,623 tokens
DeepSeek-V3.2-Speciale thinking 5,073 49,153 tokens
Claude Haiku 5.5 medium 4,888 2,364 tokens
Muse Wonder minimal 4,764 2,151 tokens
GPT-5.6 Sol medium 4,699 8,508 tokens
Gemini 3 Pro high 4,677 11,285 tokens
Claude Sonnet 5 medium 4,488 3,290 tokens
GPT-5.6 Luna xhigh 4,181 57,646 tokens
Gemini 3.5 Flash high 4,123 40,755 tokens
Grok 4 medium 4,042 13,214 tokens
GPT-6.1 Sol low 3,876 1,966 tokens
GPT-6 Luna high 3,864 15,499 tokens
Grok 4.20 thinking 3,856 9,169 tokens
GPT-6 Sol low 3,664 2,265 tokens
GPT-5.2 medium 3,592 27,238 tokens
Gemini 3.1 Pro low 3,514 2,760 tokens
o3 medium 3,309 24,563 tokens
GPT-5.6 Terra medium 3,303 6,769 tokens
DeepSeek-V4-Pro high 3,256 36,027 tokens
GPT-6 Luna medium 3,172 5,719 tokens
Claude Opus 5.5 low 3,142 144.3 tokens
GPT-5.6 Luna high 3,101 25,684 tokens
DeepSeek-V4-Flash high 2,789 34,459 tokens
Grok 4 Fast thinking 2,750 10,527 tokens
Claude Sonnet 4.5 16k 2,665 10,320 tokens
Gemini 4 Argon low 2,547 0 tokens
Step 3.5 Flash thinking 2,410 75,625 tokens
DeepSeek-V3.2 thinking 2,348 23,452 tokens
GPT-5 medium 2,247 30,260 tokens
Claude Sonnet 4 16k 2,236 15,936 tokens
GPT-5.6 Luna medium 2,105 4,699 tokens
Claude Fable 5.1 low 2,100 59.8 tokens
Claude Sonnet 5.5 2,007 0 tokens
Gemini 3 Flash high 1,996 35,228 tokens
Claude Sonnet 5.5 medium 1,882 0 tokens
Kimi K2.5 thinking 1,852 18,397 tokens
GLM-5.2 high 1,835 31,112 tokens
Grok 4.1 Fast thinking 1,817 16,184 tokens
Gemini 4 Argon medium 1,807 0 tokens
Seed2.0 Pro thinking 1,753 5,909 tokens
GPT-6 Luna low 1,640 1,230 tokens
Claude Sonnet 5.5 low 1,575 0 tokens
GPT-5 mini medium 1,562 7,908 tokens
GPT-5.6 Luna low 1,543 1,684 tokens
GPT-5.4 Nano medium 1,426 12,780 tokens
o3-mini medium 1,366 7,625 tokens
Qwen3.5 397B-A17B thinking 1,293 25,037 tokens
GPT-5 nano medium 1,218 18,259 tokens
gpt-oss-120b medium 1,144 8,592 tokens
GPT-5.4 Mini xhigh 1,141 54,534 tokens
Seed1.8 thinking 1,087 7,525 tokens
Claude Haiku 5.5 1,066 0 tokens
GPT-5.4 Mini medium 1,062 28,604 tokens
GPT-6 Sol 1,043 0 tokens
GPT-5.4 Mini high 1,040 54,185 tokens
GLM-5 thinking 986.83 17,096 tokens
Seed2.0 Lite (0215) thinking 973 5,494 tokens
Claude Haiku 5.5 low 867.33 169 tokens
MiniMax-M2.5 thinking 852.5 13,970 tokens
Grok-3 mini medium 849.33 3,451 tokens
Qwen3-235B-A22B-Thinking Jul 2025 thinking 803.67 13,654 tokens
o4-mini medium 800 7,145 tokens
Grok 3 739 0 tokens
GPT-5.4 Mini minimal 733.33 3,265 tokens
Seed2.0 Mini thinking 666.67 10,228 tokens
GPT-5.4 Mini low 653 3,333 tokens
gpt-oss-20b medium 569 10,172 tokens
Gemini 2.5 Pro Jun 2025 16k 556 13,109 tokens
Qwen3.5-35B-A3B thinking 447.33 13,256 tokens
GPT-6 Luna 446.33 0 tokens
GLM-4.5 thinking 400.33 10,558 tokens
Qwen3.5-122B-A10B thinking 392.67 17,063 tokens
DeepSeek V4.1 Flash 1017 380 0 tokens
Gemini 2.5 Flash Jun 2025 16k 362.33 19,825 tokens
Olmo 3 32B Think thinking 333.67 17,460 tokens
o1-mini medium 295.33 1,839 tokens
DeepSeek-R1 May 2025 thinking 288.67 4,948 tokens
Qwen3-4B 16k 263.33 8,386 tokens
GLM-4.5-Air thinking 238.67 12,054 tokens
MiniMax-M1-80k thinking 194.67 20,498 tokens
Qwen3-1.7B 16k 133.33 7,405 tokens
Qwen3-14B 58.33 0 tokens
Qwen3-0.6B 16k 31.67 4,777 tokens

The best score, 37152.2, costs $1.91 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Mistral AI Google DeepMind Anthropic OpenAI Meta DeepSeek Cost frontier One model at each effort level
0.00 10000.0 20000.0 30000.0 40000.0 $0.00001 $0.0001 $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale Muse Wonder Full, max Muse Wonder Full, xhigh Muse Wonder, max Muse Wonder Full, high DeepSeek V4.1 Flash 1017, max DeepSeek V4.1 Flash 1017, high Claude Haiku 5.5, xhigh 0.00 10000.0 20000.0 30000.0 40000.0 $0.00001 $0.0001 $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale Muse Wonder Full, max Muse Wonder Full, xhigh

Source: Official LisanBench leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Muse Wonder Full max 37152.2 $1.91 47,781
Muse Wonder Full xhigh 32190.2 $1.57 39,264
Muse Wonder max 27973.3 $1.11 55,410
Muse Wonder Full high 24760.3 $0.89 22,241
Claude Opus 5.5 xhigh 24029.0 $0.26 13,113
Muse Wonder xhigh 22588.7 $0.68 33,787
DeepSeek V4.1 Flash 1017 max 21107.3 $0.05 80,912
Muse Wonder Full medium 18245.3 $0.53 13,217
Muse Wonder high 18145.3 $0.42 20,754
Claude Opus 5 high 15428.3 $0.51 20,223
Claude Fable 5.1 xhigh 15100.3 $2.91 58,167
DeepSeek V4.1 Flash 1017 high 14750.0 $0.03 52,131
Claude Opus 4.7 xhigh 14400.0 $0.76 30,193
Claude Opus 4.6 16k 14206.3 $0.36 14,230
Muse Wonder medium 13807.0 $0.25 12,440
Claude Sonnet 5.5 xhigh 13787.0 $0.07 7,055
Claude Fable 5 medium 13680.3 $0.64 12,704
Muse Wonder Full low 13338.3 $0.28 6,989
Claude Haiku 5.5 xhigh 12897.0 $0.01 25,066
Claude Sonnet 4.6 16k 12060.0 $0.20 13,196
DeepSeek V4.1 Flash 1017 low 11638.3 $0.02 33,234
Kimi K3 max 10520.7 $0.42 28,138
GPT-5.5 medium 9844.7 $0.39 12,805
Claude Opus 4.8 high 9463.0 $0.25 9,876
Muse Wonder low 8833.3 $0.14 7,041
GPT-6.1 Sol high 8736.0 $0.43 42,732
Claude Opus 5.5 high 8660.3 $0.04 1,907
GPT-6 Sol high 8222.0 $0.19 19,016
Claude Fable 5.1 high 8171.3 $0.21 4,038
Claude Opus 5 medium 8112.7 $0.12 4,689
Claude Haiku 5.5 high 7608.0 $0.003 6,462
Gemini 4 Argon high 7560.3 $1.20 97,220
GPT-6 Astra medium 7392.7 $1.25 24,942
Claude Opus 5.5 medium 7150.0 $0.03 1,431
Claude Fable 5.1 medium 6797.0 $0.11 2,164
Gemini 3.1 Pro high 6445.0 $0.38 31,533
Claude Sonnet 5.5 high 6368.0 $0.02 1,776
GPT-6.1 Sol medium 6006.3 $0.16 15,955
Muse Wonder Full minimal 6005.0 $0.10 2,537
Claude Sonnet 5 high 5724.0 $0.08 7,922
GPT-6 Sol medium 5675.0 $0.10 9,782
Claude Opus 4.5 16k 5513.0 $0.38 15,272
GPT-5.4 medium 5334.7 $0.35 23,538
GPT-6 Astra low 5108.3 $0.23 4,601
DeepSeek-V3.2-Speciale thinking 5072.7 $0.02 49,683
Claude Haiku 5.5 medium 4888.0 $0.001 2,795
Muse Wonder minimal 4764.0 $0.05 2,415
GPT-5.6 Sol medium 4699.0 $0.10 9,746
Gemini 3 Pro high 4677.0 $0.15 12,569
Claude Sonnet 5 medium 4488.3 $0.04 3,498
GPT-5.6 Luna xhigh 4180.7 $0.07 58,744
Gemini 3.5 Flash high 4123.0 $0.38 42,323
Grok 4 medium 4042.2 $0.18 12,082
GPT-6.1 Sol low 3875.7 $0.03 3,415
GPT-6 Luna high 3863.7 $0.008 15,970
Grok 4.20 thinking 3856.3 $0.06 9,737
GPT-6 Sol low 3664.3 $0.03 2,844
GPT-5.2 medium 3592.3 $0.35 24,456
Gemini 3.1 Pro low 3513.7 $0.04 3,001
o3 medium 3308.7 $0.20 25,082
GPT-5.6 Terra medium 3302.7 $0.11 7,213
DeepSeek-V4-Pro high 3256.3 $0.03 35,168
GPT-6 Luna medium 3172.3 $0.003 6,586
Claude Opus 5.5 low 3142.3 $0.01 484
GPT-5.6 Luna high 3101.3 $0.03 26,957
DeepSeek-V4-Flash high 2789.0 $0.01 34,918
Grok 4 Fast thinking 2749.6 10,929
Claude Sonnet 4.5 16k 2665.3 $0.16 10,769
Gemini 4 Argon low 2546.7 $0.36 32,876
Step 3.5 Flash thinking 2409.7 $0.02 75,860
DeepSeek-V3.2 thinking 2348.3 $0.01 24,399
GPT-5 medium 2247.0 $0.31 30,668
Claude Sonnet 4 16k 2236.0 $0.24 16,215
GPT-5.6 Luna medium 2104.7 $0.006 5,046
Claude Fable 5.1 low 2099.7 $0.03 518
Claude Sonnet 5.5 2006.7 $0.004 332
Gemini 3 Flash high 1996.0 $0.11 36,456
Claude Sonnet 5.5 medium 1882.3 $0.004 262
Kimi K2.5 thinking 1851.7 $0.05 18,869
GLM-5.2 high 1834.7 $0.14 32,287
Grok 4.1 Fast thinking 1817.3 $0.008 16,369
Gemini 4 Argon medium 1807.0 $0.40 36,716
Claude Opus 5 1765.3 $0.02 510
Seed2.0 Pro thinking 1753.0 $0.03 8,261
GPT-6 Luna low 1640.0 $0.0007 1,368
Claude Sonnet 5.5 low 1574.7 $0.003 205
GPT-5 mini medium 1561.7 $0.02 8,009
GPT-5.6 Luna low 1543.0 $0.003 2,731
GPT-5.4 Nano medium 1426.0 $0.02 12,969
o3-mini medium 1366.3 $0.03 7,759
Qwen3.5 397B-A17B thinking 1293.3 $0.09 26,052
GPT-5 nano medium 1217.7 $0.007 18,338
Claude Opus 4.8 1158.0 $0.01 317
gpt-oss-120b medium 1143.7 $0.002 8,712
GPT-5.4 Mini xhigh 1140.7 $0.25 54,712
Seed1.8 thinking 1087.3 $0.002 8,166
Claude Haiku 5.5 1065.7 $0.0002 273
GPT-5.4 Mini medium 1062.3 $0.13 28,747
GPT-6 Sol 1042.7 $0.006 544
GPT-5.4 Mini high 1039.7 $0.24 54,364
GPT-5.6 Sol 1038.3 $0.01 1,112
GPT-5.5 1018.3 $0.01 273
GLM-5 thinking 986.8 $0.07 22,258
Claude Sonnet 4.6 981.3 $0.03 2,142
Seed2.0 Lite (0215) thinking 973.0 $0.003 6,312
Claude Sonnet 5 883.3 $0.004 325
Claude Haiku 5.5 low 867.3 $0.0003 486
MiniMax-M2.5 thinking 852.5 $0.04 26,510
Grok-3 mini medium 849.3 $0.003 5,952
Qwen3-235B-A22B-Thinking Jul 2025 thinking 803.7 $0.01 14,708
o4-mini medium 800.0 $0.03 7,238
Grok 3 739.0 $0.02 1,321
GPT-5.4 Mini minimal 733.3 $0.02 3,333
Claude Opus 4.7 731.7 $0.009 247
Seed2.0 Mini thinking 666.7 $0.004 12,560
GPT-5.4 Mini low 653.0 $0.02 3,400
Claude Opus 4 598.0 $0.04 466
gpt-oss-20b medium 569.0 $0.001 10,235
Gemini 2.5 Pro Jun 2025 16k 556.0 $0.18 18,346
Claude 3.7 Sonnet 498.3 $0.01 750
Gemini 1.5 Pro Feb 2024 492.0 $0.002 306
GPT-5.6 Terra 478.3 $0.002 112
Claude Sonnet 4 462.0 $0.01 597
Claude 3.5 Sonnet (October 2024) 459.7 $0.006 343
Qwen3.5-35B-A3B thinking 447.3 $0.005 17,361
GPT-6 Luna 446.3 $0.001 2,303
GLM-4.5 thinking 400.3 $0.03 12,115
Claude Opus 4.1 394.7 $0.10 1,212
Qwen3.5-122B-A10B thinking 392.7 $0.02 20,325
DeepSeek V4.1 Flash 1017 380.0 $0.004 6,538
DeepSeek-V3 372.3 $0.001 2,303
DeepSeek-V4-Pro 367.7 $0.003 3,262
Claude 3.5 Sonnet 365.3 $0.02 1,199
Claude Opus 4.6 364.0 $0.02 729
Gemini 2.5 Flash Jun 2025 16k 362.3 $0.07 26,953
DeepSeek-V3.2 362.3 $0.004 8,680
GPT-5.6 Luna 348.7 $0.0005 326
Kimi K2 Jul 2025 347.3 $0.002 943
GPT-5.4 341.8 $0.002 83
Olmo 3 32B Think thinking 333.7 $0.01 20,877
GPT-4o Mar 2025 302.7 $0.005 242
GPT-4 Turbo Apr 2024 300.0 $0.02 626
o1-mini medium 295.3 $0.009 1,876
DeepSeek-R1 May 2025 thinking 288.7 $0.01 5,364
Gemini 2.5 Flash Jun 2025 267.7 $0.02 6,864
Qwen3-4B 16k 263.3 $0.003 12,625
Claude 3 Opus 249.0 $0.07 881
GLM-4.5-Air thinking 238.7 $0.01 12,499
Claude Sonnet 4.5 238.3 $0.07 4,903
GLM-5.2 231.7 $0.07 16,645
Claude Opus 4.5 226.7 $0.06 2,362
Gemini 1.5 Flash May 2024 218.0 $0.0002 599
Grok 4.1 Fast 218.0 $0.0005 728
Qwen3-Coder-480B-A35B 213.7 $0.003 3,477
Llama 3.1-405B 200.0 $0.005 856
Gemini 2.0 Flash Feb 2025 197.7 $0.0004 904
GLM-4.7 thinking 197.0 $0.12 56,457
MiniMax-M1-80k thinking 194.7 $0.04 17,971
GPT-3.5 Turbo Jan 2024 189.7 $0.0006 86
DeepSeek-V4-Flash 182.7 $0.001 4,183
Devstral Medium 180.3 $0.0009 374
ERNIE-4.5-300B-A47B 176.3 $0.002 1,554
Gemma 3 27B 175.3 $0.0002 1,191
GPT-4.1 172.0 $0.003 249
Gemini 2.0 Flash-Lite 157.3 $0.0008 2,719
GLM-4.6 thinking 157.0 $0.06 25,722
Llama 3.1-70B 151.3 $0.0002 174
Amazon Nova Pro 146.0 $0.02 4,855
Claude 3.5 Haiku 144.0 $0.001 286
Mistral Large 2 Nov 2024 138.0 $0.001 77
Claude Haiku 4.5 137.0 $0.09 17,085
Claude 3 Haiku 135.7 $0.0004 239
Qwen3-1.7B 16k 133.3 $0.002 8,330
Qwen3-235B-A22B-Instruct Jul 2025 133.3 $0.0003 572
Llama 4 Maverick 131.0 $0.002 3,937
Mistral Medium 3 130.7 $0.002 894
Llama 4 Scout 126.3 $0.0004 1,121
GPT-4.1 mini 126.3 $0.0003 111
Gemini 2.5 Flash-Lite Jun 2025 16k 122.3 $0.003 8,626
GPT-5.4 Mini 121.7 $0.0004 34
Gemini 2.5 Flash-Lite Jun 2025 120.0 $0.004 9,399
Gemma 3 12B 114.0 $0.0002 1,077
MiMo-V2-Flash thinking 104.3 6,285
GPT-4o mini 89.7 $0.0002 335
Amazon Nova Lite 84.0 $0.001 4,651
Gemini 1.5 Flash 8B 77.0 $0.00004 194
Amazon Nova Micro 76.0 $0.0003 1,866
Qwen3-30B-A3B-Instruct Jul 2025 74.7 $0.0003 809
Qwen3-32B 68.0 $0.002 3,826
Qwen3-14B 58.3 $0.003 15,306
GPT-4.1 nano 54.0 $0.0004 966
Mistral Small 3.2 52.7 $0.0005 2,288
Qwen3-8B 51.3 $0.0007 3,600
Gemma 3 4B 50.0 $0.0002 2,464
Mistral NeMo 48.7 $0.00001 135
Ministral 3 14B 41.7 $0.0008 3,843
Devstral Small 1.0 40.0 $0.0003 781
GPT-5.4 Nano 36.7 $0.0001 65
Ministral 3 8B Instruct 32.3 $0.0003 1,968
Qwen3-0.6B 16k 31.7 $0.001 4,793
Qwen3-4B 27.7 $0.0008 3,674
Ministral 3 3B Instruct 23.3 $0.0001 924
Qwen3-1.7B 20.3 $0.0007 2,985
Llama 3.1-8B 12.7 $0.0002 4,528
Llama 3.2 3B 8.00 $0.0003 752
Llama 3.2 1B 3.67 $0.0008 3,905
Qwen3-0.6B 0.67 $0.0001 342

9 models have held number one on the official leaderboard since its release in May 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI xAI Google DeepMind Anthropic Meta
Own leaderboard 9 leaders
Muse Wonder Full 37152.2

Source: Official LisanBench leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0.00 10000.0 20000.0 30000.0 40000.0 2024 2025 2026 0.00 10000.0 20000.0 30000.0 40000.0 2024 2025 2026

Source: Official LisanBench leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Games and long tasks

All 82 in Games and long tasks
  • ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6 Astra 99.95%
  • Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Each game scores in-game progress or outcomes, and the board ranks models by their average rank across the games. Gemini 2.5 Pro 3.90
  • MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. Seed2.0 Pro 49%
  • TextQuests Only in system cards Agents play 25 classic interactive fiction games. The score is harmful actions per 100 steps, where lower is better. The board also reports game progress. Llama 4 Scout 7.00
  • GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. GPT-6.1 Terra 76.8%
  • Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%