Benchmark Almanac

LLM Chess

GPT-6 Astra 1,614 leads since September 2026
Not in model cards

Models play chess in multi-turn dialogs against a random player or the Komodo Dragon engine. The board reports wins, Elo and game duration, which tracks instruction following.

Dragon was added as a stronger opponent for Elo anchoring once reasoning models beat the random player. Paper at NeurIPS 2025 FoRLM workshop. About 140 GitHub stars.

Made by
Maxim Saplin
Released
September 2024
Kind
Puzzles and chess
Level
Everyday
Scored
Ranked by matches
Answers
Agent task
Tasks
All public
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
November 2026
Scores refreshed
November 12th, 2026
133
Entries on its own leaderboard, 123 models we track

GPT-6 Astra leads with 1,614

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

133 entries. Scroll sideways for the rest.

  1. 1,614 GPT-6 Astra
  2. 1,612 GPT-6.1 Sol
  3. 1,609 Gemini 4 Argon
  4. 1,575 Claude Opus 5.5
  5. 1,550 GPT-5.6 Sol
  6. 1,546 Gemini 3.8 Flash
  7. 1,545 GPT-6 Sol
  8. 1,532 GPT-5.5
  9. 1,521 GPT-6.1 Terra
  10. 1,511 Gemini 3.1 Pro
  11. 1,322 GPT-5.4
  12. 1,308 Mercury 3
  13. 1,293 GPT-5.6 Terra
  14. 1,285 Claude Opus 5
  15. 1,219 GPT-6 Luna
  16. 1,195 Claude Sonnet 5.5
  17. 1,185 GPT-5.6 Luna
  18. 1,143 Claude Haiku 5.5
  19. 1,107 Gemini 3.7 Flash
  20. 1,094 Gemini 3.6 Flash
  21. 1,087 DeepSeek V4.1 Flash
  22. 1,086 GPT-5
  23. 1,085 GPT-5.2
  24. 1,053 GPT-5.3 Codex
  25. 1,034 Gemini 3 Pro
  26. 953 Gemini 3.5 Flash
  27. 948 GPT-5.1
  28. 846 Qwen 3.8 27B
  29. 843 Grok 4.20
  30. 822 Grok 4.7
  31. 806 Claude Sonnet 5
  32. 800 Grok 4.1 Fast
  33. 778 o3
  34. 759 GPT-5 mini
  35. 735 Grok 4 Fast
  36. 691 Claude Opus 4.7
  37. 683 GPT-5 nano
  38. 605 GLM-5.3
  39. 572 GLM-5.3-Flash
  40. 544 GPT-5.1-Codex-mini
  41. 541 unsloth/qwen3.8-27b@q2_k_xl [community quantization]
  42. 504 GPT-5.1-Codex
  43. 450 Claude Opus 4.8
  44. 446 Claude Opus 4.5
  45. 443 Grok 4.6
  46. 440 o4-mini
  47. 430 Grok-3 mini
  48. 416 GPT-5.4 Mini
  49. 403 Ternary-Bonsai-2-27B-gguf:TQ1-0
  50. 403 o3-mini
  51. 304 agents-a1
  52. 300 DeepSeek-R1
  53. 280 o1
  54. 270 Qwen3.6 27B
  55. 214 Muse Glimmer
  56. 207 GPT-5.4 Nano
  57. 198 Claude Opus 4.6
  58. 190 Claude Sonnet 4.6
  59. 189 DeepSeek-V3.2
  60. 179 Grok 4.3
  61. 142 GPT-5.2 Instant
  62. 142 non-gemini-25pro-t03-mini41-t00-mini41-t03
  63. 127 Claude Haiku 4.5
  64. 115 Claude Sonnet 4
  65. 113 Claude Sonnet 4.5
  66. 111 Kimi K2.5
  67. 46 o1-preview
  68. 29 Claude Opus 4
  69. 4 non-deepseek-r1-t03-mini41-t10-mini41-t03
  70. -1 Claude 3.7 Sonnet
  71. -26 Qwen3.5-35B-A3B
  72. -37 Qwen3-30B-A3B-Thinking
  73. -38 Claude Opus 4.1
  74. -38 Muse Ember
  75. -52 o1-mini
  76. -63 Composer 2
  77. -71 Gemini 2.5 Pro
  78. -87 GPT-5 Chat
  79. -98 gpt-4-32k
  80. -99 Claude 3.5 Sonnet (October 2024)
  81. -99 qwen
  82. -100 Llama 4 Maverick
  83. -104 GPT-4 Turbo
  84. -108 GPT-4o
  85. -111 GPT-4.1
  86. -111 GPT-4.5
  87. -120 gpt-4 [snapshot unspecified]
  88. -120 Claude 3.5 Haiku
  89. -128 Qwen3-Next-80B-A3B
  90. -131 GPT-5.1 Instant
  91. -133 Gemini 3.1 Flash-Lite
  92. -134 GPT-4o
  93. -134 Claude 3.5 Sonnet
  94. -137 Grok 3
  95. -140 Qwen3-235B-A22B-Instruct
  96. -144 Gemini 2.5 Pro
  97. -146 Claude 3 Opus
  98. -147 gemini-2.5-pro [snapshot unspecified]
  99. -158 GPT-4o
  100. -167 GPT-4.1 mini
  101. -179 Gemini 3 Flash
  102. -200 Nemotron 3 Nano 30B-A3B
  103. -200 Qwen3.5-9B
  104. -212 Composer 1.5
  105. -216 DeepSeek-R1
  106. -217 gpt-oss-120b
  107. -232 gpt-oss-20b
  108. -252 GPT-4o mini
  109. -278 Llama 3-70B
  110. -289 gemini-2.5-flash [snapshot unspecified]
  111. -299 GLM-5
  112. -311 Gemini 2.0 Flash
  113. -322 Claude 3 Sonnet
  114. -328 glm-4.7-flash
  115. -335 Grok-2
  116. -367 Gemini 1.5 Flash
  117. -402 Llama 4 Scout
  118. -412 Gemma 2 27B
  119. -458 K2-Think
  120. -546 Gemma 2 9B
  121. -558 Qwen3.5-4B
  122. -579 DeepSeek-V3
  123. -608 Llama 3.3 70B
  124. -616 Qwen Plus
  125. -637 Qwen3-4B
  126. -651 Gemini 1.5 Pro
  127. -672 Qwen2.5-72B
  128. -672 Gemini 2.0 Flash
  129. -689 Kimi K2
  130. -796 Llama 3.1-8B
  131. -812 Gemini 2.0 Flash-Lite
  132. -812 Composer 1
  133. -823 DeepSeek-V3
  1. 1 GPT-6 Astra high 1,614
  2. 2 GPT-6.1 Sol xhigh 1,612
  3. 3 Gemini 4 Argon high 1,609
  4. 4 Claude Opus 5.5 high 1,575
  5. 5 GPT-5.6 Sol xhigh 1,550
  6. 6 Gemini 3.8 Flash high 1,546
  7. 7 GPT-6 Sol xhigh 1,545
  8. 8 GPT-5.5 medium 1,532
  9. 9 GPT-6.1 Terra xhigh 1,521
  10. 10 Gemini 3.1 Pro high 1,511
  11. 11 GPT-5.4 high 1,322
  12. 12 Mercury 3 high 1,308
  13. 13 GPT-5.6 Terra xhigh 1,293
  14. 14 Claude Opus 5 high 1,285
  15. 15 GPT-6 Luna max 1,219
  16. 16 Claude Sonnet 5.5 high 1,195
  17. 17 GPT-5.6 Luna xhigh 1,185
  18. 18 Claude Haiku 5.5 high 1,143
  19. 19 Gemini 3.7 Flash high 1,107
  20. 20 Gemini 3.6 Flash high 1,094
  21. 21 DeepSeek V4.1 Flash high 1,087
  22. 22 GPT-5 medium 1,086
  23. 23 GPT-5.2 medium 1,085
  24. 24 GPT-5.3 Codex medium 1,053
  25. 25 Gemini 3 Pro high 1,034
  26. 26 Gemini 3.5 Flash medium 953
  27. 27 GPT-5.1 medium 948
  28. 28 Qwen 3.8 27B precision q4_k_m 846
  29. 29 Grok 4.20 reasoning 843
  30. 30 Grok 4.7 high 822
  31. 31 Claude Sonnet 5 high 806
  32. 32 Grok 4.1 Fast reasoning 800
  33. 33 o3 medium 778
  34. 34 GPT-5 mini high 759
  35. 35 Grok 4 Fast reasoning 735
  36. 36 Claude Opus 4.7 high 691
  37. 37 GPT-5 nano high 683
  38. 38 GLM-5.3 high 605
  39. 39 GLM-5.3-Flash high 572
  40. 40 GPT-5.1-Codex-mini high 544
  41. 41 unsloth/qwen3.8-27b@q2_k_xl [community quantization] 541
  42. 42 GPT-5.1-Codex low 504
  43. 43 Claude Opus 4.8 high 450
  44. 44 Claude Opus 4.5 thinking 16000 446
  45. 45 Grok 4.6 high 443
  46. 46 o4-mini high 440
  47. 47 Grok-3 mini high 430
  48. 48 GPT-5.4 Mini high 416
  49. 49 Ternary-Bonsai-2-27B-gguf:TQ1-0 high 403
  50. 50 o3-mini high 403
  51. 51 agents-a1 precision q4_k_m 304
  52. 52 DeepSeek-R1 May 2025 300
  53. 53 o1 medium 280
  54. 54 Qwen3.6 27B precision q4_k_s 270
  55. 55 Muse Glimmer precision q4_k_m 214
  56. 56 GPT-5.4 Nano high 207
  57. 57 Claude Opus 4.6 high 198
  58. 58 Claude Sonnet 4.6 high 190
  59. 59 DeepSeek-V3.2 non-reasoning 189
  60. 60 Grok 4.3 high 179
  61. 61 GPT-5.2 Instant 142
  62. 62 non-gemini-25pro-t03-mini41-t00-mini41-t03 142
  63. 63 Claude Haiku 4.5 thinking 16000 127
  64. 64 Claude Sonnet 4 thinking 16000 115
  65. 65 Claude Sonnet 4.5 thinking 16000 113
  66. 66 Kimi K2.5 111
  67. 67 o1-preview 46
  68. 68 Claude Opus 4 thinking 16000 29
  69. 69 non-deepseek-r1-t03-mini41-t10-mini41-t03 4
  70. 70 Claude 3.7 Sonnet thinking 10000 -1
  71. 71 Qwen3.5-35B-A3B precision q4_k_m -26
  72. 72 Qwen3-30B-A3B-Thinking Jul 2025, precision q4_k_m -37
  73. 73 Claude Opus 4.1 -38
  74. 74 Muse Ember xhigh -38
  75. 75 o1-mini -52
  76. 76 Composer 2 -63
  77. 77 Gemini 2.5 Pro May 2025 -71
  78. 78 GPT-5 Chat -87
  79. 79 gpt-4-32k -98
  80. 80 Claude 3.5 Sonnet (October 2024) -99
  81. 81 qwen max -99
  82. 82 Llama 4 Maverick -100
  83. 83 GPT-4 Turbo Apr 2024 -104
  84. 84 GPT-4o Nov 2024 -108
  85. 85 GPT-4.1 -111
  86. 86 GPT-4.5 -111
  87. 87 gpt-4 [snapshot unspecified] -120
  88. 88 Claude 3.5 Haiku -120
  89. 89 Qwen3-Next-80B-A3B -128
  90. 90 GPT-5.1 Instant -131
  91. 91 Gemini 3.1 Flash-Lite -133
  92. 92 GPT-4o Aug 2024 -134
  93. 93 Claude 3.5 Sonnet -134
  94. 94 Grok 3 -137
  95. 95 Qwen3-235B-A22B-Instruct Jul 2025 -140
  96. 96 Gemini 2.5 Pro Mar 2025 -144
  97. 97 Claude 3 Opus -146
  98. 98 gemini-2.5-pro [snapshot unspecified] -147
  99. 99 GPT-4o May 2024 -158
  100. 100 GPT-4.1 mini -167
  101. 101 Gemini 3 Flash -179
  102. 102 Nemotron 3 Nano 30B-A3B precision q3_k_l -200
  103. 103 Qwen3.5-9B precision bf16 -200
  104. 104 Composer 1.5 -212
  105. 105 DeepSeek-R1 -216
  106. 106 gpt-oss-120b medium -217
  107. 107 gpt-oss-20b medium -232
  108. 108 GPT-4o mini -252
  109. 109 Llama 3-70B -278
  110. 110 gemini-2.5-flash [snapshot unspecified] -289
  111. 111 GLM-5 -299
  112. 112 Gemini 2.0 Flash Feb 2025 -311
  113. 113 Claude 3 Sonnet -322
  114. 114 glm-4.7-flash precision q4 -328
  115. 115 Grok-2 Dec 2024 -335
  116. 116 Gemini 1.5 Flash May 2024 -367
  117. 117 Llama 4 Scout -402
  118. 118 Gemma 2 27B precision q6_k_l -412
  119. 119 K2-Think precision iq4_xs -458
  120. 120 Gemma 2 9B -546
  121. 121 Qwen3.5-4B precision bf16 -558
  122. 122 DeepSeek-V3 Mar 2025 -579
  123. 123 Llama 3.3 70B -608
  124. 124 Qwen Plus Jan 2025 -616
  125. 125 Qwen3-4B precision q8 -637
  126. 126 Gemini 1.5 Pro Sep 2024 -651
  127. 127 Qwen2.5-72B -672
  128. 128 Gemini 2.0 Flash Dec 2024 -672
  129. 129 Kimi K2 Jul 2025 -689
  130. 130 Llama 3.1-8B -796
  131. 131 Gemini 2.0 Flash-Lite -812
  132. 132 Composer 1 -812
  133. 133 DeepSeek-V3 -823

Source: Official LLM Chess leaderboard

#EntryScoreSettingDateSource
1 GPT-6 Astra 1,614 high Sep 2026 Official leaderboard
2 GPT-6.1 Sol 1,612 xhigh Sep 2026 Official leaderboard
3 Gemini 4 Argon 1,609 high Sep 2026 Official leaderboard
4 Claude Opus 5.5 1,575 high Sep 2026 Official leaderboard
5 GPT-5.6 Sol 1,550 xhigh Jul 2026 Official leaderboard
6 Gemini 3.8 Flash 1,546 high Sep 2026 Official leaderboard
7 GPT-6 Sol 1,545 xhigh Sep 2026 Official leaderboard
8 GPT-5.5 1,532 medium Apr 2026 Official leaderboard
9 GPT-6.1 Terra 1,521 xhigh Oct 2026 Official leaderboard
10 Gemini 3.1 Pro 1,511 high Feb 2026 Official leaderboard
11 GPT-5.4 1,322 high Mar 2026 Official leaderboard
12 Mercury 3 1,308 high Oct 2026 Official leaderboard
13 GPT-5.6 Terra 1,293 xhigh Jul 2026 Official leaderboard
14 Claude Opus 5 1,285 high Jul 2026 Official leaderboard
15 GPT-6 Luna 1,219 max Sep 2026 Official leaderboard
16 Claude Sonnet 5.5 1,195 high Sep 2026 Official leaderboard
17 GPT-5.6 Luna 1,185 xhigh Jul 2026 Official leaderboard
18 Claude Haiku 5.5 1,143 high Oct 2026 Official leaderboard
19 Gemini 3.7 Flash 1,107 high Aug 2026 Official leaderboard
20 Gemini 3.6 Flash 1,094 high Jul 2026 Official leaderboard
21 DeepSeek V4.1 Flash 1,087 high Sep 2026 Official leaderboard
22 GPT-5 1,086 medium Aug 2025 Official leaderboard
23 GPT-5.2 1,085 medium Dec 2025 Official leaderboard
24 GPT-5.3 Codex 1,053 medium Feb 2026 Official leaderboard
25 Gemini 3 Pro 1,034 high Nov 2025 Official leaderboard
26 Gemini 3.5 Flash 953 medium May 2026 Official leaderboard
27 GPT-5.1 948 medium Nov 2025 Official leaderboard
28 Qwen 3.8 27B 846 precision q4_k_m Aug 2026 Official leaderboard
29 Grok 4.20 843 reasoning Feb 2026 Official leaderboard
30 Grok 4.7 822 high Sep 2026 Official leaderboard
31 Claude Sonnet 5 806 high Jun 2026 Official leaderboard
32 Grok 4.1 Fast 800 reasoning Nov 2025 Official leaderboard
33 o3 778 medium Apr 2025 Official leaderboard
34 GPT-5 mini 759 high Aug 2025 Official leaderboard
35 Grok 4 Fast 735 reasoning Sep 2025 Official leaderboard
36 Claude Opus 4.7 691 high Apr 2026 Official leaderboard
37 GPT-5 nano 683 high Aug 2025 Official leaderboard
38 GLM-5.3 605 high Aug 2026 Official leaderboard
39 GLM-5.3-Flash 572 high Aug 2026 Official leaderboard
40 GPT-5.1-Codex-mini 544 high Nov 2025 Official leaderboard
41 unsloth/qwen3.8-27b@q2_k_xl [community quantization] 541 Aug 2026 Official leaderboard
42 GPT-5.1-Codex 504 low Nov 2025 Official leaderboard
43 Claude Opus 4.8 450 high May 2026 Official leaderboard
44 Claude Opus 4.5 446 thinking 16000 Nov 2025 Official leaderboard
45 Grok 4.6 443 high Aug 2026 Official leaderboard
46 o4-mini 440 high Apr 2025 Official leaderboard
47 Grok-3 mini 430 high Apr 2025 Official leaderboard
48 GPT-5.4 Mini 416 high Mar 2026 Official leaderboard
49 Ternary-Bonsai-2-27B-gguf:TQ1-0 403 high Sep 2026 Official leaderboard
50 o3-mini 403 high Jan 2025 Official leaderboard
51 agents-a1 304 precision q4_k_m Jul 2026 Official leaderboard
52 DeepSeek-R1 May 2025 300 May 2025 Official leaderboard
53 o1 280 medium Dec 2024 Official leaderboard
54 Qwen3.6 27B 270 precision q4_k_s Apr 2026 Official leaderboard
55 Muse Glimmer 214 precision q4_k_m Aug 2026 Official leaderboard
56 GPT-5.4 Nano 207 high Mar 2026 Official leaderboard
57 Claude Opus 4.6 198 high Feb 2026 Official leaderboard
58 Claude Sonnet 4.6 190 high Feb 2026 Official leaderboard
59 DeepSeek-V3.2 189 non-reasoning Dec 2025 Official leaderboard
60 Grok 4.3 179 high Apr 2026 Official leaderboard
61 GPT-5.2 Instant 142 Dec 2025 Official leaderboard
62 non-gemini-25pro-t03-mini41-t00-mini41-t03 142 Apr 2025 Official leaderboard
63 Claude Haiku 4.5 127 thinking 16000 Oct 2025 Official leaderboard
64 Claude Sonnet 4 115 thinking 16000 May 2025 Official leaderboard
65 Claude Sonnet 4.5 113 thinking 16000 Sep 2025 Official leaderboard
66 Kimi K2.5 111 Jan 2026 Official leaderboard
67 o1-preview 46 Sep 2024 Official leaderboard
68 Claude Opus 4 29 thinking 16000 May 2025 Official leaderboard
69 non-deepseek-r1-t03-mini41-t10-mini41-t03 4 Apr 2025 Official leaderboard
70 Claude 3.7 Sonnet -1 thinking 10000 Feb 2025 Official leaderboard
71 Qwen3.5-35B-A3B -26 precision q4_k_m Feb 2026 Official leaderboard
72 Qwen3-30B-A3B-Thinking Jul 2025 -37 precision q4_k_m Jul 2025 Official leaderboard
73 Claude Opus 4.1 -38 Aug 2025 Official leaderboard
74 Muse Ember -38 xhigh Nov 2026 Official leaderboard
75 o1-mini -52 Sep 2024 Official leaderboard
76 Composer 2 -63 Cursor CLI Mar 2026 Official leaderboard
77 Gemini 2.5 Pro May 2025 -71 May 2025 Official leaderboard
78 GPT-5 Chat -87 Aug 2025 Official leaderboard
79 gpt-4-32k -98 Jan 2025 Official leaderboard
80 Claude 3.5 Sonnet (October 2024) -99 Oct 2024 Official leaderboard
81 qwen -99 max Jan 2025 Official leaderboard
82 Llama 4 Maverick -100 Apr 2025 Official leaderboard
83 GPT-4 Turbo Apr 2024 -104 Apr 2024 Official leaderboard
84 GPT-4o Nov 2024 -108 Nov 2024 Official leaderboard
85 GPT-4.1 -111 Apr 2025 Official leaderboard
86 GPT-4.5 -111 Feb 2025 Official leaderboard
87 gpt-4 [snapshot unspecified] -120 Jan 2025 Official leaderboard
88 Claude 3.5 Haiku -120 Oct 2024 Official leaderboard
89 Qwen3-Next-80B-A3B -128 Sep 2025 Official leaderboard
90 GPT-5.1 Instant -131 Nov 2025 Official leaderboard
91 Gemini 3.1 Flash-Lite -133 Mar 2026 Official leaderboard
92 GPT-4o Aug 2024 -134 Aug 2024 Official leaderboard
93 Claude 3.5 Sonnet -134 Jun 2024 Official leaderboard
94 Grok 3 -137 Apr 2025 Official leaderboard
95 Qwen3-235B-A22B-Instruct Jul 2025 -140 Jul 2025 Official leaderboard
96 Gemini 2.5 Pro Mar 2025 -144 Mar 2025 Official leaderboard
97 Claude 3 Opus -146 Feb 2024 Official leaderboard
98 gemini-2.5-pro [snapshot unspecified] -147 Jun 2025 Official leaderboard
99 GPT-4o May 2024 -158 May 2024 Official leaderboard
100 GPT-4.1 mini -167 Apr 2025 Official leaderboard
101 Gemini 3 Flash -179 Dec 2025 Official leaderboard
102 Nemotron 3 Nano 30B-A3B -200 precision q3_k_l Dec 2025 Official leaderboard
103 Qwen3.5-9B -200 precision bf16 Feb 2026 Official leaderboard
104 Composer 1.5 -212 Cursor CLI Feb 2026 Official leaderboard
105 DeepSeek-R1 -216 Jan 2025 Official leaderboard
106 gpt-oss-120b -217 medium Aug 2025 Official leaderboard
107 gpt-oss-20b -232 medium Aug 2025 Official leaderboard
108 GPT-4o mini -252 Jul 2024 Official leaderboard
109 Llama 3-70B -278 Apr 2024 Official leaderboard
110 gemini-2.5-flash [snapshot unspecified] -289 Jun 2025 Official leaderboard
111 GLM-5 -299 Feb 2026 Official leaderboard
112 Gemini 2.0 Flash Feb 2025 -311 Feb 2025 Official leaderboard
113 Claude 3 Sonnet -322 Feb 2024 Official leaderboard
114 glm-4.7-flash -328 precision q4 Jan 2026 Official leaderboard
115 Grok-2 Dec 2024 -335 Dec 2024 Official leaderboard
116 Gemini 1.5 Flash May 2024 -367 May 2024 Official leaderboard
117 Llama 4 Scout -402 Apr 2025 Official leaderboard
118 Gemma 2 27B -412 precision q6_k_l Jun 2024 Official leaderboard
119 K2-Think -458 precision iq4_xs Sep 2025 Official leaderboard
120 Gemma 2 9B -546 Jun 2024 Official leaderboard
121 Qwen3.5-4B -558 precision bf16 Feb 2026 Official leaderboard
122 DeepSeek-V3 Mar 2025 -579 Mar 2025 Official leaderboard
123 Llama 3.3 70B -608 Dec 2024 Official leaderboard
124 Qwen Plus Jan 2025 -616 Jan 2025 Official leaderboard
125 Qwen3-4B -637 precision q8 Apr 2025 Official leaderboard
126 Gemini 1.5 Pro Sep 2024 -651 Sep 2024 Official leaderboard
127 Qwen2.5-72B -672 Sep 2024 Official leaderboard
128 Gemini 2.0 Flash Dec 2024 -672 Dec 2024 Official leaderboard
129 Kimi K2 Jul 2025 -689 Jul 2025 Official leaderboard
130 Llama 3.1-8B -796 Jul 2024 Official leaderboard
131 Gemini 2.0 Flash-Lite -812 Feb 2025 Official leaderboard
132 Composer 1 -812 Cursor CLI Oct 2025 Official leaderboard
133 DeepSeek-V3 -823 Dec 2024 Official leaderboard

Cost/Game versus Elo

The source compares game cost and Elo. Completion tokens remain a separate per-move metric. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

-1,000 0 1,000 2,000 $0.0001$0.001$0.01$0.1$1$10$100 Cost per game ($), log scale, lower is better Elo (elo) Claude 3 Opus: Elo -146 elo, cost per game $2.32 Claude 3 Sonnet: Elo -322 elo, cost per game $0.4 GPT-4 Turbo: Elo -104 elo, cost per game $0.85 Llama 3-70B: Elo -278 elo, cost per game $0.02 GPT-4o: Elo -158 elo, cost per game $0.27 Gemini 1.5 Flash: Elo -367 elo, cost per game $0.003 Claude 3.5 Sonnet: Elo -134 elo, cost per game $0.57 Gemma 2 27B (precision q6_k_l): Elo -412 elo, cost per game $0.02 GPT-4o mini: Elo -252 elo, cost per game $0.02 GPT-4o: Elo -134 elo, cost per game $0.21 o1-mini: Elo -52.4 elo, cost per game $2.41 o1-preview: Elo 46.3 elo, cost per game $22.6 Qwen2.5-72B: Elo -672 elo, cost per game $0.01 Gemini 1.5 Pro: Elo -651 elo, cost per game $0.15 Claude 3.5 Haiku: Elo -120 elo, cost per game $0.15 Claude 3.5 Sonnet (October 2024): Elo -98.7 elo, cost per game $0.56 GPT-4o: Elo -108 elo, cost per game $0.32 Gemini 2.0 Flash: Elo -672 elo, cost per game $0.01 Llama 3.3 70B: Elo -608 elo, cost per game $0.01 o1 (medium): Elo 280 elo, cost per game $21.9 o1 (low): Elo 154 elo, cost per game $13.7 DeepSeek-V3: Elo -823 elo, cost per game $0.03 DeepSeek-R1: Elo -216 elo, cost per game $0.94 Qwen Plus: Elo -616 elo, cost per game $0.07 o3-mini (high): Elo 403 elo, cost per game $3.62 o3-mini (medium): Elo 82.4 elo, cost per game $1.61 o3-mini (low): Elo -184 elo, cost per game $0.45 Gemini 2.0 Flash-Lite: Elo -812 elo, cost per game $0.007 Claude 3.7 Sonnet: Elo -76.4 elo, cost per game $0.9 Claude 3.7 Sonnet (thinking 10000): Elo -1 elo, cost per game $2.08 Claude 3.7 Sonnet (thinking 1024): Elo -36.1 elo, cost per game $1.32 Claude 3.7 Sonnet (thinking 2048): Elo -62.4 elo, cost per game $1.61 Claude 3.7 Sonnet (thinking 5000): Elo -36.1 elo, cost per game $1.92 GPT-4.5: Elo -111 elo, cost per game $6.48 DeepSeek-V3: Elo -579 elo, cost per game $0.05 Gemini 2.5 Pro: Elo -144 elo, cost per game $0.56 Llama 4 Maverick: Elo -99.7 elo, cost per game $0.06 Llama 4 Scout: Elo -402 elo, cost per game $0.02 GPT-4.1: Elo -111 elo, cost per game $0.14 GPT-4.1 mini: Elo -167 elo, cost per game $0.86 o3 (low): Elo 739 elo, cost per game $8.16 o3 (medium): Elo 778 elo, cost per game $10.8 o4-mini (high): Elo 440 elo, cost per game $2.71 o4-mini (low; precision PGN): Elo 110 elo, cost per game $0.58 o4-mini (medium): Elo 240 elo, cost per game $1.11 o4-mini (low): Elo 104 elo, cost per game $0.51 Qwen3-4B (precision q8): Elo -637 elo, cost per game $0.13 Gemini 2.5 Pro: Elo -70.7 elo, cost per game $0.58 Claude Opus 4 (thinking 16000): Elo 29.3 elo, cost per game $7.92 Claude Opus 4: Elo -10.9 elo, cost per game $3.28 Claude Sonnet 4 (thinking 16000): Elo 115 elo, cost per game $2.33 Claude Sonnet 4: Elo -10.9 elo, cost per game $0.64 DeepSeek-R1: Elo 300 elo, cost per game $1.91 Kimi K2: Elo -689 elo, cost per game $0.06 Qwen3-235B-A22B-Instruct: Elo -140 elo, cost per game $0.08 Qwen3-30B-A3B-Thinking (precision q4_k_m): Elo -37.3 elo, cost per game $0.55 Claude Opus 4.1: Elo -37.9 elo, cost per game $2.97 gpt-oss-120b (low): Elo -284 elo, cost per game $0.06 gpt-oss-120b (high): Elo -252 elo, cost per game $0.04 gpt-oss-120b (medium): Elo -217 elo, cost per game $0.05 gpt-oss-20b (high): Elo -258 elo, cost per game $0.02 gpt-oss-20b (low): Elo -300 elo, cost per game $0.02 GPT-5 (medium): Elo 1,086 elo, cost per game $6.36 GPT-5 (low): Elo 858 elo, cost per game $2.56 GPT-5 Chat: Elo -87.3 elo, cost per game $0.12 GPT-5 mini (high): Elo 759 elo, cost per game $1.28 GPT-5 mini (medium): Elo 349 elo, cost per game $0.44 GPT-5 mini (low): Elo -55.6 elo, cost per game $0.13 GPT-5 nano (high): Elo 683 elo, cost per game $0.49 GPT-5 nano (medium): Elo 413 elo, cost per game $0.19 K2-Think (precision iq4_xs): Elo -458 elo, cost per game $0.06 Qwen3-Next-80B-A3B: Elo -128 elo, cost per game $0.12 Grok 4 Fast (non-reasoning): Elo -191 elo, cost per game $0.02 Claude Sonnet 4.5: Elo -35.9 elo, cost per game $0.5 Claude Sonnet 4.5 (thinking 16000): Elo 113 elo, cost per game $2.2 Claude Haiku 4.5: Elo -42 elo, cost per game $0.19 Claude Haiku 4.5 (thinking 16000): Elo 127 elo, cost per game $1.15 GPT-5.1-Codex (low): Elo 504 elo, cost per game $1.38 GPT-5.1-Codex-mini (high): Elo 544 elo, cost per game $1.1 GPT-5.1 Instant: Elo -131 elo, cost per game $0.12 GPT-5.1 (medium): Elo 948 elo, cost per game $3.21 GPT-5.1 (low): Elo 615 elo, cost per game $0.91 Gemini 3 Pro (high): Elo 1,034 elo, cost per game $4.23 Claude Opus 4.5 (thinking 16000): Elo 446 elo, cost per game $7.34 Claude Opus 4.5: Elo 71.1 elo, cost per game $0.87 DeepSeek-V3.2 (non-reasoning): Elo 189 elo, cost per game $0.09 GPT-5.2 (low): Elo 783 elo, cost per game $1.3 GPT-5.2 (medium): Elo 1,085 elo, cost per game $3.12 GPT-5.2 Instant: Elo 142 elo, cost per game $0.77 Gemini 3 Flash: Elo -179 elo, cost per game $0.7 Kimi K2.5: Elo 111 elo, cost per game $0.97 Claude Opus 4.6 (high): Elo 198 elo, cost per game $8.76 GPT-5.3 Codex (medium): Elo 1,053 elo, cost per game $1.84 GLM-5: Elo -299 elo, cost per game $0.1 Claude Sonnet 4.6 (high): Elo 190 elo, cost per game $10.8 Claude Sonnet 4.6: Elo 111 elo, cost per game $0.63 Grok 4.20 (non-reasoning): Elo -125 elo, cost per game $0.11 Gemini 3.1 Pro (high): Elo 1,511 elo, cost per game $4.19 Qwen3.5-35B-A3B (precision q4_k_m): Elo -26 elo, cost per game $0.46 Gemini 3.1 Flash-Lite: Elo -133 elo, cost per game $0.02 GPT-5.4 (low): Elo 348 elo, cost per game $7.69 GPT-5.4 (medium): Elo 1,253 elo, cost per game $4.9 GPT-5.4 (high): Elo 1,322 elo, cost per game $5.12 GPT-5.4 Mini (low): Elo 176 elo, cost per game $0.26 GPT-5.4 Mini (high): Elo 416 elo, cost per game $3.23 GPT-5.4 Nano (high): Elo 207 elo, cost per game $0.74 Claude Opus 4.7 (high): Elo 691 elo, cost per game $5.57 Qwen3.6 27B (precision q4_k_s): Elo 270 elo, cost per game $1.46 GPT-5.5 (medium): Elo 1,532 elo, cost per game $6.57 GPT-5.5 (high): Elo 1,528 elo, cost per game $8.25 Grok 4.3 (high): Elo 179 elo, cost per game $0.19 Gemini 3.5 Flash (medium): Elo 953 elo, cost per game $2.52 Claude Opus 4.8 (high): Elo 450 elo, cost per game $4 Claude Sonnet 5 (high): Elo 806 elo, cost per game $2.77 Claude Sonnet 5: Elo 272 elo, cost per game $0.57 GPT-5.6 Luna (xhigh): Elo 1,185 elo, cost per game $2.51 GPT-5.6 Luna (medium): Elo 730 elo, cost per game $0.63 GPT-5.6 Luna (high): Elo 919 elo, cost per game $1.5 GPT-5.6 Sol (high): Elo 1,549 elo, cost per game $6.2 GPT-5.6 Sol (xhigh): Elo 1,550 elo, cost per game $8.23 GPT-5.6 Sol (medium): Elo 1,240 elo, cost per game $3.71 GPT-5.6 Terra (medium): Elo 936 elo, cost per game $1.32 GPT-5.6 Terra (high): Elo 1,211 elo, cost per game $2.66 GPT-5.6 Terra (xhigh): Elo 1,293 elo, cost per game $3.32 Gemini 3.6 Flash (low): Elo 498 elo, cost per game $0.85 Gemini 3.6 Flash (medium): Elo 921 elo, cost per game $1.61 Gemini 3.6 Flash (high): Elo 1,094 elo, cost per game $2.79 Claude Opus 5 (high): Elo 1,285 elo, cost per game $7.56 Muse Glimmer (precision q4_k_m): Elo 214 elo, cost per game $1.17 Grok 4.6 (high): Elo 443 elo, cost per game $0.47 Gemini 3.7 Flash (high): Elo 1,107 elo, cost per game $0.75 GLM-5.3 (high): Elo 605 elo, cost per game $1.12 Qwen 3.8 27B (precision q4_k_m): Elo 846 elo, cost per game $2.29 GLM-5.3-Flash (high): Elo 572 elo, cost per game $0.1 Gemini 3.8 Flash (high): Elo 1,546 elo, cost per game $2.19 GPT-6 Astra (xhigh): Elo 1,579 elo, cost per game $5.47 DeepSeek V4.1 Flash (high): Elo 1,087 elo, cost per game $0.74 Grok 4.7 (high): Elo 822 elo, cost per game $2.71 Claude Opus 5.5 (high): Elo 1,575 elo, cost per game $1.4 GPT-6 Sol (high): Elo 1,311 elo, cost per game $1.41 GPT-6 Sol (xhigh): Elo 1,545 elo, cost per game $3.41 Claude Sonnet 5.5 (high): Elo 1,195 elo, cost per game $1.53 GPT-6.1 Terra (xhigh): Elo 1,521 elo, cost per game $1.13 Muse Ember (max): Elo -337 elo, cost per game $0.21 Muse Ember (xhigh): Elo -38.3 elo, cost per game $0.44 unsloth/qwen3.8-27b@q2_k_xl [community quantization]: Elo 541 elo, cost per game $3.1. Not in our model list Ternary-Bonsai-2-27B-gguf:TQ1-0 (high): Elo 403 elo, cost per game $2.47. Not in our model list agents-a1 (precision q4_k_m): Elo 304 elo, cost per game $1.27. Not in our model list non-gemini-25pro-t03-mini41-t00-mini41-t03: Elo 142 elo, cost per game $0.09. Not in our model list non-deepseek-r1-t03-mini41-t10-mini41-t03: Elo 4.1 elo, cost per game $2.22. Not in our model list gpt-4-32k: Elo -97.8 elo, cost per game $2.23. Not in our model list qwen (max): Elo -98.9 elo, cost per game $0.13. Not in our model list gpt-4 [snapshot unspecified]: Elo -120 elo, cost per game $1.9. Not in our model list gemini-2.5-pro [snapshot unspecified]: Elo -147 elo, cost per game $0.35. Not in our model list gemini-2.5-flash [snapshot unspecified]: Elo -289 elo, cost per game $0.05. Not in our model list Gemma 2 9B: Elo -546 elo, cost per game $0.001 Llama 3.1-8B: Elo -796 elo, cost per game $0.0009 Gemini 2.0 Flash: Elo -311 elo, cost per game $0.01 gpt-oss-20b (medium): Elo -232 elo, cost per game $0.02 GPT-5 nano (low): Elo -117 elo, cost per game $0.03 Grok 4.1 Fast (non-reasoning): Elo -118 elo, cost per game $0.02 Nemotron 3 Nano 30B-A3B (precision q3_k_l): Elo -200 elo, cost per game $0.02 glm-4.7-flash (precision q4): Elo -328 elo, cost per game $0.003 Gemini 3.8 Flash (medium): Elo 1,527 elo, cost per game $0.86 GPT-6 Astra (high): Elo 1,614 elo, cost per game $4.64 GPT-6 Luna (xhigh): Elo 855 elo, cost per game $0.07 GPT-6 Luna (high): Elo 730 elo, cost per game $0.05 GPT-6 Luna (max): Elo 1,219 elo, cost per game $0.25 GPT-6.1 Sol (high): Elo 1,454 elo, cost per game $0.76 GPT-6.1 Sol (xhigh): Elo 1,612 elo, cost per game $1.65 Gemini 4 Argon (high): Elo 1,609 elo, cost per game $1.27 Claude Haiku 5.5 (high): Elo 1,143 elo, cost per game $0.08 Mercury 3 (high): Elo 1,308 elo, cost per game $0.48 GPT-6.1 Terra (high): Elo 1,391 elo, cost per game $0.57 GPT-6 Astra (high)GPT-6.1 Sol (xhigh)Gemini 4 Argon (high)Gemini 3.8 Flash (medium)GPT-6.1 Sol (high)GPT-6.1 Terra (high)Mercury 3 (high)GPT-6 Luna (max)Claude Haiku 5.5 (high)GPT-6 Luna (xhigh)GPT-6 Luna (high)GPT-5 nano (low)Grok 4.1 Fast (non-reasoning)Nemotron 3 Nano 30B-A3B (precision q3_k_l)gpt-oss-20b (medium)Gemini 2.0 Flashglm-4.7-flash (precision q4)Gemma 2 9BLlama 3.1-8B -1,000 0 1,000 2,000 $0.0001$0.001$0.01$0.1$1$10$100 Cost per game ($), log scale, lower is better Elo (elo) Claude 3 Opus: Elo -146 elo, cost per game $2.32 Claude 3 Sonnet: Elo -322 elo, cost per game $0.4 GPT-4 Turbo: Elo -104 elo, cost per game $0.85 Llama 3-70B: Elo -278 elo, cost per game $0.02 GPT-4o: Elo -158 elo, cost per game $0.27 Gemini 1.5 Flash: Elo -367 elo, cost per game $0.003 Claude 3.5 Sonnet: Elo -134 elo, cost per game $0.57 Gemma 2 27B (precision q6_k_l): Elo -412 elo, cost per game $0.02 GPT-4o mini: Elo -252 elo, cost per game $0.02 GPT-4o: Elo -134 elo, cost per game $0.21 o1-mini: Elo -52.4 elo, cost per game $2.41 o1-preview: Elo 46.3 elo, cost per game $22.6 Qwen2.5-72B: Elo -672 elo, cost per game $0.01 Gemini 1.5 Pro: Elo -651 elo, cost per game $0.15 Claude 3.5 Haiku: Elo -120 elo, cost per game $0.15 Claude 3.5 Sonnet (October 2024): Elo -98.7 elo, cost per game $0.56 GPT-4o: Elo -108 elo, cost per game $0.32 Gemini 2.0 Flash: Elo -672 elo, cost per game $0.01 Llama 3.3 70B: Elo -608 elo, cost per game $0.01 o1 (medium): Elo 280 elo, cost per game $21.9 o1 (low): Elo 154 elo, cost per game $13.7 DeepSeek-V3: Elo -823 elo, cost per game $0.03 DeepSeek-R1: Elo -216 elo, cost per game $0.94 Qwen Plus: Elo -616 elo, cost per game $0.07 o3-mini (high): Elo 403 elo, cost per game $3.62 o3-mini (medium): Elo 82.4 elo, cost per game $1.61 o3-mini (low): Elo -184 elo, cost per game $0.45 Gemini 2.0 Flash-Lite: Elo -812 elo, cost per game $0.007 Claude 3.7 Sonnet: Elo -76.4 elo, cost per game $0.9 Claude 3.7 Sonnet (thinking 10000): Elo -1 elo, cost per game $2.08 Claude 3.7 Sonnet (thinking 1024): Elo -36.1 elo, cost per game $1.32 Claude 3.7 Sonnet (thinking 2048): Elo -62.4 elo, cost per game $1.61 Claude 3.7 Sonnet (thinking 5000): Elo -36.1 elo, cost per game $1.92 GPT-4.5: Elo -111 elo, cost per game $6.48 DeepSeek-V3: Elo -579 elo, cost per game $0.05 Gemini 2.5 Pro: Elo -144 elo, cost per game $0.56 Llama 4 Maverick: Elo -99.7 elo, cost per game $0.06 Llama 4 Scout: Elo -402 elo, cost per game $0.02 GPT-4.1: Elo -111 elo, cost per game $0.14 GPT-4.1 mini: Elo -167 elo, cost per game $0.86 o3 (low): Elo 739 elo, cost per game $8.16 o3 (medium): Elo 778 elo, cost per game $10.8 o4-mini (high): Elo 440 elo, cost per game $2.71 o4-mini (low; precision PGN): Elo 110 elo, cost per game $0.58 o4-mini (medium): Elo 240 elo, cost per game $1.11 o4-mini (low): Elo 104 elo, cost per game $0.51 Qwen3-4B (precision q8): Elo -637 elo, cost per game $0.13 Gemini 2.5 Pro: Elo -70.7 elo, cost per game $0.58 Claude Opus 4 (thinking 16000): Elo 29.3 elo, cost per game $7.92 Claude Opus 4: Elo -10.9 elo, cost per game $3.28 Claude Sonnet 4 (thinking 16000): Elo 115 elo, cost per game $2.33 Claude Sonnet 4: Elo -10.9 elo, cost per game $0.64 DeepSeek-R1: Elo 300 elo, cost per game $1.91 Kimi K2: Elo -689 elo, cost per game $0.06 Qwen3-235B-A22B-Instruct: Elo -140 elo, cost per game $0.08 Qwen3-30B-A3B-Thinking (precision q4_k_m): Elo -37.3 elo, cost per game $0.55 Claude Opus 4.1: Elo -37.9 elo, cost per game $2.97 gpt-oss-120b (low): Elo -284 elo, cost per game $0.06 gpt-oss-120b (high): Elo -252 elo, cost per game $0.04 gpt-oss-120b (medium): Elo -217 elo, cost per game $0.05 gpt-oss-20b (high): Elo -258 elo, cost per game $0.02 gpt-oss-20b (low): Elo -300 elo, cost per game $0.02 GPT-5 (medium): Elo 1,086 elo, cost per game $6.36 GPT-5 (low): Elo 858 elo, cost per game $2.56 GPT-5 Chat: Elo -87.3 elo, cost per game $0.12 GPT-5 mini (high): Elo 759 elo, cost per game $1.28 GPT-5 mini (medium): Elo 349 elo, cost per game $0.44 GPT-5 mini (low): Elo -55.6 elo, cost per game $0.13 GPT-5 nano (high): Elo 683 elo, cost per game $0.49 GPT-5 nano (medium): Elo 413 elo, cost per game $0.19 K2-Think (precision iq4_xs): Elo -458 elo, cost per game $0.06 Qwen3-Next-80B-A3B: Elo -128 elo, cost per game $0.12 Grok 4 Fast (non-reasoning): Elo -191 elo, cost per game $0.02 Claude Sonnet 4.5: Elo -35.9 elo, cost per game $0.5 Claude Sonnet 4.5 (thinking 16000): Elo 113 elo, cost per game $2.2 Claude Haiku 4.5: Elo -42 elo, cost per game $0.19 Claude Haiku 4.5 (thinking 16000): Elo 127 elo, cost per game $1.15 GPT-5.1-Codex (low): Elo 504 elo, cost per game $1.38 GPT-5.1-Codex-mini (high): Elo 544 elo, cost per game $1.1 GPT-5.1 Instant: Elo -131 elo, cost per game $0.12 GPT-5.1 (medium): Elo 948 elo, cost per game $3.21 GPT-5.1 (low): Elo 615 elo, cost per game $0.91 Gemini 3 Pro (high): Elo 1,034 elo, cost per game $4.23 Claude Opus 4.5 (thinking 16000): Elo 446 elo, cost per game $7.34 Claude Opus 4.5: Elo 71.1 elo, cost per game $0.87 DeepSeek-V3.2 (non-reasoning): Elo 189 elo, cost per game $0.09 GPT-5.2 (low): Elo 783 elo, cost per game $1.3 GPT-5.2 (medium): Elo 1,085 elo, cost per game $3.12 GPT-5.2 Instant: Elo 142 elo, cost per game $0.77 Gemini 3 Flash: Elo -179 elo, cost per game $0.7 Kimi K2.5: Elo 111 elo, cost per game $0.97 Claude Opus 4.6 (high): Elo 198 elo, cost per game $8.76 GPT-5.3 Codex (medium): Elo 1,053 elo, cost per game $1.84 GLM-5: Elo -299 elo, cost per game $0.1 Claude Sonnet 4.6 (high): Elo 190 elo, cost per game $10.8 Claude Sonnet 4.6: Elo 111 elo, cost per game $0.63 Grok 4.20 (non-reasoning): Elo -125 elo, cost per game $0.11 Gemini 3.1 Pro (high): Elo 1,511 elo, cost per game $4.19 Qwen3.5-35B-A3B (precision q4_k_m): Elo -26 elo, cost per game $0.46 Gemini 3.1 Flash-Lite: Elo -133 elo, cost per game $0.02 GPT-5.4 (low): Elo 348 elo, cost per game $7.69 GPT-5.4 (medium): Elo 1,253 elo, cost per game $4.9 GPT-5.4 (high): Elo 1,322 elo, cost per game $5.12 GPT-5.4 Mini (low): Elo 176 elo, cost per game $0.26 GPT-5.4 Mini (high): Elo 416 elo, cost per game $3.23 GPT-5.4 Nano (high): Elo 207 elo, cost per game $0.74 Claude Opus 4.7 (high): Elo 691 elo, cost per game $5.57 Qwen3.6 27B (precision q4_k_s): Elo 270 elo, cost per game $1.46 GPT-5.5 (medium): Elo 1,532 elo, cost per game $6.57 GPT-5.5 (high): Elo 1,528 elo, cost per game $8.25 Grok 4.3 (high): Elo 179 elo, cost per game $0.19 Gemini 3.5 Flash (medium): Elo 953 elo, cost per game $2.52 Claude Opus 4.8 (high): Elo 450 elo, cost per game $4 Claude Sonnet 5 (high): Elo 806 elo, cost per game $2.77 Claude Sonnet 5: Elo 272 elo, cost per game $0.57 GPT-5.6 Luna (xhigh): Elo 1,185 elo, cost per game $2.51 GPT-5.6 Luna (medium): Elo 730 elo, cost per game $0.63 GPT-5.6 Luna (high): Elo 919 elo, cost per game $1.5 GPT-5.6 Sol (high): Elo 1,549 elo, cost per game $6.2 GPT-5.6 Sol (xhigh): Elo 1,550 elo, cost per game $8.23 GPT-5.6 Sol (medium): Elo 1,240 elo, cost per game $3.71 GPT-5.6 Terra (medium): Elo 936 elo, cost per game $1.32 GPT-5.6 Terra (high): Elo 1,211 elo, cost per game $2.66 GPT-5.6 Terra (xhigh): Elo 1,293 elo, cost per game $3.32 Gemini 3.6 Flash (low): Elo 498 elo, cost per game $0.85 Gemini 3.6 Flash (medium): Elo 921 elo, cost per game $1.61 Gemini 3.6 Flash (high): Elo 1,094 elo, cost per game $2.79 Claude Opus 5 (high): Elo 1,285 elo, cost per game $7.56 Muse Glimmer (precision q4_k_m): Elo 214 elo, cost per game $1.17 Grok 4.6 (high): Elo 443 elo, cost per game $0.47 Gemini 3.7 Flash (high): Elo 1,107 elo, cost per game $0.75 GLM-5.3 (high): Elo 605 elo, cost per game $1.12 Qwen 3.8 27B (precision q4_k_m): Elo 846 elo, cost per game $2.29 GLM-5.3-Flash (high): Elo 572 elo, cost per game $0.1 Gemini 3.8 Flash (high): Elo 1,546 elo, cost per game $2.19 GPT-6 Astra (xhigh): Elo 1,579 elo, cost per game $5.47 DeepSeek V4.1 Flash (high): Elo 1,087 elo, cost per game $0.74 Grok 4.7 (high): Elo 822 elo, cost per game $2.71 Claude Opus 5.5 (high): Elo 1,575 elo, cost per game $1.4 GPT-6 Sol (high): Elo 1,311 elo, cost per game $1.41 GPT-6 Sol (xhigh): Elo 1,545 elo, cost per game $3.41 Claude Sonnet 5.5 (high): Elo 1,195 elo, cost per game $1.53 GPT-6.1 Terra (xhigh): Elo 1,521 elo, cost per game $1.13 Muse Ember (max): Elo -337 elo, cost per game $0.21 Muse Ember (xhigh): Elo -38.3 elo, cost per game $0.44 unsloth/qwen3.8-27b@q2_k_xl [community quantization]: Elo 541 elo, cost per game $3.1. Not in our model list Ternary-Bonsai-2-27B-gguf:TQ1-0 (high): Elo 403 elo, cost per game $2.47. Not in our model list agents-a1 (precision q4_k_m): Elo 304 elo, cost per game $1.27. Not in our model list non-gemini-25pro-t03-mini41-t00-mini41-t03: Elo 142 elo, cost per game $0.09. Not in our model list non-deepseek-r1-t03-mini41-t10-mini41-t03: Elo 4.1 elo, cost per game $2.22. Not in our model list gpt-4-32k: Elo -97.8 elo, cost per game $2.23. Not in our model list qwen (max): Elo -98.9 elo, cost per game $0.13. Not in our model list gpt-4 [snapshot unspecified]: Elo -120 elo, cost per game $1.9. Not in our model list gemini-2.5-pro [snapshot unspecified]: Elo -147 elo, cost per game $0.35. Not in our model list gemini-2.5-flash [snapshot unspecified]: Elo -289 elo, cost per game $0.05. Not in our model list Gemma 2 9B: Elo -546 elo, cost per game $0.001 Llama 3.1-8B: Elo -796 elo, cost per game $0.0009 Gemini 2.0 Flash: Elo -311 elo, cost per game $0.01 gpt-oss-20b (medium): Elo -232 elo, cost per game $0.02 GPT-5 nano (low): Elo -117 elo, cost per game $0.03 Grok 4.1 Fast (non-reasoning): Elo -118 elo, cost per game $0.02 Nemotron 3 Nano 30B-A3B (precision q3_k_l): Elo -200 elo, cost per game $0.02 glm-4.7-flash (precision q4): Elo -328 elo, cost per game $0.003 Gemini 3.8 Flash (medium): Elo 1,527 elo, cost per game $0.86 GPT-6 Astra (high): Elo 1,614 elo, cost per game $4.64 GPT-6 Luna (xhigh): Elo 855 elo, cost per game $0.07 GPT-6 Luna (high): Elo 730 elo, cost per game $0.05 GPT-6 Luna (max): Elo 1,219 elo, cost per game $0.25 GPT-6.1 Sol (high): Elo 1,454 elo, cost per game $0.76 GPT-6.1 Sol (xhigh): Elo 1,612 elo, cost per game $1.65 Gemini 4 Argon (high): Elo 1,609 elo, cost per game $1.27 Claude Haiku 5.5 (high): Elo 1,143 elo, cost per game $0.08 Mercury 3 (high): Elo 1,308 elo, cost per game $0.48 GPT-6.1 Terra (high): Elo 1,391 elo, cost per game $0.57 GPT-6 Astra (high)GPT-6.1 Sol (xhigh)Gemini 4 Argon (high)Gemini 3.8 Flash (medium)GPT-6.1 Sol (high)GPT-6.1 Terra (high)Mercury 3 (high)GPT-6 Luna (max)Claude Haiku 5.5 (high)GPT-6 Luna (xhigh)GPT-6 Luna (high)GPT-5 nano (low)Grok 4.1 Fast (non-reasoning)Nemotron 3 Nano 30B-A3B (precision q3_k_l)gpt-oss-20b (medium)Gemini 2.0 Flashglm-4.7-flash (precision q4)Gemma 2 9BLlama 3.1-8B
ModelSettingEloCost per game
GPT-6 Astra high 1,614 elo $4.64
GPT-6.1 Sol xhigh 1,612 elo $1.65
Gemini 4 Argon high 1,609 elo $1.27
GPT-6 Astra xhigh 1,579 elo $5.47
Claude Opus 5.5 high 1,575 elo $1.4
GPT-5.6 Sol xhigh 1,550 elo $8.23
GPT-5.6 Sol high 1,549 elo $6.2
Gemini 3.8 Flash high 1,546 elo $2.19
GPT-6 Sol xhigh 1,545 elo $3.41
GPT-5.5 medium 1,532 elo $6.57
GPT-5.5 high 1,528 elo $8.25
Gemini 3.8 Flash medium 1,527 elo $0.86
GPT-6.1 Terra xhigh 1,521 elo $1.13
Gemini 3.1 Pro high 1,511 elo $4.19
GPT-6.1 Sol high 1,454 elo $0.76
GPT-6.1 Terra high 1,391 elo $0.57
GPT-5.4 high 1,322 elo $5.12
GPT-6 Sol high 1,311 elo $1.41
Mercury 3 high 1,308 elo $0.48
GPT-5.6 Terra xhigh 1,293 elo $3.32
Claude Opus 5 high 1,285 elo $7.56
GPT-5.4 medium 1,253 elo $4.9
GPT-5.6 Sol medium 1,240 elo $3.71
GPT-6 Luna max 1,219 elo $0.25
GPT-5.6 Terra high 1,211 elo $2.66
Claude Sonnet 5.5 high 1,195 elo $1.53
GPT-5.6 Luna xhigh 1,185 elo $2.51
Claude Haiku 5.5 high 1,143 elo $0.08
Gemini 3.7 Flash high 1,107 elo $0.75
Gemini 3.6 Flash high 1,094 elo $2.79
DeepSeek V4.1 Flash high 1,087 elo $0.74
GPT-5 medium 1,086 elo $6.36
GPT-5.2 medium 1,085 elo $3.12
GPT-5.3 Codex medium 1,053 elo $1.84
Gemini 3 Pro high 1,034 elo $4.23
Gemini 3.5 Flash medium 952.9 elo $2.52
GPT-5.1 medium 947.6 elo $3.21
GPT-5.6 Terra medium 935.9 elo $1.32
Gemini 3.6 Flash medium 921.3 elo $1.61
GPT-5.6 Luna high 919.4 elo $1.5
GPT-5 low 858.2 elo $2.57
GPT-6 Luna xhigh 855 elo $0.07
Qwen 3.8 27B precision q4_k_m 845.6 elo $2.29
Grok 4.7 high 822.2 elo $2.71
Claude Sonnet 5 high 805.9 elo $2.77
GPT-5.2 low 782.5 elo $1.3
o3 medium 777.6 elo $10.75
GPT-5 mini high 759 elo $1.28
o3 low 738.6 elo $8.16
GPT-6 Luna high 730.3 elo $0.05
GPT-5.6 Luna medium 730.1 elo $0.63
Claude Opus 4.7 high 691.1 elo $5.57
GPT-5 nano high 683 elo $0.49
GPT-5.1 low 614.7 elo $0.91
GLM-5.3 high 605.2 elo $1.12
GLM-5.3-Flash high 571.7 elo $0.1
GPT-5.1-Codex-mini high 544 elo $1.1
unsloth/qwen3.8-27b@q2_k_xl [community quantization] 541.1 elo $3.1
GPT-5.1-Codex low 504.3 elo $1.38
Gemini 3.6 Flash low 498.4 elo $0.85
Claude Opus 4.8 high 449.8 elo $4
Claude Opus 4.5 thinking 16000 446.1 elo $7.34
Grok 4.6 high 442.7 elo $0.47
o4-mini high 440 elo $2.72
GPT-5.4 Mini high 416.4 elo $3.23
GPT-5 nano medium 413.3 elo $0.19
Ternary-Bonsai-2-27B-gguf:TQ1-0 high 403.3 elo $2.47
o3-mini high 403.1 elo $3.62
GPT-5 mini medium 348.5 elo $0.44
GPT-5.4 low 347.9 elo $7.69
agents-a1 precision q4_k_m 303.8 elo $1.27
DeepSeek-R1 May 2025 299.7 elo $1.91
o1 medium 280.1 elo $21.9
Claude Sonnet 5 272.2 elo $0.57
Qwen3.6 27B precision q4_k_s 270 elo $1.46
o4-mini medium 240.3 elo $1.11
Muse Glimmer precision q4_k_m 214.4 elo $1.17
GPT-5.4 Nano high 207.3 elo $0.74
Claude Opus 4.6 high 198.2 elo $8.76
Claude Sonnet 4.6 high 189.7 elo $10.8
DeepSeek-V3.2 non-reasoning 189.2 elo $0.09
Grok 4.3 high 178.6 elo $0.19
GPT-5.4 Mini low 176.2 elo $0.26
o1 low 153.7 elo $13.66
GPT-5.2 Instant 142.4 elo $0.77
non-gemini-25pro-t03-mini41-t00-mini41-t03 141.7 elo $0.09
Claude Haiku 4.5 thinking 16000 126.6 elo $1.15
Claude Sonnet 4 thinking 16000 115.3 elo $2.33
Claude Sonnet 4.5 thinking 16000 112.7 elo $2.2
Kimi K2.5 111.2 elo $0.97
Claude Sonnet 4.6 110.6 elo $0.63
o4-mini low; precision PGN 109.9 elo $0.58
o4-mini low 104.1 elo $0.51
o3-mini medium 82.4 elo $1.61
Claude Opus 4.5 71.1 elo $0.87
o1-preview 46.3 elo $22.56
Claude Opus 4 thinking 16000 29.3 elo $7.92
non-deepseek-r1-t03-mini41-t10-mini41-t03 4.1 elo $2.22
Claude 3.7 Sonnet thinking 10000 -1 elo $2.08
Claude Sonnet 4 -10.9 elo $0.64
Claude Opus 4 -10.9 elo $3.28
Qwen3.5-35B-A3B precision q4_k_m -26 elo $0.46
Claude Sonnet 4.5 -35.9 elo $0.5
Claude 3.7 Sonnet thinking 1024 -36.1 elo $1.32
Claude 3.7 Sonnet thinking 5000 -36.1 elo $1.92
Qwen3-30B-A3B-Thinking Jul 2025 precision q4_k_m -37.3 elo $0.55
Claude Opus 4.1 -37.9 elo $2.97
Muse Ember xhigh -38.3 elo $0.44
Claude Haiku 4.5 -42 elo $0.19
o1-mini -52.4 elo $2.41
GPT-5 mini low -55.6 elo $0.13
Claude 3.7 Sonnet thinking 2048 -62.4 elo $1.61
Gemini 2.5 Pro May 2025 -70.7 elo $0.58
Claude 3.7 Sonnet -76.4 elo $0.9
GPT-5 Chat -87.3 elo $0.12
gpt-4-32k -97.8 elo $2.23
Claude 3.5 Sonnet (October 2024) -98.7 elo $0.56
qwen max -98.9 elo $0.13
Llama 4 Maverick -99.7 elo $0.06
GPT-4 Turbo Apr 2024 -103.8 elo $0.85
GPT-4o Nov 2024 -108.1 elo $0.32
GPT-4.1 -110.6 elo $0.14
GPT-4.5 -111 elo $6.48
GPT-5 nano low -117 elo $0.03
Grok 4.1 Fast non-reasoning -118.4 elo $0.02
gpt-4 [snapshot unspecified] -119.8 elo $1.9
Claude 3.5 Haiku -120.5 elo $0.15
Grok 4.20 non-reasoning -125 elo $0.11
Qwen3-Next-80B-A3B -127.9 elo $0.12
GPT-5.1 Instant -130.7 elo $0.12
Gemini 3.1 Flash-Lite -132.9 elo $0.02
GPT-4o Aug 2024 -133.9 elo $0.21
Claude 3.5 Sonnet -134.4 elo $0.57
Qwen3-235B-A22B-Instruct Jul 2025 -140.3 elo $0.08
Gemini 2.5 Pro Mar 2025 -144.3 elo $0.56
Claude 3 Opus -145.7 elo $2.32
gemini-2.5-pro [snapshot unspecified] -147.2 elo $0.35
GPT-4o May 2024 -157.7 elo $0.27
GPT-4.1 mini -167.2 elo $0.86
Gemini 3 Flash -179.5 elo $0.7
o3-mini low -183.9 elo $0.45
Grok 4 Fast non-reasoning -191.3 elo $0.02
Nemotron 3 Nano 30B-A3B precision q3_k_l -199.6 elo $0.02
DeepSeek-R1 -216.2 elo $0.94
gpt-oss-120b medium -217.2 elo $0.05
gpt-oss-20b medium -232 elo $0.02
GPT-4o mini -251.8 elo $0.02
gpt-oss-120b high -252.1 elo $0.04
gpt-oss-20b high -257.7 elo $0.02
Llama 3-70B -278.1 elo $0.02
gpt-oss-120b low -283.6 elo $0.06
gemini-2.5-flash [snapshot unspecified] -289.3 elo $0.05
GLM-5 -299.2 elo $0.1
gpt-oss-20b low -299.9 elo $0.02
Gemini 2.0 Flash Feb 2025 -310.8 elo $0.01
Claude 3 Sonnet -321.8 elo $0.4
glm-4.7-flash precision q4 -328.1 elo $0
Muse Ember max -337.1 elo $0.21
Gemini 1.5 Flash May 2024 -366.9 elo $0
Llama 4 Scout -401.8 elo $0.02
Gemma 2 27B precision q6_k_l -412.4 elo $0.02
K2-Think precision iq4_xs -457.6 elo $0.06
Gemma 2 9B -545.7 elo $0
DeepSeek-V3 Mar 2025 -579.5 elo $0.05
Llama 3.3 70B -607.7 elo $0.01
Qwen Plus Jan 2025 -616.2 elo $0.07
Qwen3-4B precision q8 -636.9 elo $0.13
Gemini 1.5 Pro Sep 2024 -651 elo $0.16
Qwen2.5-72B -672.2 elo $0.01
Gemini 2.0 Flash Dec 2024 -672.2 elo $0.01
Kimi K2 Jul 2025 -689.3 elo $0.06
Llama 3.1-8B -795.6 elo $0
Gemini 2.0 Flash-Lite -812.4 elo $0.01
DeepSeek-V3 -822.8 elo $0.03

The best score, 1,614, costs $4.64 a task

Runs on the cost frontier score higher than any run that's cheaper. Gemini 4 Argon at high comes within 4.4 of the best for $1.27 a task, 4 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Google DeepMind Meta OpenAI xAI NVIDIA Z.ai Anthropic Inception Labs Cost frontier One model at each effort level
0 500 1,000 1,500 2,000 $0.0001 $0.001 $0.01 $0.10 $1.00 $10 $100 Cost per task, log scale GPT-6 Astra, high Gemini 3.8 Flash, medium GPT-6.1 Terra, high Mercury 3, high GPT-6 Luna, max GPT-6 Luna, xhigh GPT-6 Luna, high 0 500 1,000 1,500 2,000 $0.0001 $0.001 $0.01 $0.10 $1.00 $10 $100 Cost per task, log scale GPT-6 Astra, high GPT-6.1 Sol, high

Source: Official LLM Chess leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GPT-6 Astra high 1,614 $4.64
GPT-6.1 Sol xhigh 1,612 $1.65
Gemini 4 Argon high 1,609 $1.27
GPT-6 Astra xhigh 1,579 $5.47
Claude Opus 5.5 high 1,575 $1.40
GPT-5.6 Sol xhigh 1,550 $8.23
GPT-5.6 Sol high 1,549 $6.20
Gemini 3.8 Flash high 1,546 $2.19
GPT-6 Sol xhigh 1,545 $3.41
GPT-5.5 medium 1,532 $6.57
GPT-5.5 high 1,528 $8.25
Gemini 3.8 Flash medium 1,527 $0.86
GPT-6.1 Terra xhigh 1,521 $1.13
Gemini 3.1 Pro high 1,511 $4.19
GPT-6.1 Sol high 1,454 $0.76
GPT-6.1 Terra high 1,391 $0.57
GPT-5.4 high 1,322 $5.12
GPT-6 Sol high 1,311 $1.41
Mercury 3 high 1,308 $0.48
GPT-5.6 Terra xhigh 1,293 $3.32
Claude Opus 5 high 1,285 $7.56
GPT-5.4 medium 1,253 $4.90
GPT-5.6 Sol medium 1,240 $3.71
GPT-6 Luna max 1,219 $0.25
GPT-5.6 Terra high 1,211 $2.66
Claude Sonnet 5.5 high 1,195 $1.53
GPT-5.6 Luna xhigh 1,185 $2.51
Claude Haiku 5.5 high 1,143 $0.08
Gemini 3.7 Flash high 1,107 $0.75
Gemini 3.6 Flash high 1,094 $2.79
DeepSeek V4.1 Flash high 1,087 $0.74
GPT-5 medium 1,086 $6.36
GPT-5.2 medium 1,085 $3.12
GPT-5.3 Codex medium 1,053 $1.84
Gemini 3 Pro high 1,034 $4.23
Gemini 3.5 Flash medium 953 $2.52
GPT-5.1 medium 948 $3.21
GPT-5.6 Terra medium 936 $1.32
Gemini 3.6 Flash medium 921 $1.61
GPT-5.6 Luna high 919 $1.50
GPT-5 low 858 $2.56
GPT-6 Luna xhigh 855 $0.07
Qwen 3.8 27B precision q4_k_m 846 $2.29
Grok 4.20 reasoning 843
Grok 4.7 high 822 $2.71
Claude Sonnet 5 high 806 $2.77
Grok 4.1 Fast reasoning 800
GPT-5.2 low 783 $1.30
o3 medium 778 $11
GPT-5 mini high 759 $1.28
o3 low 739 $8.16
Grok 4 Fast reasoning 735
GPT-6 Luna high 730 $0.05
GPT-5.6 Luna medium 730 $0.63
Claude Opus 4.7 high 691 $5.57
GPT-5 nano high 683 $0.49
GPT-5.1 low 615 $0.91
GLM-5.3 high 605 $1.12
GLM-5.3-Flash high 572 $0.10
GPT-5.1-Codex-mini high 544 $1.10
GPT-5.1-Codex low 504 $1.38
Gemini 3.6 Flash low 498 $0.85
Claude Opus 4.8 high 450 $4.00
Claude Opus 4.5 thinking 16000 446 $7.34
Grok 4.6 high 443 $0.47
o4-mini high 440 $2.71
Grok-3 mini high 430
GPT-5.4 Mini high 416 $3.23
GPT-5 nano medium 413 $0.19
o3-mini high 403 $3.62
GPT-5 mini medium 349 $0.44
GPT-5.4 low 348 $7.69
DeepSeek-R1 May 2025 300 $1.91
o1 medium 280 $22
Claude Sonnet 5 272 $0.57
Qwen3.6 27B precision q4_k_s 270 $1.46
o4-mini medium 240 $1.11
Muse Glimmer precision q4_k_m 214 $1.17
GPT-5.4 Nano high 207 $0.74
Claude Opus 4.6 high 198 $8.76
Claude Sonnet 4.6 high 190 $11
DeepSeek-V3.2 non-reasoning 189 $0.09
Grok 4.3 high 179 $0.19
GPT-5.4 Mini low 176 $0.26
o1 low 154 $14
GPT-5.2 Instant 142 $0.77
Claude Haiku 4.5 thinking 16000 127 $1.15
Claude Sonnet 4 thinking 16000 115 $2.33
Claude Sonnet 4.5 thinking 16000 113 $2.20
Kimi K2.5 111 $0.97
Claude Sonnet 4.6 111 $0.63
o4-mini low; precision PGN 110 $0.58
o4-mini low 104 $0.51
o3-mini medium 82 $1.61
Claude Opus 4.5 71 $0.87
o1-preview 46 $23
Claude Opus 4 thinking 16000 29 $7.92
Claude 3.7 Sonnet thinking 10000 -1 $2.08
Claude Opus 4 -11 $3.28
Claude Sonnet 4 -11 $0.64
Qwen3.5-35B-A3B precision q4_k_m -26 $0.46
Grok-3 mini low -34
Claude Sonnet 4.5 -36 $0.50
Claude 3.7 Sonnet thinking 1024 -36 $1.32
Claude 3.7 Sonnet thinking 5000 -36 $1.92
Grok-3 mini -37
Qwen3-30B-A3B-Thinking Jul 2025 precision q4_k_m -37 $0.55
Claude Opus 4.1 -38 $2.97
Muse Ember xhigh -38 $0.44
Claude Haiku 4.5 -42 $0.19
o1-mini -52 $2.41
GPT-5 mini low -56 $0.13
Claude 3.7 Sonnet thinking 2048 -62 $1.61
Composer 2 Cursor CLI -63
Gemini 2.5 Pro May 2025 -71 $0.58
Claude 3.7 Sonnet -76 $0.90
GPT-5 Chat -87 $0.12
Claude 3.5 Sonnet (October 2024) -99 $0.56
Llama 4 Maverick -100 $0.06
GPT-4 Turbo Apr 2024 -104 $0.85
GPT-4o Nov 2024 -108 $0.32
GPT-4.1 -111 $0.14
GPT-4.5 -111 $6.48
GPT-5 nano low -117 $0.03
Grok 4.1 Fast non-reasoning -118 $0.02
Claude 3.5 Haiku -120 $0.15
Grok 4.20 non-reasoning -125 $0.11
Qwen3-Next-80B-A3B -128 $0.12
GPT-5.1 Instant -131 $0.12
Gemini 3.1 Flash-Lite -133 $0.02
GPT-4o Aug 2024 -134 $0.21
Claude 3.5 Sonnet -134 $0.57
Grok 3 -137
Qwen3-235B-A22B-Instruct Jul 2025 -140 $0.08
Gemini 2.5 Pro Mar 2025 -144 $0.56
Claude 3 Opus -146 $2.32
GPT-4o May 2024 -158 $0.27
GPT-4.1 mini -167 $0.86
Gemini 3 Flash -179 $0.70
o3-mini low -184 $0.45
Grok 4 Fast non-reasoning -191 $0.02
Nemotron 3 Nano 30B-A3B precision q3_k_l -200 $0.02
Qwen3.5-9B precision bf16 -200
Composer 1.5 Cursor CLI -212
DeepSeek-R1 -216 $0.94
gpt-oss-120b medium -217 $0.05
gpt-oss-20b medium -232 $0.02
GPT-4o mini -252 $0.02
gpt-oss-120b high -252 $0.04
gpt-oss-20b high -258 $0.02
Qwen3.5-9B precision q8_0 -258
Llama 3-70B -278 $0.02
gpt-oss-120b low -284 $0.06
GLM-5 -299 $0.10
gpt-oss-20b low -300 $0.02
Gemini 2.0 Flash Feb 2025 -311 $0.01
Claude 3 Sonnet -322 $0.40
glm-4.7-flash precision q4 -328 $0.003
Grok-2 Dec 2024 -335
Muse Ember max -337 $0.21
Gemini 1.5 Flash May 2024 -367 $0.003
Llama 4 Scout -402 $0.02
Gemma 2 27B precision q6_k_l -412 $0.02
K2-Think precision iq4_xs -458 $0.06
Gemma 2 9B -546 $0.001
Qwen3.5-4B precision bf16 -558
DeepSeek-V3 Mar 2025 -579 $0.05
Llama 3.3 70B -608 $0.01
Qwen Plus Jan 2025 -616 $0.07
Qwen3-4B precision q8 -637 $0.13
Gemini 1.5 Pro Sep 2024 -651 $0.15
Qwen2.5-72B -672 $0.01
Gemini 2.0 Flash Dec 2024 -672 $0.01
Kimi K2 Jul 2025 -689 $0.06
Llama 3.1-8B -796 $0.0009
Gemini 2.0 Flash-Lite -812 $0.007
Composer 1 Cursor CLI -812
DeepSeek-V3 -823 $0.03

11 models have held number one on the official leaderboard since its release in September 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI xAI Google DeepMind
Own leaderboard 11 leaders
GPT-6 Astra 1,614

Source: Official LLM Chess leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
-1,000 0 1,000 2,000 2024 2025 2026 2027 -1,000 0 1,000 2,000 2024 2025 2026 2027

Source: Official LLM Chess leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Games and long tasks

All 82 in Games and long tasks
  • ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6 Astra 99.95%
  • Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Each game scores in-game progress or outcomes, and the board ranks models by their average rank across the games. Gemini 2.5 Pro 3.90
  • MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. Seed2.0 Pro 49%
  • TextQuests Only in system cards Agents play 25 classic interactive fiction games. The score is harmful actions per 100 steps, where lower is better. The board also reports game progress. Llama 4 Scout 7.00
  • GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. GPT-6.1 Terra 76.8%
  • Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%