Benchmark Almanac

LLM Chess Arena (Louis Guichard)

Gemini 3.5 Flash 1,495 leads since May 2026
Not in model cards

Models play each other from a text diagram, piece locations, and recent moves. A model forfeits after three failed attempts to make a legal move.

Guichard’s Elo standings cover 47 models with completed games. The standings include average time per move and cost per game.

Made by
Louis Guichard
Released
August 2025
Kind
Multi-agent games
Level
School
Scored
Ranked by matches
Answers
Agent task
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Actions
Interaction
Multi-agent
Environment
Chess
Items from
Live play or votes
Refresh
Live
Run by
A host
In views
Games, Agents, Kept fresh
Strongest model tested
GPT-5.6 Terra #18 on the capability index
Last model added
July 2026
Scores refreshed
November 20th, 2026
47
Models scored on its own leaderboard

Gemini 3.5 Flash leads with 1,495 among models released by July 2026

Best score per model from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best.

The newest model with a score here came out in July 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

Bars start at 800.

  1. 1 Gemini 3.5 Flash 1,495
  2. 2 GPT-5 mini 1,484
  3. 3 GPT-5.6 Luna 1,484
  4. 4 Gemini 3 Pro 1,483
  5. 5 GPT-5.1 1,429
  6. 6 Grok 4.1 Fast 1,425
  7. 7 GPT-5 1,418
  8. 8 Grok 4 1,413
  9. 9 Gemini 3 Flash 1,365
  10. 10 GPT-5 nano 1,362
  11. 11 o3 1,347
  12. 12 GPT-5.4 Nano 1,332
  13. 13 Kimi K2.5 1,286
  14. 14 Grok 4.5 1,283
  15. 15 GPT-5.6 Terra 1,278
  16. 16 gpt-oss-120b 1,275
  17. 17 DeepSeek-V4-Pro 1,255
  18. 18 Claude Opus 4.1 1,232
  19. 19 Gemma 4 31B IT 1,229
  20. 20 Grok 4.3 1,222
  21. 21 Gemma 4 26B A4B 1,173
  22. 22 MiniMax-M2.1 1,172
  23. 23 Qwen 3.6 Flash 1,157
  24. 24 gpt-oss-20b 1,149
  25. 25 MiniMax-M2 1,147
  26. 26 DeepSeek-V3.2 1,143
  27. 27 Gemini 2.5 Pro Jun 2025 1,098
  28. 28 Mistral Medium 3.5 1,094
  29. 29 Claude Sonnet 4.5 1,093
  30. 30 GPT-5.4 Mini 1,056
  31. 31 Claude Sonnet 4 1,050
  32. 32 Mistral Large 3 1,050
  33. 33 Gemini 3.1 Flash-Lite 1,049
  34. 34 GPT-4o Mar 2025 1,045
  35. 35 Mistral Medium 3.1 1,038
  36. 36 Grok 3 1,035
  37. 37 DeepSeek-V3.1 1,035
  38. 38 Llama 4 Maverick 1,023
  39. 39 DeepSeek-V3 Mar 2025 1,011
  40. 40 DeepSeek-R1 May 2025 1,004
  41. 41 Gemini 2.5 Flash Sep 2025 968
  42. 42 Kimi K2 Sep 2025 923
  43. 43 Mistral Small 4 893
  44. 44 Qwen3-30B-A3B 882
  45. 45 Ministral 3 14B 873
  46. 46 Gemma 3 27B 863
  47. 47 Gemini 2.5 Flash-Lite Sep 2025 832

Source: Official LLM Chess Arena (Louis Guichard) leaderboard

#ModelScoreSettingDateSource
1 Gemini 3.5 Flash 1,495 May 2026 Official leaderboard
2 GPT-5 mini 1,484 Aug 2025 Official leaderboard
3 GPT-5.6 Luna 1,484 Jul 2026 Official leaderboard
4 Gemini 3 Pro 1,483 Nov 2025 Official leaderboard
5 GPT-5.1 1,429 Nov 2025 Official leaderboard
6 Grok 4.1 Fast 1,425 Nov 2025 Official leaderboard
7 GPT-5 1,418 Aug 2025 Official leaderboard
8 Grok 4 1,413 Jul 2025 Official leaderboard
9 Gemini 3 Flash 1,365 Dec 2025 Official leaderboard
10 GPT-5 nano 1,362 Aug 2025 Official leaderboard
11 o3 1,347 Apr 2025 Official leaderboard
12 GPT-5.4 Nano 1,332 Mar 2026 Official leaderboard
13 Kimi K2.5 1,286 Jan 2026 Official leaderboard
14 Grok 4.5 1,283 Jul 2026 Official leaderboard
15 GPT-5.6 Terra 1,278 Jul 2026 Official leaderboard
16 gpt-oss-120b 1,275 Aug 2025 Official leaderboard
17 DeepSeek-V4-Pro 1,255 Apr 2026 Official leaderboard
18 Claude Opus 4.1 1,232 Aug 2025 Official leaderboard
19 Gemma 4 31B IT 1,229 Apr 2026 Official leaderboard
20 Grok 4.3 1,222 Apr 2026 Official leaderboard
21 Gemma 4 26B A4B 1,173 Apr 2026 Official leaderboard
22 MiniMax-M2.1 1,172 Dec 2025 Official leaderboard
23 Qwen 3.6 Flash 1,157 Apr 2026 Official leaderboard
24 gpt-oss-20b 1,149 Aug 2025 Official leaderboard
25 MiniMax-M2 1,147 Oct 2025 Official leaderboard
26 DeepSeek-V3.2 1,143 Dec 2025 Official leaderboard
27 Gemini 2.5 Pro Jun 2025 1,098 Jun 2025 Official leaderboard
28 Mistral Medium 3.5 1,094 Apr 2026 Official leaderboard
29 Claude Sonnet 4.5 1,093 Sep 2025 Official leaderboard
30 GPT-5.4 Mini 1,056 Mar 2026 Official leaderboard
31 Claude Sonnet 4 1,050 May 2025 Official leaderboard
32 Mistral Large 3 1,050 Dec 2025 Official leaderboard
33 Gemini 3.1 Flash-Lite 1,049 Mar 2026 Official leaderboard
34 GPT-4o Mar 2025 1,045 Mar 2025 Official leaderboard
35 Mistral Medium 3.1 1,038 Aug 2025 Official leaderboard
36 Grok 3 1,035 Apr 2025 Official leaderboard
37 DeepSeek-V3.1 1,035 Aug 2025 Official leaderboard
38 Llama 4 Maverick 1,023 Apr 2025 Official leaderboard
39 DeepSeek-V3 Mar 2025 1,011 Mar 2025 Official leaderboard
40 DeepSeek-R1 May 2025 1,004 May 2025 Official leaderboard
41 Gemini 2.5 Flash Sep 2025 968 Sep 2025 Official leaderboard
42 Kimi K2 Sep 2025 923 Sep 2025 Official leaderboard
43 Mistral Small 4 893 Mar 2026 Official leaderboard
44 Qwen3-30B-A3B 882 Apr 2025 Official leaderboard
45 Ministral 3 14B 873 Dec 2025 Official leaderboard
46 Gemma 3 27B 863 Mar 2025 Official leaderboard
47 Gemini 2.5 Flash-Lite Sep 2025 832 Sep 2025 Official leaderboard

3 models have held number one on the official leaderboard since its release in August 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

xAI OpenAI Google DeepMind
Own leaderboard Not tested since Jul 2026

Source: Official LLM Chess Arena (Louis Guichard) leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
800 1,000 1,200 1,400 1,600 2025 2026 800 1,000 1,200 1,400 1,600 2025 2026

Source: Official LLM Chess Arena (Louis Guichard) leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Games and long tasks

All 261 in Games and long tasks
  • ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6.2 Astra 100%
  • Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Each game scores in-game progress or outcomes, and the board ranks models by their average rank across the games. Gemini 2.5 Pro 3.90
  • MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. Seed2.0 Pro 49%
  • TextQuests Only in system cards Agents play 25 classic interactive fiction games. The score is harmful actions per 100 steps, where lower is better. The board also reports game progress. Llama 4 Scout 7.00
  • GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. GPT-6.1 Terra 76.8%
  • Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%

Data audit

All 9 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (47 entries).
  • Pass Models scored 47 models we track.
  • Pass Newest score The newest is from July 2026, for GPT-5.6 Luna.
  • Note Frontier models None of the top 10 has a score. The strongest tested is GPT-5.6 Terra (#18).
  • Note Headroom Scored as an Elo rating, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on chess.louisguichard.fr: 47 rows for models we track at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check