Benchmark Almanac

CivBench

GLM-5.3 1,634 leads since October 2026
Not in model cards

Models play full games of Civilization V with the Vox Populi mod against each other on matched maps and seeds. Pairwise skill ratings rank long-term strategy.

Also tracks diplomacy, strategy choices and win prediction quality. The Vox Deorum repo has about 110 stars. Release month is the first Reddit post found.

Made by
Vox Deorum
Released
October 2026
Kind
Video games
Level
Expert
Scored
Ranked by matches
Answers
Agent task
Tasks
All public
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
September 2026
Scores refreshed
November 12th, 2026
23
Entries on its own leaderboard, 19 models we track

GLM-5.3 leads with 1,634 among models released by September 2026

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in September 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

23 entries. Scroll sideways for the rest. Bars start at 1,300.

  1. 1,634 GLM-5.3
  2. 1,602 GPT-6.1 Sol
  3. 1,593 Claude Opus 5.5
  4. 1,573 Kimi-K2.7
  5. 1,557 GPT-6 Astra
  6. 1,537 DeepSeek V4.1 Flash
  7. 1,535 GLM-5.2
  8. 1,516 Qwen 3.8 27B
  9. 1,510 GPT-6 Luna
  10. 1,504 Kimi K2.6
  11. 1,499 Qwen3.8-Flash-Next
  12. 1,497 Claude Sonnet 5.5
  13. 1,482 GLM-5.1
  14. 1,477 DeepSeek-V4-Flash
  15. 1,470 MiniMax-M2.7
  16. 1,468 GLM-5.3-Flash
  17. 1,460 Qwen-3.5
  18. 1,458 Gemma-4
  19. 1,448 MiniMax-M3
  20. 1,441 Nemotron 3 Super
  21. 1,422 gpt-oss-120b
  22. 1,391 Qwen3.6 27B
  23. 1,350 Jev!

Bars start at 1,300.

  1. 1 GLM-5.3 1,634
  2. 2 GPT-6.1 Sol 1,602
  3. 3 Claude Opus 5.5 1,593
  4. 4 Kimi-K2.7 1,573
  5. 5 GPT-6 Astra 1,557
  6. 6 DeepSeek V4.1 Flash 1,537
  7. 7 GLM-5.2 1,535
  8. 8 Qwen 3.8 27B 1,516
  9. 9 GPT-6 Luna 1,510
  10. 10 Kimi K2.6 1,504
  11. 11 Qwen3.8-Flash-Next 1,499
  12. 12 Claude Sonnet 5.5 1,497
  13. 13 GLM-5.1 1,482
  14. 14 DeepSeek-V4-Flash 1,477
  15. 15 MiniMax-M2.7 1,470
  16. 16 GLM-5.3-Flash 1,468
  17. 17 Qwen-3.5 1,460
  18. 18 Gemma-4 1,458
  19. 19 MiniMax-M3 1,448
  20. 20 Nemotron 3 Super 1,441
  21. 21 gpt-oss-120b 1,422
  22. 22 Qwen3.6 27B 1,391
  23. 23 Jev! 1,350

Source: Official CivBench leaderboard

#EntryScoreSettingDateSource
1 GLM-5.3 1,634 every 5 turns Aug 2026 Official leaderboard
2 GPT-6.1 Sol 1,602 every 5 turns Sep 2026 Official leaderboard
3 Claude Opus 5.5 1,593 every 5 turns Sep 2026 Official leaderboard
4 Kimi-K2.7 1,573 every turn Official leaderboard
5 GPT-6 Astra 1,557 every 5 turns Sep 2026 Official leaderboard
6 DeepSeek V4.1 Flash 1,537 every turn Sep 2026 Official leaderboard
7 GLM-5.2 1,535 every turn Jun 2026 Official leaderboard
8 Qwen 3.8 27B 1,516 every turn Aug 2026 Official leaderboard
9 GPT-6 Luna 1,510 every 5 turns Sep 2026 Official leaderboard
10 Kimi K2.6 1,504 every turn Apr 2026 Official leaderboard
11 Qwen3.8-Flash-Next 1,499 every 5 turns Aug 2026 Official leaderboard
12 Claude Sonnet 5.5 1,497 every 5 turns Sep 2026 Official leaderboard
13 GLM-5.1 1,482 every turn Apr 2026 Official leaderboard
14 DeepSeek-V4-Flash 1,477 every 5 turns Apr 2026 Official leaderboard
15 MiniMax-M2.7 1,470 every turn Mar 2026 Official leaderboard
16 GLM-5.3-Flash 1,468 every turn Aug 2026 Official leaderboard
17 Qwen-3.5 1,460 every turn Official leaderboard
18 Gemma-4 1,458 every 5 turns Official leaderboard
19 MiniMax-M3 1,448 every 5 turns Jun 2026 Official leaderboard
20 Nemotron 3 Super 1,441 every turn Mar 2026 Official leaderboard
21 gpt-oss-120b 1,422 every turn Aug 2025 Official leaderboard
22 Qwen3.6 27B 1,391 every turn Apr 2026 Official leaderboard
23 Jev! 1,350 every turn Official leaderboard

Cost versus skill

The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

1,400 1,600 1,800 2,000 $0.1$1$10$100 Cost per player per game ($), log scale, lower is better Skill rating (rating) gpt-oss-120b: Skill rating 1,422 rating, cost per player per game $0.52 Nemotron 3 Super: Skill rating 1,441 rating, cost per player per game $1.93 Nemotron 3 Super: Skill rating 1,436 rating, cost per player per game $0.77 MiniMax-M2.7: Skill rating 1,470 rating, cost per player per game $3.61 MiniMax-M2.7: Skill rating 1,444 rating, cost per player per game $1.74 GLM-5.1: Skill rating 1,482 rating, cost per player per game $15.1 Kimi K2.6: Skill rating 1,504 rating, cost per player per game $16.8 Qwen3.6 27B: Skill rating 1,391 rating, cost per player per game $4.09 Qwen3.6 27B: Skill rating 1,341 rating, cost per player per game $1.8 DeepSeek-V4-Flash: Skill rating 1,431 rating, cost per player per game $1.02 MiniMax-M3: Skill rating 1,448 rating, cost per player per game $2.79 GLM-5.2: Skill rating 1,535 rating, cost per player per game $13.2 GLM-5.2: Skill rating 1,495 rating, cost per player per game $5.6 Qwen 3.8 27B: Skill rating 1,516 rating, cost per player per game $6.8 Qwen 3.8 27B: Skill rating 1,454 rating, cost per player per game $2.72 GLM-5.3-Flash: Skill rating 1,468 rating, cost per player per game $1.74 GLM-5.3-Flash: Skill rating 1,436 rating, cost per player per game $0.88 Qwen3.8-Flash-Next: Skill rating 1,499 rating, cost per player per game $1 Qwen3.8-Flash-Next: Skill rating 1,449 rating, cost per player per game $2 GPT-6 Astra: Skill rating 1,557 rating, cost per player per game $53.9 DeepSeek V4.1 Flash: Skill rating 1,494 rating, cost per player per game $0.89 Claude Opus 5.5: Skill rating 1,593 rating, cost per player per game $32.2 GPT-6 Luna: Skill rating 1,491 rating, cost per player per game $1.16 Claude Sonnet 5.5: Skill rating 1,497 rating, cost per player per game $17.7 GPT-6.1 Sol: Skill rating 1,544 rating, cost per player per game $23.6 Kimi-K2.7: Skill rating 1,573 rating, cost per player per game $14.4. Not in our model list Kimi-K2.7: Skill rating 1,514 rating, cost per player per game $6.14. Not in our model list Qwen-3.5: Skill rating 1,460 rating, cost per player per game $7.61. Not in our model list Qwen-3.5: Skill rating 1,421 rating, cost per player per game $2.71. Not in our model list Gemma-4: Skill rating 1,377 rating, cost per player per game $1.06. Not in our model list Jev!: Skill rating 1,350 rating, cost per player per game $0.47. Not in our model list gpt-oss-120b: Skill rating 1,383 rating, cost per player per game $0.23 DeepSeek-V4-Flash: Skill rating 1,477 rating, cost per player per game $0.49 GLM-5.3: Skill rating 1,634 rating, cost per player per game $20.6 DeepSeek V4.1 Flash: Skill rating 1,537 rating, cost per player per game $2.45 GPT-6 Luna: Skill rating 1,510 rating, cost per player per game $0.61 GPT-6.1 Sol: Skill rating 1,602 rating, cost per player per game $11.7 GPT-6.1 Sol: Skill rating 1,543 rating, cost per player per game $7.4 Gemma-4: Skill rating 1,458 rating, cost per player per game $0.46. Not in our model list GLM-5.3GPT-6.1 SolGPT-6.1 SolDeepSeek V4.1 FlashGPT-6 LunaDeepSeek-V4-FlashGemma-4gpt-oss-120b 1,400 1,600 1,800 2,000 $0.1$1$10$100 Cost per player per game ($), log scale, lower is better Skill rating (rating) gpt-oss-120b: Skill rating 1,422 rating, cost per player per game $0.52 Nemotron 3 Super: Skill rating 1,441 rating, cost per player per game $1.93 Nemotron 3 Super: Skill rating 1,436 rating, cost per player per game $0.77 MiniMax-M2.7: Skill rating 1,470 rating, cost per player per game $3.61 MiniMax-M2.7: Skill rating 1,444 rating, cost per player per game $1.74 GLM-5.1: Skill rating 1,482 rating, cost per player per game $15.1 Kimi K2.6: Skill rating 1,504 rating, cost per player per game $16.8 Qwen3.6 27B: Skill rating 1,391 rating, cost per player per game $4.09 Qwen3.6 27B: Skill rating 1,341 rating, cost per player per game $1.8 DeepSeek-V4-Flash: Skill rating 1,431 rating, cost per player per game $1.02 MiniMax-M3: Skill rating 1,448 rating, cost per player per game $2.79 GLM-5.2: Skill rating 1,535 rating, cost per player per game $13.2 GLM-5.2: Skill rating 1,495 rating, cost per player per game $5.6 Qwen 3.8 27B: Skill rating 1,516 rating, cost per player per game $6.8 Qwen 3.8 27B: Skill rating 1,454 rating, cost per player per game $2.72 GLM-5.3-Flash: Skill rating 1,468 rating, cost per player per game $1.74 GLM-5.3-Flash: Skill rating 1,436 rating, cost per player per game $0.88 Qwen3.8-Flash-Next: Skill rating 1,499 rating, cost per player per game $1 Qwen3.8-Flash-Next: Skill rating 1,449 rating, cost per player per game $2 GPT-6 Astra: Skill rating 1,557 rating, cost per player per game $53.9 DeepSeek V4.1 Flash: Skill rating 1,494 rating, cost per player per game $0.89 Claude Opus 5.5: Skill rating 1,593 rating, cost per player per game $32.2 GPT-6 Luna: Skill rating 1,491 rating, cost per player per game $1.16 Claude Sonnet 5.5: Skill rating 1,497 rating, cost per player per game $17.7 GPT-6.1 Sol: Skill rating 1,544 rating, cost per player per game $23.6 Kimi-K2.7: Skill rating 1,573 rating, cost per player per game $14.4. Not in our model list Kimi-K2.7: Skill rating 1,514 rating, cost per player per game $6.14. Not in our model list Qwen-3.5: Skill rating 1,460 rating, cost per player per game $7.61. Not in our model list Qwen-3.5: Skill rating 1,421 rating, cost per player per game $2.71. Not in our model list Gemma-4: Skill rating 1,377 rating, cost per player per game $1.06. Not in our model list Jev!: Skill rating 1,350 rating, cost per player per game $0.47. Not in our model list gpt-oss-120b: Skill rating 1,383 rating, cost per player per game $0.23 DeepSeek-V4-Flash: Skill rating 1,477 rating, cost per player per game $0.49 GLM-5.3: Skill rating 1,634 rating, cost per player per game $20.6 DeepSeek V4.1 Flash: Skill rating 1,537 rating, cost per player per game $2.45 GPT-6 Luna: Skill rating 1,510 rating, cost per player per game $0.61 GPT-6.1 Sol: Skill rating 1,602 rating, cost per player per game $11.7 GPT-6.1 Sol: Skill rating 1,543 rating, cost per player per game $7.4 Gemma-4: Skill rating 1,458 rating, cost per player per game $0.46. Not in our model list GLM-5.3GPT-6.1 SolGPT-6.1 SolDeepSeek V4.1 FlashGPT-6 LunaDeepSeek-V4-FlashGemma-4gpt-oss-120b
ModelSettingSkill ratingCost per player per game
GLM-5.3 every 5 turns 1,634 rating $20.55
GPT-6.1 Sol every 5 turns 1,602 rating $11.66
Claude Opus 5.5 every 5 turns 1,593 rating $32.22
Kimi-K2.7 every turn 1,573 rating $14.37
GPT-6 Astra every 5 turns 1,557 rating $53.9
GPT-6.1 Sol every turn 1,544 rating $23.62
GPT-6.1 Sol agentic 1,543 rating $7.4
DeepSeek V4.1 Flash every turn 1,537 rating $2.45
GLM-5.2 every turn 1,535 rating $13.2
Qwen 3.8 27B every turn 1,516 rating $6.8
Kimi-K2.7 every 5 turns 1,514 rating $6.14
GPT-6 Luna every 5 turns 1,510 rating $0.61
Kimi K2.6 every turn 1,504 rating $16.84
Qwen3.8-Flash-Next every 5 turns 1,499 rating $1
Claude Sonnet 5.5 every 5 turns 1,497 rating $17.74
GLM-5.2 every 5 turns 1,495 rating $5.6
DeepSeek V4.1 Flash every 5 turns 1,494 rating $0.89
GPT-6 Luna every turn 1,491 rating $1.16
GLM-5.1 every turn 1,482 rating $15.07
DeepSeek-V4-Flash every 5 turns 1,477 rating $0.49
MiniMax-M2.7 every turn 1,470 rating $3.61
GLM-5.3-Flash every turn 1,468 rating $1.74
Qwen-3.5 every turn 1,460 rating $7.61
Gemma-4 every 5 turns 1,458 rating $0.46
Qwen 3.8 27B every 5 turns 1,454 rating $2.72
Qwen3.8-Flash-Next every turn 1,449 rating $2
MiniMax-M3 every 5 turns 1,448 rating $2.79
MiniMax-M2.7 every 5 turns 1,444 rating $1.74
Nemotron 3 Super every turn 1,441 rating $1.93
Nemotron 3 Super every 5 turns 1,436 rating $0.77
GLM-5.3-Flash every 5 turns 1,436 rating $0.88
DeepSeek-V4-Flash every turn 1,431 rating $1.02
gpt-oss-120b every turn 1,422 rating $0.52
Qwen-3.5 every 5 turns 1,421 rating $2.71
Qwen3.6 27B every turn 1,391 rating $4.09
gpt-oss-120b every 5 turns 1,383 rating $0.23
Gemma-4 every turn 1,377 rating $1.06
Jev! every turn 1,350 rating $0.47
Qwen3.6 27B every 5 turns 1,341 rating $1.8

The best score, 1,634, costs $21 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI DeepSeek Z.ai Cost frontier One model at each effort level
0 500 1,000 1,500 2,000 $0.10 $1.00 $10 $100 Cost per task, log scale GLM-5.3 GPT-6.1 Sol DeepSeek V4.1 Flash GPT-6 Luna gpt-oss-120b 0 500 1,000 1,500 2,000 $0.10 $1.00 $10 $100 Cost per task, log scale GLM-5.3 GPT-6.1 Sol

Source: Official CivBench leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GLM-5.3 every 5 turns 1,634 $21 3,203,151
GPT-6.1 Sol every 5 turns 1,602 $12 54,415
Claude Opus 5.5 every 5 turns 1,593 $32 105,746
GPT-6 Astra every 5 turns 1,557 $54 39,913
GPT-6.1 Sol every turn 1,544 $24 124,843
GPT-6.1 Sol agentic 1,543 $7.40 67,384
DeepSeek V4.1 Flash every turn 1,537 $2.45 1,690,237
GLM-5.2 every turn 1,535 $13 4,263,283
Qwen 3.8 27B every turn 1,516 $6.80 1,756,343
GPT-6 Luna every 5 turns 1,510 $0.61 40,338
Kimi K2.6 every turn 1,504 $17 2,386,068
Qwen3.8-Flash-Next every 5 turns 1,499 $1.00 465,933
Claude Sonnet 5.5 every 5 turns 1,497 $18 191,348
GLM-5.2 every 5 turns 1,495 $5.60 1,565,875
DeepSeek V4.1 Flash every 5 turns 1,494 $0.89 350,501
GPT-6 Luna every turn 1,491 $1.16 133,690
GLM-5.1 every turn 1,482 $15 1,125,884
DeepSeek-V4-Flash every 5 turns 1,477 $0.49 219,647
MiniMax-M2.7 every turn 1,470 $3.61 420,677
GLM-5.3-Flash every turn 1,468 $1.74 583,509
Qwen 3.8 27B every 5 turns 1,454 $2.72 601,206
Qwen3.8-Flash-Next every turn 1,449 $2.00 1,194,039
MiniMax-M3 every 5 turns 1,448 $2.79 926,360
MiniMax-M2.7 every 5 turns 1,444 $1.74 131,874
Nemotron 3 Super every turn 1,441 $1.93 1,651,510
Nemotron 3 Super every 5 turns 1,436 $0.77 654,165
GLM-5.3-Flash every 5 turns 1,436 $0.88 239,153
DeepSeek-V4-Flash every turn 1,431 $1.02 539,393
gpt-oss-120b every turn 1,422 $0.52 558,464
Qwen3.6 27B every turn 1,391 $4.09 530,614
gpt-oss-120b every 5 turns 1,383 $0.23 189,376
Qwen3.6 27B every 5 turns 1,341 $1.80 175,365

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Z.ai
Own leaderboard 1 leader
GLM-5.3 1,634

Source: Official CivBench leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
1,300 1,400 1,500 1,600 1,700 2026 1,300 1,400 1,500 1,600 1,700 2026

Source: Official CivBench leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Games and long tasks

All 82 in Games and long tasks
  • ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6 Astra 99.95%
  • Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Each game scores in-game progress or outcomes, and the board ranks models by their average rank across the games. Gemini 2.5 Pro 3.90
  • MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. Seed2.0 Pro 49%
  • TextQuests Only in system cards Agents play 25 classic interactive fiction games. The score is harmful actions per 100 steps, where lower is better. The board also reports game progress. Llama 4 Scout 7.00
  • GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. GPT-6.1 Terra 76.8%
  • Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%