Benchmark Almanac

LLM Debate Benchmark

Claude Fable 5.1 1,706 leads since September 2026
Not in model cards

Models debate an opponent over multiple rounds. Judges compare the arguments.

The author publishes the model comparison in the repository.

Made by
Lech Mazur
Kind
Multi-turn conversation
Level
University
Scored
Ranked by matches
Answers
Open-ended
Tasks
All public
Measures
Preference
Language
English
In and out
Text → Text
Interaction
Multi-agent
Items from
Live play or votes
Refresh
Live
Run by
Anyone
In views
Agents, Kept fresh
Strongest model tested
GPT-6.2 Astra #1 on the capability index
Last model added
November 2026
Scores refreshed
November 20th, 2026
66
Entries on its own leaderboard, 58 models we track

Claude Fable 5.1 leads with 1,706 Claude Fable 5.1 leads with 1,706

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

Bars start at 1,000.

  1. 1 Claude Fable 5.1 high 1,706
  2. 2 Claude Fable 5 high 1,705
  3. 3 Claude Opus 5 high 1,704
  4. 4 Claude Fable 5.2 high 1,688
  5. 5 Claude Opus 5.5 high 1,680
  6. 6 Qwen4-Max high 1,679
  7. 7 Kimi K3 1,679
  8. 8 Gemini 4 Carbon high 1,676
  9. 9 GPT-6.2 Astra high 1,673
  10. 10 Gemini 4 Argon high 1,649
  11. 11 Tencent Hy4 high 1,647
  12. 12 Muse Wonder high 1,637
  13. 13 Claude Opus 4.7 high 1,627
  14. 14 GPT-5.6 Sol high 1,625
  15. 15 Muse Spark 1.1 high 1,617
  16. 16 GLM-5.3 high 1,613
  17. 17 Claude Opus 4.8 high 1,610
  18. 18 GPT-6 Astra high 1,607
  19. 19 Grok 4.7 high 1,601
  20. 20 Kimi K3.1 Pro high 1,601
  21. 21 Grok 4.6 high 1,585
  22. 22 Xiaomi MiMo V2.6 Pro thinking 1,584
  23. 23 DeepSeek V4.1 Flash 1017 high 1,579
  24. 24 Muse Spark 1.3 high 1,578
  25. 25 Claude Sonnet 5 high 1,568
  26. 26 DeepSeek V4.1 Flash high 1,564
  27. 27 Tencent Hy4 preview high 1,561
  28. 28 DeepSeek V4 Pro 0813 high 1,552
  29. 29 GLM-5.2 max 1,542
  30. 30 Claude Sonnet 4.6 high 1,540
  31. 31 Qwen 3.8 Max 1,535
  32. 32 GPT-5.4 high 1,525
  33. 33 Step 5 Preview high 1,521
  34. 34 MiniMax-M3.1-Flash high 1,509
  35. 35 GPT-5.5 high 1,507
  36. 36 GLM-5.1 1,498
  37. 37 Kimi K2.6 1,490
  38. 38 Gemini 3.8 Flash high 1,488
  39. 39 MiniMax-M3 1,483
  40. 40 Grok 4.5 high 1,470
  41. 41 Gemini 3.1 Pro 1,468
  42. 42 ByteDance Seed2.1 Pro 1,468
  43. 43 Xiaomi MiMo V2.5 Pro 1,463
  44. 44 Qwen 3.6 Max (Preview) 1,454
  45. 45 Mistral Large 4 high 1,442
  46. 46 Gemini 3.7 Flash high 1,434
  47. 47 ByteDance Seed2.0 Pro 1,434
  48. 48 Kimi K2.5 thinking 1,431
  49. 49 DeepSeek-V4-Pro 1,427
  50. 50 Qwen3.7-Max 1,414
  51. 51 MiniMax-M2.7 1,412
  52. 52 Grok 4.20 0309 Reasoning 1,390
  53. 53 Gemini 3.5 Flash 1,387
  54. 54 Baidu Ernie 5.1 1,370
  55. 55 Xiaomi MiMo V2 Pro 1,369
  56. 56 Qwen3.5 397B-A17B 1,368
  57. 57 Tencent Hy3 preview high 1,366
  58. 58 Step 3.7 Flash high 1,355
  59. 59 Grok 4.3 1,350
  60. 60 DeepSeek-V3.2 1,341
  61. 61 Mistral Medium 3.5 high 1,321
  62. 62 Gemini 3.1 Flash-Lite 1,308
  63. 63 gpt-oss-120b 1,242
  64. 64 Baidu Ernie 5.0 1,225
  65. 65 Mistral Large 3 1,170
  66. 66 Llama 4 Maverick 1,006

Source: Official LLM Debate Benchmark leaderboard

Bars start at 1,000.

  1. 1 Claude Fable 5.1 high 1,706
  2. 2 Claude Fable 5 high 1,705
  3. 3 Claude Opus 5 high 1,704
  4. 4 Claude Fable 5.2 high 1,688
  5. 5 Claude Opus 5.5 high 1,680
  6. 6 Qwen4-Max high 1,679
  7. 7 Kimi K3 1,679
  8. 8 Gemini 4 Carbon high 1,676
  9. 9 GPT-6.2 Astra high 1,673
  10. 10 Gemini 4 Argon high 1,649
  11. 11 Tencent Hy4 high 1,647
  12. 12 Muse Wonder high 1,637
  13. 13 Claude Opus 4.7 high 1,627
  14. 14 GPT-5.6 Sol high 1,625
  15. 15 Muse Spark 1.1 high 1,617
  16. 16 GLM-5.3 high 1,613
  17. 17 Claude Opus 4.8 high 1,610
  18. 18 GPT-6 Astra high 1,607
  19. 19 Grok 4.7 high 1,601
  20. 20 Kimi K3.1 Pro high 1,601
  21. 21 Grok 4.6 high 1,585
  22. 22 DeepSeek V4.1 Flash 1017 high 1,579
  23. 23 Muse Spark 1.3 high 1,578
  24. 24 Claude Sonnet 5 high 1,568
  25. 25 DeepSeek V4.1 Flash high 1,564
  26. 26 Tencent Hy4 preview high 1,561
  27. 27 DeepSeek V4 Pro 0813 high 1,552
  28. 28 GLM-5.2 max 1,542
  29. 29 Claude Sonnet 4.6 high 1,540
  30. 30 Qwen 3.8 Max 1,535
  31. 31 GPT-5.4 high 1,525
  32. 32 Step 5 Preview high 1,521
  33. 33 MiniMax-M3.1-Flash high 1,509
  34. 34 GPT-5.5 high 1,507
  35. 35 GLM-5.1 1,498
  36. 36 Kimi K2.6 1,490
  37. 37 Gemini 3.8 Flash high 1,488
  38. 38 MiniMax-M3 1,483
  39. 39 Grok 4.5 high 1,470
  40. 40 Gemini 3.1 Pro 1,468
  41. 41 Qwen 3.6 Max (Preview) 1,454
  42. 42 Mistral Large 4 high 1,442
  43. 43 Gemini 3.7 Flash high 1,434
  44. 44 Kimi K2.5 thinking 1,431
  45. 45 DeepSeek-V4-Pro 1,427
  46. 46 Qwen3.7-Max 1,414
  47. 47 MiniMax-M2.7 1,412
  48. 48 Gemini 3.5 Flash 1,387
  49. 49 Qwen3.5 397B-A17B 1,368
  50. 50 Tencent Hy3 preview high 1,366
  51. 51 Step 3.7 Flash high 1,355
  52. 52 Grok 4.3 1,350
  53. 53 DeepSeek-V3.2 1,341
  54. 54 Mistral Medium 3.5 high 1,321
  55. 55 Gemini 3.1 Flash-Lite 1,308
  56. 56 gpt-oss-120b 1,242
  57. 57 Mistral Large 3 1,170
  58. 58 Llama 4 Maverick 1,006

Source: Official LLM Debate Benchmark leaderboard

#EntryScoreSettingDateSource
1 Claude Fable 5.1 1,706 high Sep 2026 Official leaderboard
2 Claude Fable 5 1,705 high Jun 2026 Official leaderboard
3 Claude Opus 5 1,704 high Jul 2026 Official leaderboard
4 Claude Fable 5.2 1,688 high Nov 2026 Official leaderboard
5 Claude Opus 5.5 1,680 high Sep 2026 Official leaderboard
6 Qwen4-Max 1,679 high Nov 2026 Official leaderboard
7 Kimi K3 1,679 Jul 2026 Official leaderboard
8 Gemini 4 Carbon 1,676 high Oct 2026 Official leaderboard
9 GPT-6.2 Astra 1,673 high Nov 2026 Official leaderboard
10 Gemini 4 Argon 1,649 high Sep 2026 Official leaderboard
11 Tencent Hy4 1,647 high Nov 2026 Official leaderboard
12 Muse Wonder 1,637 high Oct 2026 Official leaderboard
13 Claude Opus 4.7 1,627 high Apr 2026 Official leaderboard
14 GPT-5.6 Sol 1,625 high Jul 2026 Official leaderboard
15 Muse Spark 1.1 1,617 high Jul 2026 Official leaderboard
16 GLM-5.3 1,613 high Aug 2026 Official leaderboard
17 Claude Opus 4.8 1,610 high May 2026 Official leaderboard
18 GPT-6 Astra 1,607 high Sep 2026 Official leaderboard
19 Grok 4.7 1,601 high Sep 2026 Official leaderboard
20 Kimi K3.1 Pro 1,601 high Oct 2026 Official leaderboard
21 Grok 4.6 1,585 high Aug 2026 Official leaderboard
22 Xiaomi MiMo V2.6 Pro 1,584 thinking Official leaderboard
23 DeepSeek V4.1 Flash 1017 1,579 high Oct 2026 Official leaderboard
24 Muse Spark 1.3 1,578 high Sep 2026 Official leaderboard
25 Claude Sonnet 5 1,568 high Jun 2026 Official leaderboard
26 DeepSeek V4.1 Flash 1,564 high Sep 2026 Official leaderboard
27 Tencent Hy4 preview 1,561 high Aug 2026 Official leaderboard
28 DeepSeek V4 Pro 0813 1,552 high Aug 2026 Official leaderboard
29 GLM-5.2 1,542 max Jun 2026 Official leaderboard
30 Claude Sonnet 4.6 1,540 high Feb 2026 Official leaderboard
31 Qwen 3.8 Max 1,535 Aug 2026 Official leaderboard
32 GPT-5.4 1,525 high Mar 2026 Official leaderboard
33 Step 5 Preview 1,521 high Sep 2026 Official leaderboard
34 MiniMax-M3.1-Flash 1,509 high Oct 2026 Official leaderboard
35 GPT-5.5 1,507 high Apr 2026 Official leaderboard
36 GLM-5.1 1,498 Apr 2026 Official leaderboard
37 Kimi K2.6 1,490 Apr 2026 Official leaderboard
38 Gemini 3.8 Flash 1,488 high Sep 2026 Official leaderboard
39 MiniMax-M3 1,483 Jun 2026 Official leaderboard
40 Grok 4.5 1,470 high Jul 2026 Official leaderboard
41 Gemini 3.1 Pro 1,468 Feb 2026 Official leaderboard
42 ByteDance Seed2.1 Pro 1,468 Official leaderboard
43 Xiaomi MiMo V2.5 Pro 1,463 Official leaderboard
44 Qwen 3.6 Max (Preview) 1,454 Apr 2026 Official leaderboard
45 Mistral Large 4 1,442 high Oct 2026 Official leaderboard
46 Gemini 3.7 Flash 1,434 high Aug 2026 Official leaderboard
47 ByteDance Seed2.0 Pro 1,434 Official leaderboard
48 Kimi K2.5 1,431 thinking Jan 2026 Official leaderboard
49 DeepSeek-V4-Pro 1,427 Apr 2026 Official leaderboard
50 Qwen3.7-Max 1,414 May 2026 Official leaderboard
51 MiniMax-M2.7 1,412 Mar 2026 Official leaderboard
52 Grok 4.20 0309 1,390 Reasoning Official leaderboard
53 Gemini 3.5 Flash 1,387 May 2026 Official leaderboard
54 Baidu Ernie 5.1 1,370 Official leaderboard
55 Xiaomi MiMo V2 Pro 1,369 Official leaderboard
56 Qwen3.5 397B-A17B 1,368 Feb 2026 Official leaderboard
57 Tencent Hy3 preview 1,366 high Apr 2026 Official leaderboard
58 Step 3.7 Flash 1,355 high May 2026 Official leaderboard
59 Grok 4.3 1,350 Apr 2026 Official leaderboard
60 DeepSeek-V3.2 1,341 Dec 2025 Official leaderboard
61 Mistral Medium 3.5 1,321 high Apr 2026 Official leaderboard
62 Gemini 3.1 Flash-Lite 1,308 Mar 2026 Official leaderboard
63 gpt-oss-120b 1,242 Aug 2025 Official leaderboard
64 Baidu Ernie 5.0 1,225 Official leaderboard
65 Mistral Large 3 1,170 Dec 2025 Official leaderboard
66 Llama 4 Maverick 1,006 Apr 2025 Official leaderboard
#ModelScoreSettingDateSource
1 Claude Fable 5.1 1,706 high Sep 2026 Official leaderboard
2 Claude Fable 5 1,705 high Jun 2026 Official leaderboard
3 Claude Opus 5 1,704 high Jul 2026 Official leaderboard
4 Claude Fable 5.2 1,688 high Nov 2026 Official leaderboard
5 Claude Opus 5.5 1,680 high Sep 2026 Official leaderboard
6 Qwen4-Max 1,679 high Nov 2026 Official leaderboard
7 Kimi K3 1,679 Jul 2026 Official leaderboard
8 Gemini 4 Carbon 1,676 high Oct 2026 Official leaderboard
9 GPT-6.2 Astra 1,673 high Nov 2026 Official leaderboard
10 Gemini 4 Argon 1,649 high Sep 2026 Official leaderboard
11 Tencent Hy4 1,647 high Nov 2026 Official leaderboard
12 Muse Wonder 1,637 high Oct 2026 Official leaderboard
13 Claude Opus 4.7 1,627 high Apr 2026 Official leaderboard
14 GPT-5.6 Sol 1,625 high Jul 2026 Official leaderboard
15 Muse Spark 1.1 1,617 high Jul 2026 Official leaderboard
16 GLM-5.3 1,613 high Aug 2026 Official leaderboard
17 Claude Opus 4.8 1,610 high May 2026 Official leaderboard
18 GPT-6 Astra 1,607 high Sep 2026 Official leaderboard
19 Grok 4.7 1,601 high Sep 2026 Official leaderboard
20 Kimi K3.1 Pro 1,601 high Oct 2026 Official leaderboard
21 Grok 4.6 1,585 high Aug 2026 Official leaderboard
22 DeepSeek V4.1 Flash 1017 1,579 high Oct 2026 Official leaderboard
23 Muse Spark 1.3 1,578 high Sep 2026 Official leaderboard
24 Claude Sonnet 5 1,568 high Jun 2026 Official leaderboard
25 DeepSeek V4.1 Flash 1,564 high Sep 2026 Official leaderboard
26 Tencent Hy4 preview 1,561 high Aug 2026 Official leaderboard
27 DeepSeek V4 Pro 0813 1,552 high Aug 2026 Official leaderboard
28 GLM-5.2 1,542 max Jun 2026 Official leaderboard
29 Claude Sonnet 4.6 1,540 high Feb 2026 Official leaderboard
30 Qwen 3.8 Max 1,535 Aug 2026 Official leaderboard
31 GPT-5.4 1,525 high Mar 2026 Official leaderboard
32 Step 5 Preview 1,521 high Sep 2026 Official leaderboard
33 MiniMax-M3.1-Flash 1,509 high Oct 2026 Official leaderboard
34 GPT-5.5 1,507 high Apr 2026 Official leaderboard
35 GLM-5.1 1,498 Apr 2026 Official leaderboard
36 Kimi K2.6 1,490 Apr 2026 Official leaderboard
37 Gemini 3.8 Flash 1,488 high Sep 2026 Official leaderboard
38 MiniMax-M3 1,483 Jun 2026 Official leaderboard
39 Grok 4.5 1,470 high Jul 2026 Official leaderboard
40 Gemini 3.1 Pro 1,468 Feb 2026 Official leaderboard
41 Qwen 3.6 Max (Preview) 1,454 Apr 2026 Official leaderboard
42 Mistral Large 4 1,442 high Oct 2026 Official leaderboard
43 Gemini 3.7 Flash 1,434 high Aug 2026 Official leaderboard
44 Kimi K2.5 1,431 thinking Jan 2026 Official leaderboard
45 DeepSeek-V4-Pro 1,427 Apr 2026 Official leaderboard
46 Qwen3.7-Max 1,414 May 2026 Official leaderboard
47 MiniMax-M2.7 1,412 Mar 2026 Official leaderboard
48 Gemini 3.5 Flash 1,387 May 2026 Official leaderboard
49 Qwen3.5 397B-A17B 1,368 Feb 2026 Official leaderboard
50 Tencent Hy3 preview 1,366 high Apr 2026 Official leaderboard
51 Step 3.7 Flash 1,355 high May 2026 Official leaderboard
52 Grok 4.3 1,350 Apr 2026 Official leaderboard
53 DeepSeek-V3.2 1,341 Dec 2025 Official leaderboard
54 Mistral Medium 3.5 1,321 high Apr 2026 Official leaderboard
55 Gemini 3.1 Flash-Lite 1,308 Mar 2026 Official leaderboard
56 gpt-oss-120b 1,242 Aug 2025 Official leaderboard
57 Mistral Large 3 1,170 Dec 2025 Official leaderboard
58 Llama 4 Maverick 1,006 Apr 2025 Official leaderboard

The best score, 1,706, costs $0.39 a task

Runs on the cost frontier score higher than any run that's cheaper. Qwen4-Max at high comes within 26.7 of the best for $0.07 a task, 5 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Anthropic DeepSeek Qwen Cost frontier One model at each effort level
0 500 1,000 1,500 2,000 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Claude Fable 5.1, high Qwen4-Max, high DeepSeek V4.1 Flash 1017, high 0 500 1,000 1,500 2,000 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Claude Fable 5.1, high DeepSeek V4.1 Flash 1017, high

Source: Official LLM Debate Benchmark leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5.1 high 1,706 $0.39
Claude Fable 5 high 1,705
Claude Opus 5 high 1,704
Claude Fable 5.2 high 1,688 $0.30
Claude Opus 5.5 high 1,680 $0.15
Qwen4-Max high 1,679 $0.07
Kimi K3 1,679
Gemini 4 Carbon high 1,676 $0.41
GPT-6.2 Astra high 1,673 $0.32
Gemini 4 Argon high 1,649 $0.20
Tencent Hy4 high 1,647 $0.07
Muse Wonder high 1,637
Claude Opus 4.7 high 1,627
GPT-5.6 Sol high 1,625
Muse Spark 1.1 high 1,617
GLM-5.3 high 1,613
Claude Opus 4.8 high 1,610
GPT-6 Astra high 1,607
Grok 4.7 high 1,601
Kimi K3.1 Pro high 1,601
Grok 4.6 high 1,585
DeepSeek V4.1 Flash 1017 high 1,579 $0.006
Muse Spark 1.3 high 1,578
Claude Sonnet 5 high 1,568
DeepSeek V4.1 Flash high 1,564
Tencent Hy4 preview high 1,561
DeepSeek V4 Pro 0813 high 1,552
GLM-5.2 max 1,542
Claude Sonnet 4.6 high 1,540
Qwen 3.8 Max 1,535
GPT-5.4 high 1,525
Step 5 Preview high 1,521 $0.05
Claude Sonnet 4.6 no reasoning 1,520
MiniMax-M3.1-Flash high 1,509 $0.009
GPT-5.5 high 1,507
GLM-5.1 1,498
Kimi K2.6 1,490
Gemini 3.8 Flash high 1,488
MiniMax-M3 1,483
GPT-5.4 no reasoning 1,473
Grok 4.5 high 1,470
Gemini 3.1 Pro 1,468
Qwen 3.6 Max (Preview) 1,454
Mistral Large 4 high 1,442
Gemini 3.7 Flash high 1,434
Kimi K2.5 thinking 1,431
DeepSeek-V4-Pro 1,427
Qwen3.7-Max 1,414
MiniMax-M2.7 1,412
Gemini 3.5 Flash 1,387
Qwen3.5 397B-A17B 1,368
Tencent Hy3 preview high 1,366
Step 3.7 Flash high 1,355
Grok 4.3 1,350
DeepSeek-V3.2 1,341
Mistral Medium 3.5 high 1,321
Gemini 3.1 Flash-Lite 1,308
gpt-oss-120b 1,242
Mistral Large 3 1,170
Llama 4 Maverick 1,006

The best score, 1,706, costs $0.39 a task

Runs on the cost frontier score higher than any run that's cheaper. Qwen4-Max at high comes within 26.7 of the best for $0.07 a task, 5 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Anthropic DeepSeek Qwen Cost frontier One model at each effort level
0 500 1,000 1,500 2,000 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Claude Fable 5.1, high Qwen4-Max, high DeepSeek V4.1 Flash 1017, high 0 500 1,000 1,500 2,000 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Claude Fable 5.1, high DeepSeek V4.1 Flash 1017, high

Source: Official LLM Debate Benchmark leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5.1 high 1,706 $0.39
Claude Fable 5 high 1,705
Claude Opus 5 high 1,704
Claude Fable 5.2 high 1,688 $0.30
Claude Opus 5.5 high 1,680 $0.15
Qwen4-Max high 1,679 $0.07
Kimi K3 1,679
Gemini 4 Carbon high 1,676 $0.41
GPT-6.2 Astra high 1,673 $0.32
Gemini 4 Argon high 1,649 $0.20
Tencent Hy4 high 1,647 $0.07
Muse Wonder high 1,637
Claude Opus 4.7 high 1,627
GPT-5.6 Sol high 1,625
Muse Spark 1.1 high 1,617
GLM-5.3 high 1,613
Claude Opus 4.8 high 1,610
GPT-6 Astra high 1,607
Grok 4.7 high 1,601
Kimi K3.1 Pro high 1,601
Grok 4.6 high 1,585
DeepSeek V4.1 Flash 1017 high 1,579 $0.006
Muse Spark 1.3 high 1,578
Claude Sonnet 5 high 1,568
DeepSeek V4.1 Flash high 1,564
Tencent Hy4 preview high 1,561
DeepSeek V4 Pro 0813 high 1,552
GLM-5.2 max 1,542
Claude Sonnet 4.6 high 1,540
Qwen 3.8 Max 1,535
GPT-5.4 high 1,525
Step 5 Preview high 1,521 $0.05
Claude Sonnet 4.6 no reasoning 1,520
MiniMax-M3.1-Flash high 1,509 $0.009
GPT-5.5 high 1,507
GLM-5.1 1,498
Kimi K2.6 1,490
Gemini 3.8 Flash high 1,488
MiniMax-M3 1,483
GPT-5.4 no reasoning 1,473
Grok 4.5 high 1,470
Gemini 3.1 Pro 1,468
Qwen 3.6 Max (Preview) 1,454
Mistral Large 4 high 1,442
Gemini 3.7 Flash high 1,434
Kimi K2.5 thinking 1,431
DeepSeek-V4-Pro 1,427
Qwen3.7-Max 1,414
MiniMax-M2.7 1,412
Gemini 3.5 Flash 1,387
Qwen3.5 397B-A17B 1,368
Tencent Hy3 preview high 1,366
Step 3.7 Flash high 1,355
Grok 4.3 1,350
DeepSeek-V3.2 1,341
Mistral Medium 3.5 high 1,321
Gemini 3.1 Flash-Lite 1,308
gpt-oss-120b 1,242
Mistral Large 3 1,170
Llama 4 Maverick 1,006

8 models have held number one on the official leaderboard since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Meta OpenAI DeepSeek Moonshot AI Anthropic
Own leaderboard 8 leaders

Source: Official LLM Debate Benchmark leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
1,000 1,200 1,400 1,600 1,800 2025 2026 2027 1,000 1,200 1,400 1,600 1,800 2025 2026 2027

Source: Official LLM Debate Benchmark leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Instructions and chat

All 144 in Instructions and chat
  • IFEval In use Models answer about 500 prompts containing verifiable instructions such as word limits or required keywords. Code checks each instruction for strict and loose accuracy. Gemma 4 31B IT 98.9%
  • IFBench In use 58 new out-of-domain verifiable output constraints. Scored by programmatic constraint checks (strict or loose). Flint 89.2%
  • MultiChallenge In use Multi-turn conversations testing instruction retention, inference memory, versioned editing and self-coherence. Graded by LLM judge with rubrics. GPT-6.1 Sol 82%
  • Multi-IF In use Models follow verifiable instructions over three-turn conversations in eight languages, extending IFEval. Scored by instruction accuracy at each turn. LFM3-1.7B 89.1%
  • AdvancedIF In use Models respond to over 1,600 prompts with complex, multi-turn, and system-level instructions. A judge model checks each response against expert-written rubrics. MAI-Thinking-1 85%
  • MM-IFEval In use Multimodal models follow image-grounded instructions with exact output constraints. Checks mix rule-based verification and a judge model. GPT-4o 64.6%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (66 entries).
  • Pass Models scored 58 models we track, plus 8 other entries on its leaderboard.
  • Pass Newest score The newest is from November 2026, for GPT-6.2 Astra.
  • Pass Frontier models 7 of the top 10 have a score, led by GPT-6.2 Astra (#1).
  • Note Headroom Scored as a rating, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on github.com: 60 rows for models we track and 8 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check