Benchmark Almanac

LMCA

Claude Fable 5.2 70.6% leads since November 2026
Not in model cards

Models rate arguments against position texts on philosophy, decision theory, and AI risk. Scored by agreement with expert ratings, with a ceiling near 85.

Same benchmark as the lmca record, from the same arXiv paper 2607.27499.

Released
July 2026
Kind
Other
Level
Expert
Scored
Checked automatically
Answers
Short answer
Tasks
Kept private
Strongest model tested
Claude Fable 5.2 #1 on the capability index
Last model added
November 2026
Scores refreshed
November 12th, 2026

In the authors’ words

Large language models have improved rapidly on tasks with verifiable answers, such as mathematics and programming. Much less is known about their ability to reason about what we call conceptual questions: questions for which no ground truth is realistically accessible and no widely accepted resolution methodology exists, but on which progress can still be made by debating arguments. Most philosophical questions are of this kind, as are central components of questions in AI safety, decision theory, and social choice. Our approach is based on the view that while bottom-line conclusions on such questions are hard to evaluate, individual contextualized arguments can be evaluated far more reliably. We therefore introduce a dataset of 951 argumentative critiques of 442 position texts, spanning topics from AI safety and decision theory to ethics and politics, with 1,458 ratings by six expert raters along dimensions including centrality, strength, correctness, and clarity. We propose two scoring functions and benchmark a range of models. Performance tracks general capability rankings.

Abstract of A dataset of rated conceptual arguments (arXiv:2607.27499, CC0)
155
Entries on its own leaderboard, 147 models we track

Claude Fable 5.2 leads with 70.6%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

155 entries. Scroll sideways for the rest.

  1. 70.6 Claude Fable 5.2
  2. 68.3 Claude Sonnet 5.5
  3. 68.2 Claude Opus 5.5
  4. 65.5 Claude Fable 5.1
  5. 64.5 Claude Opus 5
  6. 64.4 GPT-6 Astra
  7. 63.7 Gemini 4 Argon
  8. 62.8 GPT-6.1 Sol
  9. 61.1 Claude Fable 5
  10. 59.2 GPT-5.6 Sol Pro
  11. 59.1 GPT-6 Sol
  12. 58.5 GPT-5.6 Sol
  13. 57.6 Kimi K3.1 Pro
  14. 57.5 Claude Opus 4.8
  15. 56.8 Muse Wonder
  16. 55.8 Claude Opus 4.6
  17. 55.5 GLM-5.3
  18. 55.0 GPT-5.6 Terra
  19. 54.3 GPT-5.5
  20. 53.9 GPT-5.5 Pro
  21. 53.9 Muse Spark 1.3
  22. 53.8 Gemini 3.1 Pro
  23. 52.9 Gemini 3.8 Flash
  24. 52.7 Kimi K3
  25. 52.2 Claude Opus 4.7
  26. 52.0 GPT-5.4
  27. 51.1 Claude Haiku 5.5
  28. 50.4 Gemini 3.7 Flash
  29. 50.0 Claude Sonnet 5
  30. 49.9 Muse Spark 1.1
  31. 49.4 Grok 4.7
  32. 48.5 GPT-5.6 Luna
  33. 48.5 Grok 4.6
  34. 48.4 Muse Spark 1.2
  35. 47.1 Gemini 3.5 Flash
  36. 47.0 DeepSeek V4.1 Flash
  37. 46.5 Claude Sonnet 4.6
  38. 46.2 Qwen 3.8 Max
  39. 45.8 GLM-5.2
  40. 45.5 Qwen3.8-2.4T-A95B
  41. 45.5 DeepSeek V4 Pro 0813
  42. 45.2 Grok 4.5
  43. 44.9 Gemini 3.6 Flash
  44. 44.5 GPT-6 Luna
  45. 44.5 Claude Opus 4.5
  46. 44.0 Qwen3.7-Max
  47. 43.9 GPT-5.2
  48. 43.9 GPT-5.1
  49. 43.1 Gemini 3 Flash
  50. 42.5 Qwen 3.6 Max (Preview)
  51. 41.7 DeepSeek V4 Flash 0731
  52. 41.4 Grok 4.20 Multi-agent
  53. 41.4 Qwen 3.8 27B
  54. 41.2 DeepSeek-V4-Pro
  55. 40.8 GPT-5.4 Mini
  56. 40.1 GPT-5.2 Instant
  57. 40.0 GPT-5
  58. 39.7 o3
  59. 39.3 Gemma 4 31B IT
  60. 38.8 Claude Sonnet 4.5
  61. 38.7 Grok 4.20
  62. 38.5 o3-pro
  63. 38.3 Grok 4.3
  64. 38.1 GPT-5.3 Instant
  65. 37.9 Qwen3.5 397B-A17B
  66. 37.6 Gemini 3.5 Flash-Lite
  67. 37.6 Qwen3.7-Plus
  68. 37.6 Inkling
  69. 37.5 Tencent Hy3
  70. 37.4 Claude Opus 4
  71. 37.3 Kimi K2.6
  72. 37.1 Claude Opus 4.1
  73. 36.9 Nemotron 3 Ultra
  74. 36.9 GPT-5.4 Nano
  75. 36.4 Qwen 3.5 Plus (hosted 397B-A17B)
  76. 35.9 DeepSeek-V4-Flash
  77. 35.8 Mistral Large 4
  78. 35.0 Gemini 3.1 Flash-Lite
  79. 34.9 Muse Glimmer
  80. 34.8 Gemini 2.5 Pro (snapshot unspecified)
  81. 34.5 Qwen3.6 27B
  82. 34.2 GPT-5 mini
  83. 34.0 Qwen3.5-27B
  84. 33.7 MiniMax-M3
  85. 33.6 Qwen3-Max
  86. 33.1 Qwen 3.6 Plus
  87. 32.2 Qwen3.5-122B-A10B
  88. 31.0 Qwen 3.6 Flash
  89. 30.9 Claude Haiku 4.5
  90. 29.7 Qwen 3.6 35B-A3B
  91. 29.7 Gemma 4 26B A4B
  92. 29.5 MiMo-V2.5-Pro
  93. 29.5 Qwen3.5-35B-A3B
  94. 29.3 Qwen3-235B-A22B-Thinking
  95. 29.1 Qwen 3.5 Flash (hosted 35B-A3B)
  96. 29.1 DeepSeek-V3.2-Exp
  97. 29.0 Claude Sonnet 4
  98. 28.8 DeepSeek-V3.2
  99. 28.6 DeepSeek-V3.1-Terminus
  100. 27.5 Gemini 2.5 Flash (snapshot unspecified)
  101. 26.5 o4-mini
  102. 26.1 Mistral Medium 3.5
  103. 25.6 GPT-4.1
  104. 25.0 Qwen3-235B-A22B
  105. 24.5 Qwen3.5-9B
  106. 24.3 DeepSeek-V3.1
  107. 24.0 Qwen Plus
  108. 23.5 Qwen3 235B-A22B (2507)
  109. 22.4 Qwen3-30B-A3B-Instruct
  110. 22.3 o1
  111. 22.1 gpt-oss-120b
  112. 21.8 Qwen3-Next-80B-A3B
  113. 21.1 GPT-4.1 mini
  114. 20.9 Nemotron 3.5 Lightning 30B-A3B
  115. 20.6 Mistral Small 4
  116. 19.5 Qwen3-30B-A3B-Thinking
  117. 19.1 Mistral Medium 3.1
  118. 19.0 o3-mini
  119. 18.2 Qwen3-14B
  120. 18.1 Gemini 2.5 Flash-Lite
  121. 17.5 Llama 3.3 70B
  122. 17.3 Qwen3-32B
  123. 17.1 GPT-4 (snapshot unspecified)
  124. 17.0 Claude 3 Opus
  125. 16.7 Mistral Large 3
  126. 16.6 GPT-4o (snapshot unspecified)
  127. 15.9 Llama 4 Maverick
  128. 15.8 Qwen3-30B-A3B
  129. 15.5 DeepSeek-V3
  130. 15.2 DeepSeek Chat
  131. 14.8 Llama 3.1-70B
  132. 14.5 gpt-oss-20b
  133. 13.4 Qwen2.5-72B
  134. 12.7 Ministral 3 8B Instruct
  135. 12.3 Ministral 3 14B
  136. 12.3 Gemma 3 27B
  137. 12.0 Llama 4 Scout
  138. 10.4 GPT-4o mini (snapshot unspecified)
  139. 10.3 Cohere Command A
  140. 9.78 GPT-4 Turbo (snapshot unspecified)
  141. 9.65 GPT-3.5 Turbo
  142. 9.19 c4ai-command-r-08-2024
  143. 8.84 Claude 3 Haiku
  144. 8.82 Qwen3-8B
  145. 7.93 GPT-5 nano
  146. 7.10 Gemma 2 27B
  147. 6.42 Qwen2.5-7B
  148. 5.48 Ministral 3B
  149. 5.46 GPT-4.1 nano
  150. 5.37 Llama 3.1-8B
  151. 5.01 Command R+
  152. 4.46 Gemma 3 12B
  153. 4.45 Command R7B
  154. 3.16 Gemma 3n E4B
  155. 2.78 Gemma 3 4B
  1. 1 Claude Fable 5.2 xhigh 70.6%
  2. 2 Claude Sonnet 5.5 max 68.3%
  3. 3 Claude Opus 5.5 max 68.2%
  4. 4 Claude Fable 5.1 max 65.5%
  5. 5 Claude Opus 5 xhigh 64.5%
  6. 6 GPT-6 Astra xhigh 64.4%
  7. 7 Gemini 4 Argon high 63.7%
  8. 8 GPT-6.1 Sol 62.8%
  9. 9 Claude Fable 5 high 61.1%
  10. 10 GPT-5.6 Sol Pro 59.2%
  11. 11 GPT-6 Sol 59.1%
  12. 12 GPT-5.6 Sol xhigh 58.5%
  13. 13 Kimi K3.1 Pro 57.6%
  14. 14 Claude Opus 4.8 57.5%
  15. 15 Muse Wonder 56.8%
  16. 16 Claude Opus 4.6 55.8%
  17. 17 GLM-5.3 55.5%
  18. 18 GPT-5.6 Terra max 55%
  19. 19 GPT-5.5 54.3%
  20. 20 GPT-5.5 Pro 53.9%
  21. 21 Muse Spark 1.3 53.9%
  22. 22 Gemini 3.1 Pro high 53.8%
  23. 23 Gemini 3.8 Flash medium 52.9%
  24. 24 Kimi K3 52.7%
  25. 25 Claude Opus 4.7 52.2%
  26. 26 GPT-5.4 52%
  27. 27 Claude Haiku 5.5 max 51.1%
  28. 28 Gemini 3.7 Flash high 50.4%
  29. 29 Claude Sonnet 5 high 50%
  30. 30 Muse Spark 1.1 49.9%
  31. 31 Grok 4.7 49.4%
  32. 32 GPT-5.6 Luna xhigh 48.5%
  33. 33 Grok 4.6 48.5%
  34. 34 Muse Spark 1.2 48.4%
  35. 35 Gemini 3.5 Flash 47.1%
  36. 36 DeepSeek V4.1 Flash 47%
  37. 37 Claude Sonnet 4.6 46.5%
  38. 38 Qwen 3.8 Max 46.2%
  39. 39 GLM-5.2 45.8%
  40. 40 Qwen3.8-2.4T-A95B 45.5%
  41. 41 DeepSeek V4 Pro 0813 45.5%
  42. 42 Grok 4.5 45.2%
  43. 43 Gemini 3.6 Flash 44.9%
  44. 44 GPT-6 Luna 44.5%
  45. 45 Claude Opus 4.5 44.5%
  46. 46 Qwen3.7-Max 44%
  47. 47 GPT-5.2 43.9%
  48. 48 GPT-5.1 43.9%
  49. 49 Gemini 3 Flash 43.1%
  50. 50 Qwen 3.6 Max (Preview) 42.5%
  51. 51 DeepSeek V4 Flash 0731 41.7%
  52. 52 Grok 4.20 Multi-agent 41.4%
  53. 53 Qwen 3.8 27B 41.4%
  54. 54 DeepSeek-V4-Pro 41.2%
  55. 55 GPT-5.4 Mini 40.8%
  56. 56 GPT-5.2 Instant 40.1%
  57. 57 GPT-5 40%
  58. 58 o3 39.7%
  59. 59 Gemma 4 31B IT 39.3%
  60. 60 Claude Sonnet 4.5 38.8%
  61. 61 Grok 4.20 38.7%
  62. 62 o3-pro 38.5%
  63. 63 Grok 4.3 38.3%
  64. 64 GPT-5.3 Instant 38.1%
  65. 65 Qwen3.5 397B-A17B 37.9%
  66. 66 Gemini 3.5 Flash-Lite 37.6%
  67. 67 Qwen3.7-Plus 37.6%
  68. 68 Inkling 37.6%
  69. 69 Tencent Hy3 37.5%
  70. 70 Claude Opus 4 37.4%
  71. 71 Kimi K2.6 37.3%
  72. 72 Claude Opus 4.1 37.1%
  73. 73 Nemotron 3 Ultra 36.9%
  74. 74 GPT-5.4 Nano 36.9%
  75. 75 Qwen 3.5 Plus (hosted 397B-A17B) 36.4%
  76. 76 DeepSeek-V4-Flash 35.9%
  77. 77 Mistral Large 4 35.8%
  78. 78 Gemini 3.1 Flash-Lite 35%
  79. 79 Muse Glimmer 34.9%
  80. 80 Gemini 2.5 Pro (snapshot unspecified) 34.8%
  81. 81 Qwen3.6 27B 34.5%
  82. 82 GPT-5 mini 34.2%
  83. 83 Qwen3.5-27B 34%
  84. 84 MiniMax-M3 33.7%
  85. 85 Qwen3-Max thinking 33.6%
  86. 86 Qwen 3.6 Plus 33.1%
  87. 87 Qwen3.5-122B-A10B 32.2%
  88. 88 Qwen 3.6 Flash 31%
  89. 89 Claude Haiku 4.5 30.9%
  90. 90 Qwen 3.6 35B-A3B 29.7%
  91. 91 Gemma 4 26B A4B 29.7%
  92. 92 MiMo-V2.5-Pro 29.5%
  93. 93 Qwen3.5-35B-A3B 29.5%
  94. 94 Qwen3-235B-A22B-Thinking Jul 2025 29.3%
  95. 95 Qwen 3.5 Flash (hosted 35B-A3B) 29.1%
  96. 96 DeepSeek-V3.2-Exp 29.1%
  97. 97 Claude Sonnet 4 29%
  98. 98 DeepSeek-V3.2 28.8%
  99. 99 DeepSeek-V3.1-Terminus 28.6%
  100. 100 Gemini 2.5 Flash (snapshot unspecified) 27.5%
  101. 101 o4-mini 26.5%
  102. 102 Mistral Medium 3.5 26.1%
  103. 103 GPT-4.1 25.6%
  104. 104 Qwen3-235B-A22B 25%
  105. 105 Qwen3.5-9B 24.5%
  106. 106 DeepSeek-V3.1 24.3%
  107. 107 Qwen Plus Apr 2025 24%
  108. 108 Qwen3 235B-A22B (2507) 23.5%
  109. 109 Qwen3-30B-A3B-Instruct Jul 2025 22.4%
  110. 110 o1 22.3%
  111. 111 gpt-oss-120b 22.1%
  112. 112 Qwen3-Next-80B-A3B thinking 21.8%
  113. 113 GPT-4.1 mini 21.1%
  114. 114 Nemotron 3.5 Lightning 30B-A3B 20.9%
  115. 115 Mistral Small 4 20.6%
  116. 116 Qwen3-30B-A3B-Thinking Jul 2025 19.5%
  117. 117 Mistral Medium 3.1 19.1%
  118. 118 o3-mini 19%
  119. 119 Qwen3-14B 18.2%
  120. 120 Gemini 2.5 Flash-Lite Sep 2025 18.1%
  121. 121 Llama 3.3 70B 17.5%
  122. 122 Qwen3-32B 17.3%
  123. 123 GPT-4 (snapshot unspecified) 17.1%
  124. 124 Claude 3 Opus 17%
  125. 125 Mistral Large 3 16.7%
  126. 126 GPT-4o (snapshot unspecified) 16.6%
  127. 127 Llama 4 Maverick 15.9%
  128. 128 Qwen3-30B-A3B 15.8%
  129. 129 DeepSeek-V3 Mar 2025 15.5%
  130. 130 DeepSeek Chat 15.2%
  131. 131 Llama 3.1-70B 14.8%
  132. 132 gpt-oss-20b 14.5%
  133. 133 Qwen2.5-72B 13.4%
  134. 134 Ministral 3 8B Instruct 12.7%
  135. 135 Ministral 3 14B 12.3%
  136. 136 Gemma 3 27B 12.3%
  137. 137 Llama 4 Scout 12%
  138. 138 GPT-4o mini (snapshot unspecified) 10.4%
  139. 139 Cohere Command A 10.3%
  140. 140 GPT-4 Turbo (snapshot unspecified) 9.78%
  141. 141 GPT-3.5 Turbo Jan 2024 9.65%
  142. 142 c4ai-command-r-08-2024 9.19%
  143. 143 Claude 3 Haiku 8.84%
  144. 144 Qwen3-8B 8.82%
  145. 145 GPT-5 nano 7.93%
  146. 146 Gemma 2 27B 7.10%
  147. 147 Qwen2.5-7B 6.42%
  148. 148 Ministral 3B 5.48%
  149. 149 GPT-4.1 nano 5.46%
  150. 150 Llama 3.1-8B 5.37%
  151. 151 Command R+ 5.01%
  152. 152 Gemma 3 12B 4.46%
  153. 153 Command R7B 4.45%
  154. 154 Gemma 3n E4B 3.16%
  155. 155 Gemma 3 4B 2.78%

Source: Official LMCA leaderboard

#EntryScoreSettingDateSource
1 Claude Fable 5.2 70.6% xhigh Nov 2026 Official leaderboard
2 Claude Sonnet 5.5 68.3% max Sep 2026 Official leaderboard
3 Claude Opus 5.5 68.2% max Sep 2026 Official leaderboard
4 Claude Fable 5.1 65.5% max Sep 2026 Official leaderboard
5 Claude Opus 5 64.5% xhigh Jul 2026 Official leaderboard
6 GPT-6 Astra 64.4% xhigh Sep 2026 Official leaderboard
7 Gemini 4 Argon 63.7% high Sep 2026 Official leaderboard
8 GPT-6.1 Sol 62.8% Sep 2026 Official leaderboard
9 Claude Fable 5 61.1% high Jun 2026 Official leaderboard
10 GPT-5.6 Sol Pro 59.2% Jul 2026 Official leaderboard
11 GPT-6 Sol 59.1% Sep 2026 Official leaderboard
12 GPT-5.6 Sol 58.5% xhigh Jul 2026 Official leaderboard
13 Kimi K3.1 Pro 57.6% Oct 2026 Official leaderboard
14 Claude Opus 4.8 57.5% May 2026 Official leaderboard
15 Muse Wonder 56.8% Oct 2026 Official leaderboard
16 Claude Opus 4.6 55.8% Feb 2026 Official leaderboard
17 GLM-5.3 55.5% Aug 2026 Official leaderboard
18 GPT-5.6 Terra 55% max Jul 2026 Official leaderboard
19 GPT-5.5 54.3% Apr 2026 Official leaderboard
20 GPT-5.5 Pro 53.9% Apr 2026 Official leaderboard
21 Muse Spark 1.3 53.9% Sep 2026 Official leaderboard
22 Gemini 3.1 Pro 53.8% high Feb 2026 Official leaderboard
23 Gemini 3.8 Flash 52.9% medium Sep 2026 Official leaderboard
24 Kimi K3 52.7% Jul 2026 Official leaderboard
25 Claude Opus 4.7 52.2% Apr 2026 Official leaderboard
26 GPT-5.4 52% Mar 2026 Official leaderboard
27 Claude Haiku 5.5 51.1% max Oct 2026 Official leaderboard
28 Gemini 3.7 Flash 50.4% high Aug 2026 Official leaderboard
29 Claude Sonnet 5 50% high Jun 2026 Official leaderboard
30 Muse Spark 1.1 49.9% Jul 2026 Official leaderboard
31 Grok 4.7 49.4% Sep 2026 Official leaderboard
32 GPT-5.6 Luna 48.5% xhigh Jul 2026 Official leaderboard
33 Grok 4.6 48.5% Aug 2026 Official leaderboard
34 Muse Spark 1.2 48.4% Aug 2026 Official leaderboard
35 Gemini 3.5 Flash 47.1% May 2026 Official leaderboard
36 DeepSeek V4.1 Flash 47% Sep 2026 Official leaderboard
37 Claude Sonnet 4.6 46.5% Feb 2026 Official leaderboard
38 Qwen 3.8 Max 46.2% Aug 2026 Official leaderboard
39 GLM-5.2 45.8% Jun 2026 Official leaderboard
40 Qwen3.8-2.4T-A95B 45.5% Aug 2026 Official leaderboard
41 DeepSeek V4 Pro 0813 45.5% Aug 2026 Official leaderboard
42 Grok 4.5 45.2% Jul 2026 Official leaderboard
43 Gemini 3.6 Flash 44.9% Jul 2026 Official leaderboard
44 GPT-6 Luna 44.5% Sep 2026 Official leaderboard
45 Claude Opus 4.5 44.5% Nov 2025 Official leaderboard
46 Qwen3.7-Max 44% May 2026 Official leaderboard
47 GPT-5.2 43.9% Dec 2025 Official leaderboard
48 GPT-5.1 43.9% Nov 2025 Official leaderboard
49 Gemini 3 Flash 43.1% Dec 2025 Official leaderboard
50 Qwen 3.6 Max (Preview) 42.5% Apr 2026 Official leaderboard
51 DeepSeek V4 Flash 0731 41.7% Jul 2026 Official leaderboard
52 Grok 4.20 Multi-agent 41.4% Mar 2026 Official leaderboard
53 Qwen 3.8 27B 41.4% Aug 2026 Official leaderboard
54 DeepSeek-V4-Pro 41.2% Apr 2026 Official leaderboard
55 GPT-5.4 Mini 40.8% Mar 2026 Official leaderboard
56 GPT-5.2 Instant 40.1% Dec 2025 Official leaderboard
57 GPT-5 40% Aug 2025 Official leaderboard
58 o3 39.7% Apr 2025 Official leaderboard
59 Gemma 4 31B IT 39.3% Apr 2026 Official leaderboard
60 Claude Sonnet 4.5 38.8% Sep 2025 Official leaderboard
61 Grok 4.20 38.7% Feb 2026 Official leaderboard
62 o3-pro 38.5% Jun 2025 Official leaderboard
63 Grok 4.3 38.3% Apr 2026 Official leaderboard
64 GPT-5.3 Instant 38.1% Mar 2026 Official leaderboard
65 Qwen3.5 397B-A17B 37.9% Feb 2026 Official leaderboard
66 Gemini 3.5 Flash-Lite 37.6% Jul 2026 Official leaderboard
67 Qwen3.7-Plus 37.6% Jun 2026 Official leaderboard
68 Inkling 37.6% Jul 2026 Official leaderboard
69 Tencent Hy3 37.5% Jul 2026 Official leaderboard
70 Claude Opus 4 37.4% May 2025 Official leaderboard
71 Kimi K2.6 37.3% Apr 2026 Official leaderboard
72 Claude Opus 4.1 37.1% Aug 2025 Official leaderboard
73 Nemotron 3 Ultra 36.9% Jun 2026 Official leaderboard
74 GPT-5.4 Nano 36.9% Mar 2026 Official leaderboard
75 Qwen 3.5 Plus (hosted 397B-A17B) 36.4% Feb 2026 Official leaderboard
76 DeepSeek-V4-Flash 35.9% Apr 2026 Official leaderboard
77 Mistral Large 4 35.8% Oct 2026 Official leaderboard
78 Gemini 3.1 Flash-Lite 35% Mar 2026 Official leaderboard
79 Muse Glimmer 34.9% Aug 2026 Official leaderboard
80 Gemini 2.5 Pro (snapshot unspecified) 34.8% Official leaderboard
81 Qwen3.6 27B 34.5% Apr 2026 Official leaderboard
82 GPT-5 mini 34.2% Aug 2025 Official leaderboard
83 Qwen3.5-27B 34% Feb 2026 Official leaderboard
84 MiniMax-M3 33.7% Jun 2026 Official leaderboard
85 Qwen3-Max 33.6% thinking Sep 2025 Official leaderboard
86 Qwen 3.6 Plus 33.1% Mar 2026 Official leaderboard
87 Qwen3.5-122B-A10B 32.2% Feb 2026 Official leaderboard
88 Qwen 3.6 Flash 31% Apr 2026 Official leaderboard
89 Claude Haiku 4.5 30.9% Oct 2025 Official leaderboard
90 Qwen 3.6 35B-A3B 29.7% Apr 2026 Official leaderboard
91 Gemma 4 26B A4B 29.7% Apr 2026 Official leaderboard
92 MiMo-V2.5-Pro 29.5% Apr 2026 Official leaderboard
93 Qwen3.5-35B-A3B 29.5% Feb 2026 Official leaderboard
94 Qwen3-235B-A22B-Thinking Jul 2025 29.3% Jul 2025 Official leaderboard
95 Qwen 3.5 Flash (hosted 35B-A3B) 29.1% Feb 2026 Official leaderboard
96 DeepSeek-V3.2-Exp 29.1% Sep 2025 Official leaderboard
97 Claude Sonnet 4 29% May 2025 Official leaderboard
98 DeepSeek-V3.2 28.8% Dec 2025 Official leaderboard
99 DeepSeek-V3.1-Terminus 28.6% Sep 2025 Official leaderboard
100 Gemini 2.5 Flash (snapshot unspecified) 27.5% Official leaderboard
101 o4-mini 26.5% Apr 2025 Official leaderboard
102 Mistral Medium 3.5 26.1% Apr 2026 Official leaderboard
103 GPT-4.1 25.6% Apr 2025 Official leaderboard
104 Qwen3-235B-A22B 25% Apr 2025 Official leaderboard
105 Qwen3.5-9B 24.5% Feb 2026 Official leaderboard
106 DeepSeek-V3.1 24.3% Aug 2025 Official leaderboard
107 Qwen Plus Apr 2025 24% Apr 2025 Official leaderboard
108 Qwen3 235B-A22B (2507) 23.5% Official leaderboard
109 Qwen3-30B-A3B-Instruct Jul 2025 22.4% Jul 2025 Official leaderboard
110 o1 22.3% Dec 2024 Official leaderboard
111 gpt-oss-120b 22.1% Aug 2025 Official leaderboard
112 Qwen3-Next-80B-A3B 21.8% thinking Sep 2025 Official leaderboard
113 GPT-4.1 mini 21.1% Apr 2025 Official leaderboard
114 Nemotron 3.5 Lightning 30B-A3B 20.9% Aug 2026 Official leaderboard
115 Mistral Small 4 20.6% Mar 2026 Official leaderboard
116 Qwen3-30B-A3B-Thinking Jul 2025 19.5% Jul 2025 Official leaderboard
117 Mistral Medium 3.1 19.1% Aug 2025 Official leaderboard
118 o3-mini 19% Jan 2025 Official leaderboard
119 Qwen3-14B 18.2% Apr 2025 Official leaderboard
120 Gemini 2.5 Flash-Lite Sep 2025 18.1% Sep 2025 Official leaderboard
121 Llama 3.3 70B 17.5% Dec 2024 Official leaderboard
122 Qwen3-32B 17.3% Apr 2025 Official leaderboard
123 GPT-4 (snapshot unspecified) 17.1% Official leaderboard
124 Claude 3 Opus 17% Feb 2024 Official leaderboard
125 Mistral Large 3 16.7% Dec 2025 Official leaderboard
126 GPT-4o (snapshot unspecified) 16.6% Official leaderboard
127 Llama 4 Maverick 15.9% Apr 2025 Official leaderboard
128 Qwen3-30B-A3B 15.8% Apr 2025 Official leaderboard
129 DeepSeek-V3 Mar 2025 15.5% Mar 2025 Official leaderboard
130 DeepSeek Chat 15.2% Official leaderboard
131 Llama 3.1-70B 14.8% Jul 2024 Official leaderboard
132 gpt-oss-20b 14.5% Aug 2025 Official leaderboard
133 Qwen2.5-72B 13.4% Sep 2024 Official leaderboard
134 Ministral 3 8B Instruct 12.7% Dec 2025 Official leaderboard
135 Ministral 3 14B 12.3% Dec 2025 Official leaderboard
136 Gemma 3 27B 12.3% Mar 2025 Official leaderboard
137 Llama 4 Scout 12% Apr 2025 Official leaderboard
138 GPT-4o mini (snapshot unspecified) 10.4% Official leaderboard
139 Cohere Command A 10.3% Mar 2025 Official leaderboard
140 GPT-4 Turbo (snapshot unspecified) 9.78% Official leaderboard
141 GPT-3.5 Turbo Jan 2024 9.65% Jan 2024 Official leaderboard
142 c4ai-command-r-08-2024 9.19% Aug 2024 Official leaderboard
143 Claude 3 Haiku 8.84% Mar 2024 Official leaderboard
144 Qwen3-8B 8.82% Apr 2025 Official leaderboard
145 GPT-5 nano 7.93% Aug 2025 Official leaderboard
146 Gemma 2 27B 7.10% Jun 2024 Official leaderboard
147 Qwen2.5-7B 6.42% Sep 2024 Official leaderboard
148 Ministral 3B 5.48% Oct 2024 Official leaderboard
149 GPT-4.1 nano 5.46% Apr 2025 Official leaderboard
150 Llama 3.1-8B 5.37% Jul 2024 Official leaderboard
151 Command R+ 5.01% Aug 2024 Official leaderboard
152 Gemma 3 12B 4.46% Mar 2025 Official leaderboard
153 Command R7B 4.45% Dec 2024 Official leaderboard
154 Gemma 3n E4B 3.16% Jun 2025 Official leaderboard
155 Gemma 3 4B 2.78% Mar 2025 Official leaderboard

LMCA score versus cost

The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

0% 20% 40% 60% 80% $0.0001$0.001$0.01$0.1$1$10 Cost per response ($), log scale, lower is better LMCA GPT-3.5 Turbo: LMCA 9.65%, cost per response $0.0006 Claude 3 Opus: LMCA 17%, cost per response $0.05 Claude 3 Haiku: LMCA 8.84%, cost per response $0.0006 Gemma 2 27B: LMCA 7.1%, cost per response $0.0008 Llama 3.1-70B: LMCA 14.8%, cost per response $0.0006 Llama 3.1-8B: LMCA 5.37%, cost per response $8.6 c4ai-command-r-08-2024: LMCA 9.19%, cost per response $0.0003 Command R+: LMCA 5.01%, cost per response $0.003 Qwen2.5-72B: LMCA 13.4%, cost per response $0.004 Qwen2.5-7B: LMCA 6.42%, cost per response $0.0005 Llama 3.3 70B: LMCA 17.5%, cost per response $0.001 o1: LMCA 22.3%, cost per response $0.24 o3-mini: LMCA 19%, cost per response $0.03 Gemma 3 27B: LMCA 12.3%, cost per response $0.0003 Gemma 3 4B: LMCA 2.78%, cost per response $0.0001 Cohere Command A: LMCA 10.3%, cost per response $0.008 DeepSeek-V3: LMCA 15.5%, cost per response $0.0004 Llama 4 Maverick: LMCA 15.9%, cost per response $0.0006 Llama 4 Scout: LMCA 12%, cost per response $0.0005 GPT-4.1: LMCA 25.6%, cost per response $0.007 GPT-4.1 mini: LMCA 21.1%, cost per response $0.001 GPT-4.1 nano: LMCA 5.46%, cost per response $0.0003 o3: LMCA 39.7%, cost per response $0.02 o4-mini: LMCA 26.5%, cost per response $0.006 Qwen Plus: LMCA 24%, cost per response $0.002 Qwen3-14B: LMCA 18.2%, cost per response $0.004 Qwen3-235B-A22B: LMCA 25%, cost per response $0.01 Qwen3-30B-A3B: LMCA 15.8%, cost per response $0.003 Qwen3-32B: LMCA 17.3%, cost per response $0.0006 Qwen3-8B: LMCA 8.82%, cost per response $0.002 Claude Opus 4: LMCA 37.4%, cost per response $0.12 Claude Sonnet 4: LMCA 29%, cost per response $0.04 o3-pro: LMCA 38.5%, cost per response $0.15 Gemma 3n E4B: LMCA 3.16%, cost per response $0.0001 Qwen3-235B-A22B-Thinking: LMCA 29.3%, cost per response $0.006 Qwen3-30B-A3B-Instruct: LMCA 22.4%, cost per response $0.001 Qwen3-30B-A3B-Thinking: LMCA 19.5%, cost per response $0.004 Claude Opus 4.1: LMCA 37.1%, cost per response $0.13 gpt-oss-120b: LMCA 22.1%, cost per response $0.001 gpt-oss-20b: LMCA 14.5%, cost per response $0.0003 GPT-5: LMCA 40%, cost per response $0.03 GPT-5 mini: LMCA 34.2%, cost per response $0.005 GPT-5 nano: LMCA 7.93%, cost per response $0.002 Mistral Medium 3.1: LMCA 19.1%, cost per response $0.002 DeepSeek-V3.1: LMCA 24.3%, cost per response $0.004 Qwen3-Next-80B-A3B (thinking): LMCA 21.8%, cost per response $0.006 Qwen3-Next-80B-A3B: LMCA 21.4%, cost per response $0.001 DeepSeek-V3.1-Terminus: LMCA 28.6%, cost per response $0.004 Qwen3-Max (thinking): LMCA 33.6%, cost per response $0.04 Qwen3-Max: LMCA 28.3%, cost per response $0.007 Gemini 2.5 Flash-Lite: LMCA 18.1%, cost per response $0.002 Claude Sonnet 4.5: LMCA 38.8%, cost per response $0.07 DeepSeek-V3.2-Exp: LMCA 29.1%, cost per response $0.001 Claude Haiku 4.5: LMCA 30.9%, cost per response $0.02 GPT-5.1: LMCA 43.9%, cost per response $0.04 Claude Opus 4.5: LMCA 44.5%, cost per response $0.06 DeepSeek-V3.2: LMCA 28.8%, cost per response $0.001 Ministral 3 14B: LMCA 12.3%, cost per response $0.0004 Ministral 3 8B Instruct: LMCA 12.7%, cost per response $0.0003 Mistral Large 3: LMCA 16.7%, cost per response $0.001 GPT-5.2: LMCA 43.9%, cost per response $0.11 GPT-5.2 Instant: LMCA 40.1%, cost per response $0.005 Gemini 3 Flash: LMCA 43.1%, cost per response $0.008 Claude Opus 4.6: LMCA 55.8%, cost per response $0.09 Qwen3.5 397B-A17B: LMCA 37.9%, cost per response $0.02 Qwen 3.5 Plus (hosted 397B-A17B): LMCA 36.4%, cost per response $0.008 Claude Sonnet 4.6: LMCA 46.5%, cost per response $0.07 Grok 4.20: LMCA 38.7%, cost per response $0.006 Gemini 3.1 Pro (medium): LMCA 52.8%, cost per response $0.02 Gemini 3.1 Pro (high): LMCA 53.8%, cost per response $0.05 Gemini 3.1 Pro (low): LMCA 51.2%, cost per response $0.009 Qwen3.5-122B-A10B: LMCA 32.2%, cost per response $0.02 Qwen3.5-27B: LMCA 34%, cost per response $0.01 Qwen3.5-35B-A3B: LMCA 29.5%, cost per response $0.01 Qwen3.5-9B: LMCA 24.5%, cost per response $0.001 Qwen 3.5 Flash (hosted 35B-A3B): LMCA 29.1%, cost per response $0.002 Gemini 3.1 Flash-Lite: LMCA 35%, cost per response $0.003 GPT-5.3 Instant: LMCA 38.1%, cost per response $0.005 GPT-5.4: LMCA 52%, cost per response $0.09 Grok 4.20 Multi-agent: LMCA 41.4%, cost per response $0.31 Mistral Small 4: LMCA 20.6%, cost per response $0.001 GPT-5.4 Mini: LMCA 40.8%, cost per response $0.05 GPT-5.4 Nano: LMCA 36.9%, cost per response $0.01 Qwen 3.6 Plus: LMCA 33.1%, cost per response $0.01 Gemma 4 26B A4B: LMCA 29.7%, cost per response $0.002 Gemma 4 31B IT: LMCA 39.3%, cost per response $0.001 Qwen 3.6 35B-A3B: LMCA 29.7%, cost per response $0.007 Claude Opus 4.7: LMCA 52.2%, cost per response $0.13 Kimi K2.6: LMCA 37.3%, cost per response $0.02 Qwen 3.6 Max (Preview): LMCA 42.5%, cost per response $0.02 Qwen3.6 27B: LMCA 34.5%, cost per response $0.01 GPT-5.5: LMCA 54.3%, cost per response $0.07 GPT-5.5 Pro: LMCA 53.9%, cost per response $0.41 MiMo-V2.5-Pro: LMCA 29.5%, cost per response $0.002 DeepSeek-V4-Flash: LMCA 35.9%, cost per response $0.002 DeepSeek-V4-Pro: LMCA 41.2%, cost per response $0.006 Qwen 3.6 Flash: LMCA 31%, cost per response $0.005 Mistral Medium 3.5: LMCA 26.1%, cost per response $0.04 Grok 4.3: LMCA 38.3%, cost per response $0.004 Gemini 3.5 Flash: LMCA 47.1%, cost per response $0.03 Qwen3.7-Max: LMCA 44%, cost per response $0.02 Claude Opus 4.8: LMCA 57.5%, cost per response $0.3 MiniMax-M3: LMCA 33.7%, cost per response $0.003 Qwen3.7-Plus: LMCA 37.6%, cost per response $0.005 Nemotron 3 Ultra: LMCA 36.9%, cost per response $0.005 Claude Fable 5 (max): LMCA 60.3%, cost per response $0.39 Claude Fable 5 (xhigh): LMCA 61%, cost per response $0.18 Claude Fable 5 (high): LMCA 61.1%, cost per response $0.11 Claude Fable 5 (medium): LMCA 60.3%, cost per response $0.08 Claude Fable 5 (low): LMCA 59.6%, cost per response $0.04 GLM-5.2: LMCA 45.8%, cost per response $0.02 Claude Sonnet 5 (max): LMCA 49.3%, cost per response $0.38 Claude Sonnet 5 (xhigh): LMCA 48.8%, cost per response $0.07 Claude Sonnet 5 (high): LMCA 50%, cost per response $0.03 Claude Sonnet 5 (medium): LMCA 49.7%, cost per response $0.01 Tencent Hy3: LMCA 37.5%, cost per response $0.002 Grok 4.5: LMCA 45.2%, cost per response $0.01 GPT-5.6 Luna (max): LMCA 48.5%, cost per response $0.008 GPT-5.6 Sol (max): LMCA 58.4%, cost per response $0.08 GPT-5.6 Sol (xhigh): LMCA 58.5%, cost per response $0.03 GPT-5.6 Sol (high): LMCA 56.9%, cost per response $0.02 GPT-5.6 Sol (medium): LMCA 56.4%, cost per response $0.01 GPT-5.6 Sol (low): LMCA 55.9%, cost per response $0.01 GPT-5.6 Sol Pro: LMCA 59.2%, cost per response $0.37 GPT-5.6 Terra (max): LMCA 55%, cost per response $0.1 GPT-5.6 Terra (xhigh): LMCA 52.2%, cost per response $0.02 GPT-5.6 Terra (high): LMCA 51.4%, cost per response $0.009 GPT-5.6 Terra (medium): LMCA 49.6%, cost per response $0.006 GPT-5.6 Terra (low): LMCA 49.3%, cost per response $0.005 GPT-5.6 Terra (off): LMCA 46.5%, cost per response $0.002 Muse Spark 1.1: LMCA 49.9%, cost per response $0.01 Inkling: LMCA 37.6%, cost per response $0.02 Kimi K3: LMCA 52.7%, cost per response $0.06 Gemini 3.5 Flash-Lite: LMCA 37.6%, cost per response $0.005 Gemini 3.6 Flash: LMCA 44.9%, cost per response $0.02 Claude Opus 5 (xhigh): LMCA 64.5%, cost per response $0.08 Claude Opus 5 (high): LMCA 63.7%, cost per response $0.06 Claude Opus 5 (max): LMCA 63.3%, cost per response $0.1 Claude Opus 5 (medium): LMCA 62.9%, cost per response $0.04 Claude Opus 5 (low): LMCA 62.6%, cost per response $0.02 DeepSeek V4 Flash 0731: LMCA 41.7%, cost per response $0.006 Qwen 3.8 Max: LMCA 46.2%, cost per response $0.02 Muse Spark 1.2: LMCA 48.4%, cost per response $0.02 Muse Glimmer: LMCA 34.9%, cost per response $0.003 Nemotron 3.5 Lightning 30B-A3B: LMCA 20.9%, cost per response $0.001 Grok 4.6: LMCA 48.5%, cost per response $0.03 Qwen3.8-2.4T-A95B: LMCA 45.5%, cost per response $0.02 DeepSeek V4 Pro 0813: LMCA 45.5%, cost per response $0.03 Gemini 3.7 Flash (high): LMCA 50.4%, cost per response $0.01 Gemini 3.7 Flash (medium): LMCA 49.2%, cost per response $0.007 Gemini 3.7 Flash (low): LMCA 47.1%, cost per response $0.005 Qwen 3.8 27B: LMCA 41.4%, cost per response $0.01 Claude Fable 5.1 (medium): LMCA 65.3%, cost per response $0.09 Claude Fable 5.1 (max): LMCA 65.5%, cost per response $0.38 Claude Fable 5.1 (xhigh): LMCA 65.1%, cost per response $0.2 Claude Fable 5.1 (high): LMCA 65.1%, cost per response $0.11 Claude Fable 5.1 (low): LMCA 63.5%, cost per response $0.07 Gemini 3.8 Flash (medium): LMCA 52.9%, cost per response $0.01 Gemini 3.8 Flash (high): LMCA 51.5%, cost per response $0.05 Muse Spark 1.3: LMCA 53.9%, cost per response $0.03 GPT-6 Astra (xhigh): LMCA 64.4%, cost per response $0.06 GPT-6 Astra (max): LMCA 64.1%, cost per response $0.11 GPT-6 Astra (high): LMCA 63.8%, cost per response $0.04 GPT-6 Astra (medium): LMCA 62.9%, cost per response $0.02 GPT-6 Astra (low): LMCA 62.2%, cost per response $0.01 DeepSeek V4.1 Flash: LMCA 47%, cost per response $0.009 Grok 4.7: LMCA 49.4%, cost per response $0.08 Claude Opus 5.5 (max): LMCA 68.2%, cost per response $0.18 Claude Opus 5.5 (high): LMCA 65.9%, cost per response $0.03 Claude Opus 5.5 (xhigh): LMCA 66.2%, cost per response $0.04 Claude Opus 5.5 (medium): LMCA 65.7%, cost per response $0.03 Claude Opus 5.5 (low): LMCA 64.9%, cost per response $0.02 GPT-6 Luna: LMCA 44.5%, cost per response $0.002 GPT-6 Sol: LMCA 59.1%, cost per response $0.02 Claude Sonnet 5.5 (max): LMCA 68.3%, cost per response $0.1 GPT-6.1 Sol: LMCA 62.8%, cost per response $0.02 Gemini 4 Argon (low): LMCA 62.9%, cost per response $0.02 Gemini 4 Argon (high): LMCA 63.7%, cost per response $0.06 Gemini 4 Argon (medium): LMCA 62.9%, cost per response $0.02 Mistral Large 4: LMCA 35.8%, cost per response $0.02 Claude Haiku 5.5 (max): LMCA 51.1%, cost per response $0.02 Claude Haiku 5.5 (high): LMCA 45.2%, cost per response $0.001 Claude Haiku 5.5 (off): LMCA 39.3%, cost per response $0.0005 Muse Wonder: LMCA 56.8%, cost per response $0.1 Kimi K3.1 Pro: LMCA 57.6%, cost per response $0.07 Claude Fable 5.2 (max): LMCA 70.1%, cost per response $0.16 Claude Fable 5.2 (high): LMCA 69.9%, cost per response $0.06 Gemini 2.5 Pro (snapshot unspecified): LMCA 34.8%, cost per response $0.04. Not in our model list Gemini 2.5 Flash (snapshot unspecified): LMCA 27.5%, cost per response $0.008. Not in our model list Qwen3 235B-A22B (2507): LMCA 23.5%, cost per response $0.0008. Not in our model list GPT-4 (snapshot unspecified): LMCA 17.1%, cost per response $0.06. Not in our model list GPT-4o (snapshot unspecified): LMCA 16.6%, cost per response $0.007. Not in our model list DeepSeek Chat: LMCA 15.2%, cost per response $0.0005. Not in our model list GPT-4o mini (snapshot unspecified): LMCA 10.4%, cost per response $0.0004. Not in our model list GPT-4 Turbo (snapshot unspecified): LMCA 9.78%, cost per response $0.02. Not in our model list Ministral 3B: LMCA 5.48%, cost per response $0.0002 Command R7B: LMCA 4.45%, cost per response $0.0001 Gemma 3 12B: LMCA 4.46%, cost per response $0.0001 Claude Sonnet 5 (low): LMCA 49.6%, cost per response $0.005 GPT-5.6 Luna (xhigh): LMCA 48.5%, cost per response $0.004 GPT-5.6 Luna (high): LMCA 47.4%, cost per response $0.002 GPT-5.6 Luna (medium): LMCA 44.3%, cost per response $0.0007 GPT-5.6 Luna (low): LMCA 43.8%, cost per response $0.0005 GPT-5.6 Luna (off): LMCA 41.4%, cost per response $0.0002 GPT-5.6 Sol (off): LMCA 52.8%, cost per response $0.006 GLM-5.3: LMCA 55.5%, cost per response $0.009 Gemini 3.8 Flash (low): LMCA 51.2%, cost per response $0.006 Claude Sonnet 5.5 (xhigh): LMCA 66.3%, cost per response $0.02 Claude Sonnet 5.5 (medium): LMCA 65.5%, cost per response $0.009 Claude Sonnet 5.5 (high): LMCA 65.9%, cost per response $0.01 Claude Sonnet 5.5 (low): LMCA 64.5%, cost per response $0.009 Claude Haiku 5.5 (xhigh): LMCA 48%, cost per response $0.002 Claude Haiku 5.5 (medium): LMCA 45.6%, cost per response $0.0009 Claude Haiku 5.5 (low): LMCA 44.8%, cost per response $0.0008 Claude Fable 5.2 (xhigh): LMCA 70.6%, cost per response $0.07 Claude Fable 5.2 (medium): LMCA 70%, cost per response $0.05 Claude Fable 5.2 (low): LMCA 69.6%, cost per response $0.04 Claude Fable 5.2 (xhigh)Claude Fable 5.2 (medium)Claude Fable 5.2 (low)Claude Sonnet 5.5 (xhigh)Claude Sonnet 5.5 (high)Claude Sonnet 5.5 (medium)Claude Sonnet 5.5 (low)GLM-5.3GPT-5.6 Sol (off)Gemini 3.8 Flash (low)Claude Sonnet 5 (low)GPT-5.6 Luna (xhigh)Claude Haiku 5.5 (xhigh)GPT-5.6 Luna (high)Claude Haiku 5.5 (medium)Claude Haiku 5.5 (low)GPT-5.6 Luna (medium)GPT-5.6 Luna (low)GPT-5.6 Luna (off)Ministral 3BGemma 3 12BCommand R7B 0% 20% 40% 60% 80% $0.0001$0.001$0.01$0.1$1$10 Cost per response ($), log scale, lower is better LMCA GPT-3.5 Turbo: LMCA 9.65%, cost per response $0.0006 Claude 3 Opus: LMCA 17%, cost per response $0.05 Claude 3 Haiku: LMCA 8.84%, cost per response $0.0006 Gemma 2 27B: LMCA 7.1%, cost per response $0.0008 Llama 3.1-70B: LMCA 14.8%, cost per response $0.0006 Llama 3.1-8B: LMCA 5.37%, cost per response $8.6 c4ai-command-r-08-2024: LMCA 9.19%, cost per response $0.0003 Command R+: LMCA 5.01%, cost per response $0.003 Qwen2.5-72B: LMCA 13.4%, cost per response $0.004 Qwen2.5-7B: LMCA 6.42%, cost per response $0.0005 Llama 3.3 70B: LMCA 17.5%, cost per response $0.001 o1: LMCA 22.3%, cost per response $0.24 o3-mini: LMCA 19%, cost per response $0.03 Gemma 3 27B: LMCA 12.3%, cost per response $0.0003 Gemma 3 4B: LMCA 2.78%, cost per response $0.0001 Cohere Command A: LMCA 10.3%, cost per response $0.008 DeepSeek-V3: LMCA 15.5%, cost per response $0.0004 Llama 4 Maverick: LMCA 15.9%, cost per response $0.0006 Llama 4 Scout: LMCA 12%, cost per response $0.0005 GPT-4.1: LMCA 25.6%, cost per response $0.007 GPT-4.1 mini: LMCA 21.1%, cost per response $0.001 GPT-4.1 nano: LMCA 5.46%, cost per response $0.0003 o3: LMCA 39.7%, cost per response $0.02 o4-mini: LMCA 26.5%, cost per response $0.006 Qwen Plus: LMCA 24%, cost per response $0.002 Qwen3-14B: LMCA 18.2%, cost per response $0.004 Qwen3-235B-A22B: LMCA 25%, cost per response $0.01 Qwen3-30B-A3B: LMCA 15.8%, cost per response $0.003 Qwen3-32B: LMCA 17.3%, cost per response $0.0006 Qwen3-8B: LMCA 8.82%, cost per response $0.002 Claude Opus 4: LMCA 37.4%, cost per response $0.12 Claude Sonnet 4: LMCA 29%, cost per response $0.04 o3-pro: LMCA 38.5%, cost per response $0.15 Gemma 3n E4B: LMCA 3.16%, cost per response $0.0001 Qwen3-235B-A22B-Thinking: LMCA 29.3%, cost per response $0.006 Qwen3-30B-A3B-Instruct: LMCA 22.4%, cost per response $0.001 Qwen3-30B-A3B-Thinking: LMCA 19.5%, cost per response $0.004 Claude Opus 4.1: LMCA 37.1%, cost per response $0.13 gpt-oss-120b: LMCA 22.1%, cost per response $0.001 gpt-oss-20b: LMCA 14.5%, cost per response $0.0003 GPT-5: LMCA 40%, cost per response $0.03 GPT-5 mini: LMCA 34.2%, cost per response $0.005 GPT-5 nano: LMCA 7.93%, cost per response $0.002 Mistral Medium 3.1: LMCA 19.1%, cost per response $0.002 DeepSeek-V3.1: LMCA 24.3%, cost per response $0.004 Qwen3-Next-80B-A3B (thinking): LMCA 21.8%, cost per response $0.006 Qwen3-Next-80B-A3B: LMCA 21.4%, cost per response $0.001 DeepSeek-V3.1-Terminus: LMCA 28.6%, cost per response $0.004 Qwen3-Max (thinking): LMCA 33.6%, cost per response $0.04 Qwen3-Max: LMCA 28.3%, cost per response $0.007 Gemini 2.5 Flash-Lite: LMCA 18.1%, cost per response $0.002 Claude Sonnet 4.5: LMCA 38.8%, cost per response $0.07 DeepSeek-V3.2-Exp: LMCA 29.1%, cost per response $0.001 Claude Haiku 4.5: LMCA 30.9%, cost per response $0.02 GPT-5.1: LMCA 43.9%, cost per response $0.04 Claude Opus 4.5: LMCA 44.5%, cost per response $0.06 DeepSeek-V3.2: LMCA 28.8%, cost per response $0.001 Ministral 3 14B: LMCA 12.3%, cost per response $0.0004 Ministral 3 8B Instruct: LMCA 12.7%, cost per response $0.0003 Mistral Large 3: LMCA 16.7%, cost per response $0.001 GPT-5.2: LMCA 43.9%, cost per response $0.11 GPT-5.2 Instant: LMCA 40.1%, cost per response $0.005 Gemini 3 Flash: LMCA 43.1%, cost per response $0.008 Claude Opus 4.6: LMCA 55.8%, cost per response $0.09 Qwen3.5 397B-A17B: LMCA 37.9%, cost per response $0.02 Qwen 3.5 Plus (hosted 397B-A17B): LMCA 36.4%, cost per response $0.008 Claude Sonnet 4.6: LMCA 46.5%, cost per response $0.07 Grok 4.20: LMCA 38.7%, cost per response $0.006 Gemini 3.1 Pro (medium): LMCA 52.8%, cost per response $0.02 Gemini 3.1 Pro (high): LMCA 53.8%, cost per response $0.05 Gemini 3.1 Pro (low): LMCA 51.2%, cost per response $0.009 Qwen3.5-122B-A10B: LMCA 32.2%, cost per response $0.02 Qwen3.5-27B: LMCA 34%, cost per response $0.01 Qwen3.5-35B-A3B: LMCA 29.5%, cost per response $0.01 Qwen3.5-9B: LMCA 24.5%, cost per response $0.001 Qwen 3.5 Flash (hosted 35B-A3B): LMCA 29.1%, cost per response $0.002 Gemini 3.1 Flash-Lite: LMCA 35%, cost per response $0.003 GPT-5.3 Instant: LMCA 38.1%, cost per response $0.005 GPT-5.4: LMCA 52%, cost per response $0.09 Grok 4.20 Multi-agent: LMCA 41.4%, cost per response $0.31 Mistral Small 4: LMCA 20.6%, cost per response $0.001 GPT-5.4 Mini: LMCA 40.8%, cost per response $0.05 GPT-5.4 Nano: LMCA 36.9%, cost per response $0.01 Qwen 3.6 Plus: LMCA 33.1%, cost per response $0.01 Gemma 4 26B A4B: LMCA 29.7%, cost per response $0.002 Gemma 4 31B IT: LMCA 39.3%, cost per response $0.001 Qwen 3.6 35B-A3B: LMCA 29.7%, cost per response $0.007 Claude Opus 4.7: LMCA 52.2%, cost per response $0.13 Kimi K2.6: LMCA 37.3%, cost per response $0.02 Qwen 3.6 Max (Preview): LMCA 42.5%, cost per response $0.02 Qwen3.6 27B: LMCA 34.5%, cost per response $0.01 GPT-5.5: LMCA 54.3%, cost per response $0.07 GPT-5.5 Pro: LMCA 53.9%, cost per response $0.41 MiMo-V2.5-Pro: LMCA 29.5%, cost per response $0.002 DeepSeek-V4-Flash: LMCA 35.9%, cost per response $0.002 DeepSeek-V4-Pro: LMCA 41.2%, cost per response $0.006 Qwen 3.6 Flash: LMCA 31%, cost per response $0.005 Mistral Medium 3.5: LMCA 26.1%, cost per response $0.04 Grok 4.3: LMCA 38.3%, cost per response $0.004 Gemini 3.5 Flash: LMCA 47.1%, cost per response $0.03 Qwen3.7-Max: LMCA 44%, cost per response $0.02 Claude Opus 4.8: LMCA 57.5%, cost per response $0.3 MiniMax-M3: LMCA 33.7%, cost per response $0.003 Qwen3.7-Plus: LMCA 37.6%, cost per response $0.005 Nemotron 3 Ultra: LMCA 36.9%, cost per response $0.005 Claude Fable 5 (max): LMCA 60.3%, cost per response $0.39 Claude Fable 5 (xhigh): LMCA 61%, cost per response $0.18 Claude Fable 5 (high): LMCA 61.1%, cost per response $0.11 Claude Fable 5 (medium): LMCA 60.3%, cost per response $0.08 Claude Fable 5 (low): LMCA 59.6%, cost per response $0.04 GLM-5.2: LMCA 45.8%, cost per response $0.02 Claude Sonnet 5 (max): LMCA 49.3%, cost per response $0.38 Claude Sonnet 5 (xhigh): LMCA 48.8%, cost per response $0.07 Claude Sonnet 5 (high): LMCA 50%, cost per response $0.03 Claude Sonnet 5 (medium): LMCA 49.7%, cost per response $0.01 Tencent Hy3: LMCA 37.5%, cost per response $0.002 Grok 4.5: LMCA 45.2%, cost per response $0.01 GPT-5.6 Luna (max): LMCA 48.5%, cost per response $0.008 GPT-5.6 Sol (max): LMCA 58.4%, cost per response $0.08 GPT-5.6 Sol (xhigh): LMCA 58.5%, cost per response $0.03 GPT-5.6 Sol (high): LMCA 56.9%, cost per response $0.02 GPT-5.6 Sol (medium): LMCA 56.4%, cost per response $0.01 GPT-5.6 Sol (low): LMCA 55.9%, cost per response $0.01 GPT-5.6 Sol Pro: LMCA 59.2%, cost per response $0.37 GPT-5.6 Terra (max): LMCA 55%, cost per response $0.1 GPT-5.6 Terra (xhigh): LMCA 52.2%, cost per response $0.02 GPT-5.6 Terra (high): LMCA 51.4%, cost per response $0.009 GPT-5.6 Terra (medium): LMCA 49.6%, cost per response $0.006 GPT-5.6 Terra (low): LMCA 49.3%, cost per response $0.005 GPT-5.6 Terra (off): LMCA 46.5%, cost per response $0.002 Muse Spark 1.1: LMCA 49.9%, cost per response $0.01 Inkling: LMCA 37.6%, cost per response $0.02 Kimi K3: LMCA 52.7%, cost per response $0.06 Gemini 3.5 Flash-Lite: LMCA 37.6%, cost per response $0.005 Gemini 3.6 Flash: LMCA 44.9%, cost per response $0.02 Claude Opus 5 (xhigh): LMCA 64.5%, cost per response $0.08 Claude Opus 5 (high): LMCA 63.7%, cost per response $0.06 Claude Opus 5 (max): LMCA 63.3%, cost per response $0.1 Claude Opus 5 (medium): LMCA 62.9%, cost per response $0.04 Claude Opus 5 (low): LMCA 62.6%, cost per response $0.02 DeepSeek V4 Flash 0731: LMCA 41.7%, cost per response $0.006 Qwen 3.8 Max: LMCA 46.2%, cost per response $0.02 Muse Spark 1.2: LMCA 48.4%, cost per response $0.02 Muse Glimmer: LMCA 34.9%, cost per response $0.003 Nemotron 3.5 Lightning 30B-A3B: LMCA 20.9%, cost per response $0.001 Grok 4.6: LMCA 48.5%, cost per response $0.03 Qwen3.8-2.4T-A95B: LMCA 45.5%, cost per response $0.02 DeepSeek V4 Pro 0813: LMCA 45.5%, cost per response $0.03 Gemini 3.7 Flash (high): LMCA 50.4%, cost per response $0.01 Gemini 3.7 Flash (medium): LMCA 49.2%, cost per response $0.007 Gemini 3.7 Flash (low): LMCA 47.1%, cost per response $0.005 Qwen 3.8 27B: LMCA 41.4%, cost per response $0.01 Claude Fable 5.1 (medium): LMCA 65.3%, cost per response $0.09 Claude Fable 5.1 (max): LMCA 65.5%, cost per response $0.38 Claude Fable 5.1 (xhigh): LMCA 65.1%, cost per response $0.2 Claude Fable 5.1 (high): LMCA 65.1%, cost per response $0.11 Claude Fable 5.1 (low): LMCA 63.5%, cost per response $0.07 Gemini 3.8 Flash (medium): LMCA 52.9%, cost per response $0.01 Gemini 3.8 Flash (high): LMCA 51.5%, cost per response $0.05 Muse Spark 1.3: LMCA 53.9%, cost per response $0.03 GPT-6 Astra (xhigh): LMCA 64.4%, cost per response $0.06 GPT-6 Astra (max): LMCA 64.1%, cost per response $0.11 GPT-6 Astra (high): LMCA 63.8%, cost per response $0.04 GPT-6 Astra (medium): LMCA 62.9%, cost per response $0.02 GPT-6 Astra (low): LMCA 62.2%, cost per response $0.01 DeepSeek V4.1 Flash: LMCA 47%, cost per response $0.009 Grok 4.7: LMCA 49.4%, cost per response $0.08 Claude Opus 5.5 (max): LMCA 68.2%, cost per response $0.18 Claude Opus 5.5 (high): LMCA 65.9%, cost per response $0.03 Claude Opus 5.5 (xhigh): LMCA 66.2%, cost per response $0.04 Claude Opus 5.5 (medium): LMCA 65.7%, cost per response $0.03 Claude Opus 5.5 (low): LMCA 64.9%, cost per response $0.02 GPT-6 Luna: LMCA 44.5%, cost per response $0.002 GPT-6 Sol: LMCA 59.1%, cost per response $0.02 Claude Sonnet 5.5 (max): LMCA 68.3%, cost per response $0.1 GPT-6.1 Sol: LMCA 62.8%, cost per response $0.02 Gemini 4 Argon (low): LMCA 62.9%, cost per response $0.02 Gemini 4 Argon (high): LMCA 63.7%, cost per response $0.06 Gemini 4 Argon (medium): LMCA 62.9%, cost per response $0.02 Mistral Large 4: LMCA 35.8%, cost per response $0.02 Claude Haiku 5.5 (max): LMCA 51.1%, cost per response $0.02 Claude Haiku 5.5 (high): LMCA 45.2%, cost per response $0.001 Claude Haiku 5.5 (off): LMCA 39.3%, cost per response $0.0005 Muse Wonder: LMCA 56.8%, cost per response $0.1 Kimi K3.1 Pro: LMCA 57.6%, cost per response $0.07 Claude Fable 5.2 (max): LMCA 70.1%, cost per response $0.16 Claude Fable 5.2 (high): LMCA 69.9%, cost per response $0.06 Gemini 2.5 Pro (snapshot unspecified): LMCA 34.8%, cost per response $0.04. Not in our model list Gemini 2.5 Flash (snapshot unspecified): LMCA 27.5%, cost per response $0.008. Not in our model list Qwen3 235B-A22B (2507): LMCA 23.5%, cost per response $0.0008. Not in our model list GPT-4 (snapshot unspecified): LMCA 17.1%, cost per response $0.06. Not in our model list GPT-4o (snapshot unspecified): LMCA 16.6%, cost per response $0.007. Not in our model list DeepSeek Chat: LMCA 15.2%, cost per response $0.0005. Not in our model list GPT-4o mini (snapshot unspecified): LMCA 10.4%, cost per response $0.0004. Not in our model list GPT-4 Turbo (snapshot unspecified): LMCA 9.78%, cost per response $0.02. Not in our model list Ministral 3B: LMCA 5.48%, cost per response $0.0002 Command R7B: LMCA 4.45%, cost per response $0.0001 Gemma 3 12B: LMCA 4.46%, cost per response $0.0001 Claude Sonnet 5 (low): LMCA 49.6%, cost per response $0.005 GPT-5.6 Luna (xhigh): LMCA 48.5%, cost per response $0.004 GPT-5.6 Luna (high): LMCA 47.4%, cost per response $0.002 GPT-5.6 Luna (medium): LMCA 44.3%, cost per response $0.0007 GPT-5.6 Luna (low): LMCA 43.8%, cost per response $0.0005 GPT-5.6 Luna (off): LMCA 41.4%, cost per response $0.0002 GPT-5.6 Sol (off): LMCA 52.8%, cost per response $0.006 GLM-5.3: LMCA 55.5%, cost per response $0.009 Gemini 3.8 Flash (low): LMCA 51.2%, cost per response $0.006 Claude Sonnet 5.5 (xhigh): LMCA 66.3%, cost per response $0.02 Claude Sonnet 5.5 (medium): LMCA 65.5%, cost per response $0.009 Claude Sonnet 5.5 (high): LMCA 65.9%, cost per response $0.01 Claude Sonnet 5.5 (low): LMCA 64.5%, cost per response $0.009 Claude Haiku 5.5 (xhigh): LMCA 48%, cost per response $0.002 Claude Haiku 5.5 (medium): LMCA 45.6%, cost per response $0.0009 Claude Haiku 5.5 (low): LMCA 44.8%, cost per response $0.0008 Claude Fable 5.2 (xhigh): LMCA 70.6%, cost per response $0.07 Claude Fable 5.2 (medium): LMCA 70%, cost per response $0.05 Claude Fable 5.2 (low): LMCA 69.6%, cost per response $0.04 Claude Fable 5.2 (xhigh)Claude Fable 5.2 (medium)Claude Fable 5.2 (low)Claude Sonnet 5.5 (xhigh)Claude Sonnet 5.5 (high)Claude Sonnet 5.5 (medium)Claude Sonnet 5.5 (low)GLM-5.3GPT-5.6 Sol (off)Gemini 3.8 Flash (low)Claude Sonnet 5 (low)GPT-5.6 Luna (xhigh)Claude Haiku 5.5 (xhigh)GPT-5.6 Luna (high)Claude Haiku 5.5 (medium)Claude Haiku 5.5 (low)GPT-5.6 Luna (medium)GPT-5.6 Luna (low)GPT-5.6 Luna (off)Ministral 3BGemma 3 12BCommand R7B
ModelSettingLMCACost per response
Claude Fable 5.2 xhigh 70.6% $0.07
Claude Fable 5.2 max 70.1% $0.16
Claude Fable 5.2 medium 70% $0.05
Claude Fable 5.2 high 69.9% $0.06
Claude Fable 5.2 low 69.6% $0.04
Claude Sonnet 5.5 max 68.3% $0.1
Claude Opus 5.5 max 68.2% $0.18
Claude Sonnet 5.5 xhigh 66.3% $0.02
Claude Opus 5.5 xhigh 66.2% $0.04
Claude Sonnet 5.5 high 65.9% $0.01
Claude Opus 5.5 high 65.9% $0.03
Claude Opus 5.5 medium 65.7% $0.03
Claude Fable 5.1 max 65.5% $0.38
Claude Sonnet 5.5 medium 65.5% $0.01
Claude Fable 5.1 medium 65.3% $0.09
Claude Fable 5.1 high 65.1% $0.11
Claude Fable 5.1 xhigh 65.1% $0.2
Claude Opus 5.5 low 64.9% $0.02
Claude Sonnet 5.5 low 64.5% $0.01
Claude Opus 5 xhigh 64.5% $0.08
GPT-6 Astra xhigh 64.4% $0.06
GPT-6 Astra max 64.1% $0.11
GPT-6 Astra high 63.8% $0.04
Gemini 4 Argon high 63.7% $0.06
Claude Opus 5 high 63.7% $0.06
Claude Fable 5.1 low 63.5% $0.07
Claude Opus 5 max 63.3% $0.1
GPT-6 Astra medium 62.9% $0.02
Claude Opus 5 medium 62.9% $0.04
Gemini 4 Argon low 62.9% $0.02
Gemini 4 Argon medium 62.9% $0.02
GPT-6.1 Sol 62.8% $0.02
Claude Opus 5 low 62.6% $0.02
GPT-6 Astra low 62.2% $0.01
Claude Fable 5 high 61.2% $0.11
Claude Fable 5 xhigh 61% $0.18
Claude Fable 5 max 60.3% $0.39
Claude Fable 5 medium 60.3% $0.08
Claude Fable 5 low 59.6% $0.04
GPT-5.6 Sol Pro 59.2% $0.37
GPT-6 Sol 59.1% $0.02
GPT-5.6 Sol xhigh 58.5% $0.03
GPT-5.6 Sol max 58.4% $0.08
Kimi K3.1 Pro 57.6% $0.07
Claude Opus 4.8 57.5% $0.31
GPT-5.6 Sol high 56.9% $0.02
Muse Wonder 56.8% $0.1
GPT-5.6 Sol medium 56.4% $0.01
GPT-5.6 Sol low 55.9% $0.01
Claude Opus 4.6 55.8% $0.09
GLM-5.3 55.5% $0.01
GPT-5.6 Terra max 55% $0.1
GPT-5.5 54.3% $0.07
GPT-5.5 Pro 53.9% $0.41
Muse Spark 1.3 53.9% $0.03
Gemini 3.1 Pro high 53.8% $0.05
Gemini 3.8 Flash medium 52.9% $0.01
Gemini 3.1 Pro medium 52.8% $0.02
GPT-5.6 Sol off 52.8% $0.01
Kimi K3 52.7% $0.06
GPT-5.6 Terra xhigh 52.2% $0.02
Claude Opus 4.7 52.2% $0.13
GPT-5.4 52% $0.09
Gemini 3.8 Flash high 51.6% $0.05
GPT-5.6 Terra high 51.4% $0.01
Gemini 3.1 Pro low 51.2% $0.01
Gemini 3.8 Flash low 51.2% $0.01
Claude Haiku 5.5 max 51.1% $0.02
Gemini 3.7 Flash high 50.4% $0.01
Claude Sonnet 5 high 50% $0.03
Muse Spark 1.1 49.9% $0.01
Claude Sonnet 5 medium 49.7% $0.01
Claude Sonnet 5 low 49.7% $0
GPT-5.6 Terra medium 49.6% $0.01
Grok 4.7 49.4% $0.08
GPT-5.6 Terra low 49.3% $0.01
Claude Sonnet 5 max 49.3% $0.38
Gemini 3.7 Flash medium 49.2% $0.01
Claude Sonnet 5 xhigh 48.8% $0.07
GPT-5.6 Luna xhigh 48.5% $0
GPT-5.6 Luna max 48.5% $0.01
Grok 4.6 48.5% $0.03
Muse Spark 1.2 48.4% $0.02
Claude Haiku 5.5 xhigh 48% $0
GPT-5.6 Luna high 47.4% $0
Gemini 3.5 Flash 47.2% $0.03
Gemini 3.7 Flash low 47.1% $0
DeepSeek V4.1 Flash 47% $0.01
GPT-5.6 Terra off 46.5% $0
Claude Sonnet 4.6 46.5% $0.07
Qwen 3.8 Max 46.2% $0.02
GLM-5.2 45.8% $0.02
Claude Haiku 5.5 medium 45.6% $0
Qwen3.8-2.4T-A95B 45.5% $0.02
DeepSeek V4 Pro 0813 45.5% $0.03
Claude Haiku 5.5 high 45.2% $0
Grok 4.5 45.2% $0.01
Gemini 3.6 Flash 44.9% $0.02
Claude Haiku 5.5 low 44.8% $0
GPT-6 Luna 44.5% $0
Claude Opus 4.5 44.5% $0.06
GPT-5.6 Luna medium 44.3% $0
Qwen3.7-Max 44% $0.02
GPT-5.2 43.9% $0.11
GPT-5.1 43.9% $0.04
GPT-5.6 Luna low 43.8% $0
Gemini 3 Flash 43.1% $0.01
Qwen 3.6 Max (Preview) 42.5% $0.02
DeepSeek V4 Flash 0731 41.7% $0.01
GPT-5.6 Luna off 41.4% $0
Grok 4.20 Multi-agent 41.4% $0.31
Qwen 3.8 27B 41.4% $0.02
DeepSeek-V4-Pro 41.2% $0.01
GPT-5.4 Mini 40.8% $0.05
GPT-5.2 Instant 40.1% $0.01
GPT-5 40% $0.03
o3 39.7% $0.02
Claude Haiku 5.5 off 39.3% $0
Gemma 4 31B IT 39.3% $0
Claude Sonnet 4.5 38.8% $0.07
Grok 4.20 38.7% $0.01
o3-pro 38.5% $0.15
Grok 4.3 38.3% $0
GPT-5.3 Instant 38.1% $0
Qwen3.5 397B-A17B 37.9% $0.02
Gemini 3.5 Flash-Lite 37.6% $0.01
Qwen3.7-Plus 37.6% $0
Inkling 37.6% $0.02
Tencent Hy3 37.5% $0
Claude Opus 4 37.4% $0.12
Kimi K2.6 37.3% $0.02
Claude Opus 4.1 37.1% $0.13
Nemotron 3 Ultra 36.9% $0.01
GPT-5.4 Nano 36.9% $0.01
Qwen 3.5 Plus (hosted 397B-A17B) 36.4% $0.01
DeepSeek-V4-Flash 35.9% $0
Mistral Large 4 35.8% $0.02
Gemini 3.1 Flash-Lite 35% $0
Muse Glimmer 34.9% $0
Gemini 2.5 Pro (snapshot unspecified) 34.8% $0.04
Qwen3.6 27B 34.5% $0.01
GPT-5 mini 34.2% $0.01
Qwen3.5-27B 34% $0.01
MiniMax-M3 33.7% $0
Qwen3-Max thinking 33.6% $0.04
Qwen 3.6 Plus 33.1% $0.01
Qwen3.5-122B-A10B 32.2% $0.02
Qwen 3.6 Flash 31% $0
Claude Haiku 4.5 30.9% $0.02
Qwen 3.6 35B-A3B 29.7% $0.01
Gemma 4 26B A4B 29.7% $0
MiMo-V2.5-Pro 29.5% $0
Qwen3.5-35B-A3B 29.5% $0.01
Qwen3-235B-A22B-Thinking Jul 2025 29.3% $0.01
Qwen 3.5 Flash (hosted 35B-A3B) 29.1% $0
DeepSeek-V3.2-Exp 29.1% $0
Claude Sonnet 4 29% $0.04
DeepSeek-V3.2 28.8% $0
DeepSeek-V3.1-Terminus 28.6% $0
Qwen3-Max 28.3% $0.01
Gemini 2.5 Flash (snapshot unspecified) 27.5% $0.01
o4-mini 26.5% $0.01
Mistral Medium 3.5 26.1% $0.04
GPT-4.1 25.6% $0.01
Qwen3-235B-A22B 25% $0.01
Qwen3.5-9B 24.5% $0
DeepSeek-V3.1 24.3% $0
Qwen Plus Apr 2025 24% $0
Qwen3 235B-A22B (2507) 23.5% $0
Qwen3-30B-A3B-Instruct Jul 2025 22.4% $0
o1 22.3% $0.24
gpt-oss-120b 22.1% $0
Qwen3-Next-80B-A3B thinking 21.8% $0.01
Qwen3-Next-80B-A3B 21.4% $0
GPT-4.1 mini 21.1% $0
Nemotron 3.5 Lightning 30B-A3B 20.9% $0
Mistral Small 4 20.6% $0
Qwen3-30B-A3B-Thinking Jul 2025 19.5% $0
Mistral Medium 3.1 19.1% $0
o3-mini 19% $0.03
Qwen3-14B 18.2% $0
Gemini 2.5 Flash-Lite Sep 2025 18.1% $0
Llama 3.3 70B 17.5% $0
Qwen3-32B 17.3% $0
GPT-4 (snapshot unspecified) 17.1% $0.06
Claude 3 Opus 17% $0.05
Mistral Large 3 16.7% $0
GPT-4o (snapshot unspecified) 16.6% $0.01
Llama 4 Maverick 15.9% $0
Qwen3-30B-A3B 15.8% $0
DeepSeek-V3 Mar 2025 15.5% $0
DeepSeek Chat 15.2% $0
Llama 3.1-70B 14.9% $0
gpt-oss-20b 14.5% $0
Qwen2.5-72B 13.4% $0
Ministral 3 8B Instruct 12.7% $0
Ministral 3 14B 12.3% $0
Gemma 3 27B 12.3% $0
Llama 4 Scout 12% $0
GPT-4o mini (snapshot unspecified) 10.4% $0
Cohere Command A 10.3% $0.01
GPT-4 Turbo (snapshot unspecified) 9.8% $0.02
GPT-3.5 Turbo Jan 2024 9.7% $0
c4ai-command-r-08-2024 9.2% $0
Claude 3 Haiku 8.8% $0
Qwen3-8B 8.8% $0
GPT-5 nano 7.9% $0
Gemma 2 27B 7.1% $0
Qwen2.5-7B 6.4% $0
Ministral 3B 5.5% $0
GPT-4.1 nano 5.5% $0
Llama 3.1-8B 5.4% $8.6
Command R+ 5% $0
Gemma 3 12B 4.5% $0
Command R7B 4.5% $0
Gemma 3n E4B 3.2% $0
Gemma 3 4B 2.8% $0

The best score, 70.6%, costs $0.07 a task

Runs on the cost frontier score higher than any run that's cheaper. Claude Sonnet 5.5 at high comes within 4.7 points of the best for $0.01 a task, 7 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Mistral AI Cohere Google DeepMind Anthropic OpenAI Z.ai Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.0001 $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Fable 5.2, xhigh Claude Sonnet 5.5, xhigh GLM-5.3 Gemini 3.8 Flash, low Claude Haiku 5.5, medium GPT-5.6 Luna, off Ministral 3B 0% 20% 40% 60% 80% 100% $0.0001 $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Fable 5.2, xhigh Claude Sonnet 5.5, low

Source: Official LMCA leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5.2 xhigh 70.6% $0.07
Claude Fable 5.2 max 70.1% $0.16
Claude Fable 5.2 medium 70% $0.05
Claude Fable 5.2 high 69.9% $0.06
Claude Fable 5.2 low 69.6% $0.04
Claude Sonnet 5.5 max 68.3% $0.10
Claude Opus 5.5 max 68.2% $0.18
Claude Sonnet 5.5 xhigh 66.3% $0.02
Claude Opus 5.5 xhigh 66.2% $0.04
Claude Sonnet 5.5 high 65.9% $0.01
Claude Opus 5.5 high 65.9% $0.03
Claude Opus 5.5 medium 65.7% $0.03
Claude Fable 5.1 max 65.5% $0.38
Claude Sonnet 5.5 medium 65.5% $0.009
Claude Fable 5.1 medium 65.3% $0.09
Claude Fable 5.1 high 65.1% $0.11
Claude Fable 5.1 xhigh 65.1% $0.20
Claude Opus 5.5 low 64.9% $0.02
Claude Sonnet 5.5 low 64.5% $0.009
Claude Opus 5 xhigh 64.5% $0.08
GPT-6 Astra xhigh 64.4% $0.06
GPT-6 Astra max 64.1% $0.11
GPT-6 Astra high 63.8% $0.04
Gemini 4 Argon high 63.7% $0.06
Claude Opus 5 high 63.7% $0.06
Claude Fable 5.1 low 63.5% $0.07
Claude Opus 5 max 63.3% $0.10
Claude Opus 5 medium 62.9% $0.04
GPT-6 Astra medium 62.9% $0.02
Gemini 4 Argon low 62.9% $0.02
Gemini 4 Argon medium 62.9% $0.02
GPT-6.1 Sol 62.8% $0.02
Claude Opus 5 low 62.6% $0.02
GPT-6 Astra low 62.2% $0.01
Claude Fable 5 high 61.1% $0.11
Claude Fable 5 xhigh 61% $0.18
Claude Fable 5 max 60.3% $0.39
Claude Fable 5 medium 60.3% $0.08
Claude Fable 5 low 59.6% $0.04
GPT-5.6 Sol Pro 59.2% $0.37
GPT-6 Sol 59.1% $0.02
GPT-5.6 Sol xhigh 58.5% $0.03
GPT-5.6 Sol max 58.4% $0.08
Kimi K3.1 Pro 57.6% $0.07
Claude Opus 4.8 57.5% $0.30
GPT-5.6 Sol high 56.9% $0.02
Muse Wonder 56.8% $0.10
GPT-5.6 Sol medium 56.4% $0.01
GPT-5.6 Sol low 55.9% $0.01
Claude Opus 4.6 55.8% $0.09
GLM-5.3 55.5% $0.009
GPT-5.6 Terra max 55% $0.10
GPT-5.5 54.3% $0.07
GPT-5.5 Pro 53.9% $0.41
Muse Spark 1.3 53.9% $0.03
Gemini 3.1 Pro high 53.8% $0.05
Gemini 3.8 Flash medium 52.9% $0.01
Gemini 3.1 Pro medium 52.8% $0.02
GPT-5.6 Sol off 52.8% $0.006
Kimi K3 52.7% $0.06
GPT-5.6 Terra xhigh 52.2% $0.02
Claude Opus 4.7 52.2% $0.13
GPT-5.4 52% $0.09
Gemini 3.8 Flash high 51.5% $0.05
GPT-5.6 Terra high 51.4% $0.009
Gemini 3.1 Pro low 51.2% $0.009
Gemini 3.8 Flash low 51.2% $0.006
Claude Haiku 5.5 max 51.1% $0.02
Gemini 3.7 Flash high 50.4% $0.01
Claude Sonnet 5 high 50% $0.03
Muse Spark 1.1 49.9% $0.01
Claude Sonnet 5 medium 49.7% $0.01
Claude Sonnet 5 low 49.6% $0.005
GPT-5.6 Terra medium 49.6% $0.006
Grok 4.7 49.4% $0.08
Claude Sonnet 5 max 49.3% $0.38
GPT-5.6 Terra low 49.3% $0.005
Gemini 3.7 Flash medium 49.2% $0.007
Claude Sonnet 5 xhigh 48.8% $0.07
GPT-5.6 Luna xhigh 48.5% $0.004
GPT-5.6 Luna max 48.5% $0.008
Grok 4.6 48.5% $0.03
Muse Spark 1.2 48.4% $0.02
Claude Haiku 5.5 xhigh 48% $0.002
GPT-5.6 Luna high 47.4% $0.002
Gemini 3.5 Flash 47.1% $0.03
Gemini 3.7 Flash low 47.1% $0.005
DeepSeek V4.1 Flash 47% $0.009
Claude Sonnet 4.6 46.5% $0.07
GPT-5.6 Terra off 46.5% $0.002
Qwen 3.8 Max 46.2% $0.02
GLM-5.2 45.8% $0.02
Claude Haiku 5.5 medium 45.6% $0.0009
Qwen3.8-2.4T-A95B 45.5% $0.02
DeepSeek V4 Pro 0813 45.5% $0.03
Grok 4.5 45.2% $0.01
Claude Haiku 5.5 high 45.2% $0.001
Gemini 3.6 Flash 44.9% $0.02
Claude Haiku 5.5 low 44.8% $0.0008
GPT-6 Luna 44.5% $0.002
Claude Opus 4.5 44.5% $0.06
GPT-5.6 Luna medium 44.3% $0.0007
Qwen3.7-Max 44% $0.02
GPT-5.2 43.9% $0.11
GPT-5.1 43.9% $0.04
GPT-5.6 Luna low 43.8% $0.0005
Gemini 3 Flash 43.1% $0.008
Qwen 3.6 Max (Preview) 42.5% $0.02
DeepSeek V4 Flash 0731 41.7% $0.006
GPT-5.6 Luna off 41.4% $0.0002
Grok 4.20 Multi-agent 41.4% $0.31
Qwen 3.8 27B 41.4% $0.01
DeepSeek-V4-Pro 41.2% $0.006
GPT-5.4 Mini 40.8% $0.05
GPT-5.2 Instant 40.1% $0.005
GPT-5 40% $0.03
o3 39.7% $0.02
Claude Haiku 5.5 off 39.3% $0.0005
Gemma 4 31B IT 39.3% $0.001
Claude Sonnet 4.5 38.8% $0.07
Grok 4.20 38.7% $0.006
o3-pro 38.5% $0.15
Grok 4.3 38.3% $0.004
GPT-5.3 Instant 38.1% $0.005
Qwen3.5 397B-A17B 37.9% $0.02
Gemini 3.5 Flash-Lite 37.6% $0.005
Qwen3.7-Plus 37.6% $0.005
Inkling 37.6% $0.02
Tencent Hy3 37.5% $0.002
Claude Opus 4 37.4% $0.12
Kimi K2.6 37.3% $0.02
Claude Opus 4.1 37.1% $0.13
Nemotron 3 Ultra 36.9% $0.005
GPT-5.4 Nano 36.9% $0.01
Qwen 3.5 Plus (hosted 397B-A17B) 36.4% $0.008
DeepSeek-V4-Flash 35.9% $0.002
Mistral Large 4 35.8% $0.02
Gemini 3.1 Flash-Lite 35% $0.003
Muse Glimmer 34.9% $0.003
Qwen3.6 27B 34.5% $0.01
GPT-5 mini 34.2% $0.005
Qwen3.5-27B 34% $0.01
MiniMax-M3 33.7% $0.003
Qwen3-Max thinking 33.6% $0.04
Qwen 3.6 Plus 33.1% $0.01
Qwen3.5-122B-A10B 32.2% $0.02
Qwen 3.6 Flash 31% $0.005
Claude Haiku 4.5 30.9% $0.02
Qwen 3.6 35B-A3B 29.7% $0.007
Gemma 4 26B A4B 29.7% $0.002
MiMo-V2.5-Pro 29.5% $0.002
Qwen3.5-35B-A3B 29.5% $0.01
Qwen3-235B-A22B-Thinking Jul 2025 29.3% $0.006
Qwen 3.5 Flash (hosted 35B-A3B) 29.1% $0.002
DeepSeek-V3.2-Exp 29.1% $0.001
Claude Sonnet 4 29% $0.04
DeepSeek-V3.2 28.8% $0.001
DeepSeek-V3.1-Terminus 28.6% $0.004
Qwen3-Max 28.3% $0.007
o4-mini 26.5% $0.006
Mistral Medium 3.5 26.1% $0.04
GPT-4.1 25.6% $0.007
Qwen3-235B-A22B 25% $0.01
Qwen3.5-9B 24.5% $0.001
DeepSeek-V3.1 24.3% $0.004
Qwen Plus Apr 2025 24% $0.002
Qwen3-30B-A3B-Instruct Jul 2025 22.4% $0.001
o1 22.3% $0.24
gpt-oss-120b 22.1% $0.001
Qwen3-Next-80B-A3B thinking 21.8% $0.006
Qwen3-Next-80B-A3B 21.4% $0.001
GPT-4.1 mini 21.1% $0.001
Nemotron 3.5 Lightning 30B-A3B 20.9% $0.001
Mistral Small 4 20.6% $0.001
Qwen3-30B-A3B-Thinking Jul 2025 19.5% $0.004
Mistral Medium 3.1 19.1% $0.002
o3-mini 19% $0.03
Qwen3-14B 18.2% $0.004
Gemini 2.5 Flash-Lite Sep 2025 18.1% $0.002
Llama 3.3 70B 17.5% $0.001
Qwen3-32B 17.3% $0.0006
Claude 3 Opus 17% $0.05
Mistral Large 3 16.7% $0.001
Llama 4 Maverick 15.9% $0.0006
Qwen3-30B-A3B 15.8% $0.003
DeepSeek-V3 Mar 2025 15.5% $0.0004
Llama 3.1-70B 14.8% $0.0006
gpt-oss-20b 14.5% $0.0003
Qwen2.5-72B 13.4% $0.004
Ministral 3 8B Instruct 12.7% $0.0003
Ministral 3 14B 12.3% $0.0004
Gemma 3 27B 12.3% $0.0003
Llama 4 Scout 12% $0.0005
Cohere Command A 10.3% $0.008
GPT-3.5 Turbo Jan 2024 9.65% $0.0006
c4ai-command-r-08-2024 9.19% $0.0003
Claude 3 Haiku 8.84% $0.0006
Qwen3-8B 8.82% $0.002
GPT-5 nano 7.93% $0.002
Gemma 2 27B 7.10% $0.0008
Qwen2.5-7B 6.42% $0.0005
Ministral 3B 5.48% $0.0002
GPT-4.1 nano 5.46% $0.0003
Llama 3.1-8B 5.37% $8.60
Command R+ 5.01% $0.003
Gemma 3 12B 4.46% $0.0001
Command R7B 4.45% $0.0001
Gemma 3n E4B 3.16% $0.0001
Gemma 3 4B 2.78% $0.0001

6 models have held number one on the official leaderboard since its release in July 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard 6 leaders
Claude Fable 5.2 70.6%

Source: Official LMCA leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) Highest possible score Claude Fable 5.2 Claude Fable 5.1 Claude Fable 5 Claude Opus 4.6 Claude Opus 4.5 GPT-5 GPT-4.1 0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) Highest possible score Claude Fable 5.2 Claude Fable 5

Source: Official LMCA leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Other

All 43 in Other
  • Personal Superintelligence Index (Meta) In use Meta's composite of seven personal and knowledge-work evals: Agents' Last Exam, AutomationBench-AA, FocusBench, JobBench, GDP.pdf, VBT and DRACO.
  • Artificial Analysis Intelligence Index v5 In use Version 5 of Artificial Analysis's index: nine evaluations in four equal categories, most with private test sets, run three times each in its own agent harness. Gemini 4 Carbon 59.0
  • SAGE v1.1 (Vals) In use Models grade handwritten undergraduate math and computer science work against rubrics. Scored by class-balanced accuracy of each rubric check against expert grading. Claude Opus 5.5 91.3%
  • MTEB Fading Massive Text Embedding Benchmark: retrieval, clustering, classification across languages and modalities. sonar 42%
  • Anthropic ECI (AECI) Only in system cards Index that combines a basket of capability benchmarks into one capability score to track trends across Anthropic models. Claude Opus 5.5 169.4
  • SPEED-Bench Only in system cards Measures speculative decoding across varied prompts and domains. Reports the average acceptance length of draft tokens, so higher means faster generation at the same output. Nemotron 3 Ultra 5.15