Benchmark Almanac

Open CoT Leaderboard

DeepSeek-R1-Distill-Qwen-14B 17.6% leads since January 2025
Not in model cards

Models answer logical reasoning questions with and without writing out their reasoning.

The board reports the best accuracy gain over the direct-answer baseline across a common set of prompting methods.

Released
January 2024
Kind
Language reasoning
Level
University
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Text
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
Strongest model tested
Qwen2.5-72B #166 on the capability index
Last model added
January 2025
Scores refreshed
November 20th, 2026
133
Entries on its own leaderboard, 32 models we track

DeepSeek-R1-Distill-Qwen-14B leads with 17.6% among models released by January 2025 DeepSeek-R1-Distill-Qwen-14B leads with 17.6% among models released by January 2025

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

The newest model with a score here came out in January 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

133 entries. Scroll sideways for the rest.

  1. 17.6 DeepSeek-R1-Distill-Qwen-14B
  2. 16.9 DeepSeek-R1-Distill-Qwen-32B
  3. 16.3 meta-llama/Meta-Llama-3.1-8B-Instruct
  4. 16.1 internlm/internlm2-chat-20b
  5. 16.1 deepseek-ai/DeepSeek-R1-Distill-Llama-8B
  6. 15.3 DeepSeek-R1-Distill-Llama-70B
  7. 14.8 meta-llama/Meta-Llama-3-8B-Instruct
  8. 14.5 Kukedlc/NeuralLLaMa-3-8b-ORPO-v0.3
  9. 14.3 mlabonne/Daredevil-8B-abliterated
  10. 14.1 OpenBuddy/openbuddy-mistral2-7b-v20.2-32k
  11. 14.0 meta-llama/Meta-Llama-3-70B-Instruct
  12. 14.0 arcee-ai/Llama-3.1-SuperNova-Lite
  13. 13.9 Llama-3.1-Nemotron-70B-Instruct
  14. 13.8 01-ai/Yi-1.5-34B-Chat
  15. 13.4 meta-llama/Meta-Llama-3.1-70B-Instruct
  16. 13.3 Kukedlc/NeuralLLaMa-3-8b-DT-v0.1
  17. 13.1 ai21labs/AI21-Jamba-1.5-Mini
  18. 13.1 Phi-3.5-MoE
  19. 12.8 SOLAR-10.7B-Instruct-v1.0
  20. 12.8 mistralai/Ministral-8B-Instruct-2410
  21. 12.6 CohereForAI/c4ai-command-r-plus-08-2024
  22. 12.6 DeepSeek-R1-Distill-Qwen-7B
  23. 12.4 Qwen/Qwen2.5-7B-Instruct
  24. 12.3 NousResearch/Hermes-3-Llama-3.1-70B
  25. 12.3 Qwen/Qwen2.5-14B-Instruct
  26. 12.3 arcee-ai/SuperNova-Medius
  27. 12.3 Aya Expanse 32B
  28. 11.9 internlm/internlm2-chat-7b
  29. 11.8 Qwen/Qwen2.5-32B-Instruct
  30. 11.8 OpenBuddy/openbuddy-yi1.5-9b-v21.1-32k
  31. 11.5 vicgalle/Configurable-Yi-1.5-9B-Chat
  32. 11.4 phi-3-small 7.4B
  33. 11.3 databricks/dbrx-instruct
  34. 11.2 Aya Expanse 8B
  35. 11.2 phi-3-mini 3.8B
  36. 11.1 DebateLabKIT/Llama-3.1-Argunaut-1-8B-SPIN
  37. 10.9 mistralai/Mistral-Small-Instruct-2409
  38. 10.8 mistralai/Mixtral-8x7B-Instruct-v0.1
  39. 10.8 OpenBuddy/openbuddy-llama3-8b-v21.1-8k
  40. 10.8 wenbopan/Faro-Yi-9B-DPO
  41. 10.3 Qwen/Qwen2.5-3B-Instruct
  42. 10.2 mistralai/Mixtral-8x22B-Instruct-v0.1
  43. 9.93 vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B
  44. 9.75 Yi-1.5-9B-Chat
  45. 9.69 mistralai/Mistral-Nemo-Instruct-2407
  46. 9.56 CohereForAI/aya-23-35B
  47. 9.56 Qwen2.5-72B
  48. 9.44 01-ai/Yi-1.5-9B-Chat-16K
  49. 9.33 unsloth/Phi-3.5-mini-instruct
  50. 9.33 Llama 3-70B
  51. 9.06 princeton-nlp/gemma-2-9b-it-SimPO
  52. 8.57 openchat/openchat-3.5-0106
  53. 8.42 meta-llama/Llama-3.2-3B-Instruct
  54. 8.34 Tulu 2 DPO 70B
  55. 8.32 google/gemma-2-9b-it
  56. 8.16 Gemma 2 2B IT
  57. 8.11 allenai/OLMo-7B-0724-Instruct-hf
  58. 8.08 Phi-2
  59. 8.02 01-ai/Yi-34B-Chat
  60. 7.86 Qwen2-7B-Instruct
  61. 7.68 bunnycore/Mnemosyne-7B
  62. 7.59 microsoft/Phi-3.5-mini-instruct
  63. 7.52 allenai/tulu-2-70b
  64. 7.46 ChavyvAkvar/habib-DPO-v3
  65. 7.43 Salesforce/LLaMA-3-8B-SFR-Iterative-DPO-R
  66. 7.41 Mistral-7B-Instruct-v0.2
  67. 7.33 meta-llama/Llama-2-70b-chat-hf
  68. 7.27 Locutusque/Hercules-4.0-Mistral-v0.2-7B
  69. 7.20 internlm/internlm2-7b
  70. 7.17 Deci/DeciLM-7B-instruct
  71. 7.00 Locutusque/OpenCerebrum-1.0-7b-DPO
  72. 6.91 google/gemma-7b-it
  73. 6.89 ichigoberry/pandafish-2-7b-32k
  74. 6.81 Qwen2-72B
  75. 6.74 DeepSeek-R1-Distill-Qwen-1.5B
  76. 6.41 Mixtral 8x7B
  77. 6.35 mistralai/Mistral-7B-Instruct-v0.3
  78. 5.98 vicgalle/Configurable-Janus-7B
  79. 5.93 phi-3-medium 14B
  80. 5.82 internlm/internlm2-math-7b
  81. 5.80 ibm/merlinite-7b
  82. 5.78 google/gemma-2-27b-it
  83. 5.73 LeroyDyer/Mixtral_AI_CyberTron_DeepMind_III_UFT
  84. 5.50 internlm/internlm2-math-20b
  85. 5.47 cognitivecomputations/Llama-3-8B-Instruct-abliterated-v2
  86. 5.32 01-ai/Yi-1.5-9B-32K
  87. 5.29 allenai/tulu-2-7b
  88. 5.25 CohereForAI/aya-23-8B
  89. 5.22 NousResearch/Hermes-2-Pro-Mistral-7B
  90. 5.03 allenai/tulu-2-13b
  91. 5.03 teknium/OpenHermes-2.5-Mistral-7B
  92. 4.93 openbmb/Eurus-70b-sft
  93. 4.91 allenai/tulu-2-dpo-13b
  94. 4.87 Intel/neural-chat-7b-v3-1
  95. 4.70 0-hero/Matter-0.2-7B-DPO
  96. 4.60 DebateLabKIT/Llama-3.1-Argunaut-1-8B-SFT
  97. 4.48 Deci/DeciLM-7B
  98. 4.48 SmolLM2 1.7B Instruct
  99. 4.45 LeroyDyer/Mixtral_AI_Chat_1.0
  100. 4.14 allenai/tulu-2-dpo-7b
  101. 4.04 BSC-LT/salamandra-7b-instruct
  102. 3.89 01-ai/Yi-1.5-9B
  103. 3.88 Mistral 7B v0.1
  104. 3.81 google/gemma-2b-it
  105. 3.65 meta-llama/Llama-2-13b-chat-hf
  106. 3.65 Yi-34B
  107. 3.53 DebateLabKIT/Llama-3.3-Argunaut-1-70B-SFT
  108. 3.49 microsoft/Orca-2-7b
  109. 3.37 DebateLabKIT/Llama-3.3-Argunaut-1-70B-SPIN
  110. 3.30 cookinai/LlamaReflect-8B-CoT-safetensors
  111. 3.13 microsoft/Orca-2-13b
  112. 3.07 meta-llama/Llama-3.2-1B-Instruct
  113. 2.87 NousResearch/Nous-Hermes-llama-2-7b
  114. 2.54 wandb/gemma-2b-zephyr-dpo
  115. 2.47 Kukedlc/NeuralSynthesis-7b-v0.4-slerp
  116. 2.46 meta-llama/Llama-2-13b-hf
  117. 2.45 Zephyr 7B Beta
  118. 2.26 mlabonne/AlphaMonarch-7B
  119. 2.12 upstage/SOLAR-10.7B-v1.0
  120. 2.01 openbmb/Eurus-7b-kto
  121. 1.90 LeroyDyer/Mixtral_AI_Cyber_Boss
  122. 1.87 HuggingFaceTB/SmolLM-1.7B-Instruct
  123. 1.53 Gemma 2B
  124. 1.42 meta-llama/Llama-2-7b-hf
  125. 1.39 HuggingFaceTB/SmolLM-135M-Instruct
  126. 1.21 Yi 6B
  127. 1.11 databricks/dolly-v2-3b
  128. 0.68 Gemma 7B
  129. -0.06 terrycraddock/Reflection-Llama-3.1-8B
  130. -0.23 OpenBuddy/openbuddy-zen-3b-v21.1-32k
  131. -0.33 OpenBuddy/openbuddy-zen-3b-v21.2-32k
  132. -1.33 Llama 2-70B
  133. -2.63 Llama 3-8B
  1. 1 DeepSeek-R1-Distill-Qwen-14B bfloat16 17.6%
  2. 2 DeepSeek-R1-Distill-Qwen-32B bfloat16 16.9%
  3. 3 meta-llama/Meta-Llama-3.1-8B-Instruct bfloat16 16.3%
  4. 4 internlm/internlm2-chat-20b bfloat16 16.1%
  5. 5 deepseek-ai/DeepSeek-R1-Distill-Llama-8B bfloat16 16.1%
  6. 6 DeepSeek-R1-Distill-Llama-70B bfloat16 15.3%
  7. 7 meta-llama/Meta-Llama-3-8B-Instruct bfloat16 14.8%
  8. 8 Kukedlc/NeuralLLaMa-3-8b-ORPO-v0.3 float16 14.5%
  9. 9 mlabonne/Daredevil-8B-abliterated bfloat16 14.3%
  10. 10 OpenBuddy/openbuddy-mistral2-7b-v20.2-32k bfloat16 14.1%
  11. 11 meta-llama/Meta-Llama-3-70B-Instruct bfloat16 14%
  12. 12 arcee-ai/Llama-3.1-SuperNova-Lite bfloat16 14%
  13. 13 Llama-3.1-Nemotron-70B-Instruct bfloat16 13.9%
  14. 14 01-ai/Yi-1.5-34B-Chat bfloat16 13.8%
  15. 15 meta-llama/Meta-Llama-3.1-70B-Instruct bfloat16 13.4%
  16. 16 Kukedlc/NeuralLLaMa-3-8b-DT-v0.1 float16 13.3%
  17. 17 ai21labs/AI21-Jamba-1.5-Mini bfloat16 13.1%
  18. 18 Phi-3.5-MoE bfloat16 13.1%
  19. 19 SOLAR-10.7B-Instruct-v1.0 float16 12.8%
  20. 20 mistralai/Ministral-8B-Instruct-2410 bfloat16 12.8%
  21. 21 CohereForAI/c4ai-command-r-plus-08-2024 bfloat16 12.6%
  22. 22 DeepSeek-R1-Distill-Qwen-7B bfloat16 12.6%
  23. 23 Qwen/Qwen2.5-7B-Instruct bfloat16 12.4%
  24. 24 NousResearch/Hermes-3-Llama-3.1-70B bfloat16 12.3%
  25. 25 Qwen/Qwen2.5-14B-Instruct bfloat16 12.3%
  26. 26 arcee-ai/SuperNova-Medius bfloat16 12.3%
  27. 27 Aya Expanse 32B float16 12.3%
  28. 28 internlm/internlm2-chat-7b bfloat16 11.9%
  29. 29 Qwen/Qwen2.5-32B-Instruct bfloat16 11.8%
  30. 30 OpenBuddy/openbuddy-yi1.5-9b-v21.1-32k bfloat16 11.8%
  31. 31 vicgalle/Configurable-Yi-1.5-9B-Chat float16 11.5%
  32. 32 phi-3-small 7.4B bfloat16 11.4%
  33. 33 databricks/dbrx-instruct bfloat16 11.3%
  34. 34 Aya Expanse 8B float16 11.2%
  35. 35 phi-3-mini 3.8B bfloat16 11.2%
  36. 36 DebateLabKIT/Llama-3.1-Argunaut-1-8B-SPIN bfloat16 11.1%
  37. 37 mistralai/Mistral-Small-Instruct-2409 bfloat16 10.9%
  38. 38 mistralai/Mixtral-8x7B-Instruct-v0.1 bfloat16 10.8%
  39. 39 OpenBuddy/openbuddy-llama3-8b-v21.1-8k bfloat16 10.8%
  40. 40 wenbopan/Faro-Yi-9B-DPO float16 10.8%
  41. 41 Qwen/Qwen2.5-3B-Instruct bfloat16 10.3%
  42. 42 mistralai/Mixtral-8x22B-Instruct-v0.1 bfloat16 10.2%
  43. 43 vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B float16 9.93%
  44. 44 Yi-1.5-9B-Chat bfloat16 9.75%
  45. 45 mistralai/Mistral-Nemo-Instruct-2407 bfloat16 9.69%
  46. 46 CohereForAI/aya-23-35B float16 9.56%
  47. 47 Qwen2.5-72B bfloat16 9.56%
  48. 48 01-ai/Yi-1.5-9B-Chat-16K bfloat16 9.44%
  49. 49 unsloth/Phi-3.5-mini-instruct bfloat16 9.33%
  50. 50 Llama 3-70B bfloat16 9.33%
  51. 51 princeton-nlp/gemma-2-9b-it-SimPO bfloat16 9.06%
  52. 52 openchat/openchat-3.5-0106 bfloat16 8.57%
  53. 53 meta-llama/Llama-3.2-3B-Instruct bfloat16 8.42%
  54. 54 Tulu 2 DPO 70B bfloat16 8.34%
  55. 55 google/gemma-2-9b-it bfloat16 8.32%
  56. 56 Gemma 2 2B IT bfloat16 8.16%
  57. 57 allenai/OLMo-7B-0724-Instruct-hf bfloat16 8.11%
  58. 58 Phi-2 float16 8.08%
  59. 59 01-ai/Yi-34B-Chat bfloat16 8.02%
  60. 60 Qwen2-7B-Instruct bfloat16 7.86%
  61. 61 bunnycore/Mnemosyne-7B bfloat16 7.68%
  62. 62 microsoft/Phi-3.5-mini-instruct bfloat16 7.59%
  63. 63 allenai/tulu-2-70b bfloat16 7.52%
  64. 64 ChavyvAkvar/habib-DPO-v3 float16 7.46%
  65. 65 Salesforce/LLaMA-3-8B-SFR-Iterative-DPO-R bfloat16 7.43%
  66. 66 Mistral-7B-Instruct-v0.2 float16 7.41%
  67. 67 meta-llama/Llama-2-70b-chat-hf float16 7.33%
  68. 68 Locutusque/Hercules-4.0-Mistral-v0.2-7B bfloat16 7.27%
  69. 69 internlm/internlm2-7b bfloat16 7.20%
  70. 70 Deci/DeciLM-7B-instruct float16 7.17%
  71. 71 Locutusque/OpenCerebrum-1.0-7b-DPO bfloat16 7.00%
  72. 72 google/gemma-7b-it bfloat16 6.91%
  73. 73 ichigoberry/pandafish-2-7b-32k bfloat16 6.89%
  74. 74 Qwen2-72B bfloat16 6.81%
  75. 75 DeepSeek-R1-Distill-Qwen-1.5B bfloat16 6.74%
  76. 76 Mixtral 8x7B bfloat16 6.41%
  77. 77 mistralai/Mistral-7B-Instruct-v0.3 bfloat16 6.35%
  78. 78 vicgalle/Configurable-Janus-7B float16 5.98%
  79. 79 phi-3-medium 14B bfloat16 5.93%
  80. 80 internlm/internlm2-math-7b bfloat16 5.82%
  81. 81 ibm/merlinite-7b float16 5.80%
  82. 82 google/gemma-2-27b-it bfloat16 5.78%
  83. 83 LeroyDyer/Mixtral_AI_CyberTron_DeepMind_III_UFT float16 5.73%
  84. 84 internlm/internlm2-math-20b bfloat16 5.50%
  85. 85 cognitivecomputations/Llama-3-8B-Instruct-abliterated-v2 bfloat16 5.47%
  86. 86 01-ai/Yi-1.5-9B-32K bfloat16 5.32%
  87. 87 allenai/tulu-2-7b float16 5.29%
  88. 88 CohereForAI/aya-23-8B float16 5.25%
  89. 89 NousResearch/Hermes-2-Pro-Mistral-7B bfloat16 5.22%
  90. 90 allenai/tulu-2-13b bfloat16 5.03%
  91. 91 teknium/OpenHermes-2.5-Mistral-7B bfloat16 5.03%
  92. 92 openbmb/Eurus-70b-sft float16 4.93%
  93. 93 allenai/tulu-2-dpo-13b bfloat16 4.91%
  94. 94 Intel/neural-chat-7b-v3-1 float16 4.87%
  95. 95 0-hero/Matter-0.2-7B-DPO bfloat16 4.70%
  96. 96 DebateLabKIT/Llama-3.1-Argunaut-1-8B-SFT bfloat16 4.60%
  97. 97 Deci/DeciLM-7B float16 4.48%
  98. 98 SmolLM2 1.7B Instruct bfloat16 4.48%
  99. 99 LeroyDyer/Mixtral_AI_Chat_1.0 float16 4.45%
  100. 100 allenai/tulu-2-dpo-7b bfloat16 4.14%
  101. 101 BSC-LT/salamandra-7b-instruct bfloat16 4.04%
  102. 102 01-ai/Yi-1.5-9B bfloat16 3.89%
  103. 103 Mistral 7B v0.1 float16 3.88%
  104. 104 google/gemma-2b-it bfloat16 3.81%
  105. 105 meta-llama/Llama-2-13b-chat-hf float16 3.65%
  106. 106 Yi-34B bfloat16 3.65%
  107. 107 DebateLabKIT/Llama-3.3-Argunaut-1-70B-SFT bfloat16 3.53%
  108. 108 microsoft/Orca-2-7b float16 3.49%
  109. 109 DebateLabKIT/Llama-3.3-Argunaut-1-70B-SPIN bfloat16 3.37%
  110. 110 cookinai/LlamaReflect-8B-CoT-safetensors bfloat16 3.30%
  111. 111 microsoft/Orca-2-13b float16 3.13%
  112. 112 meta-llama/Llama-3.2-1B-Instruct bfloat16 3.07%
  113. 113 NousResearch/Nous-Hermes-llama-2-7b bfloat16 2.87%
  114. 114 wandb/gemma-2b-zephyr-dpo bfloat16 2.54%
  115. 115 Kukedlc/NeuralSynthesis-7b-v0.4-slerp bfloat16 2.47%
  116. 116 meta-llama/Llama-2-13b-hf float16 2.46%
  117. 117 Zephyr 7B Beta bfloat16 2.45%
  118. 118 mlabonne/AlphaMonarch-7B bfloat16 2.26%
  119. 119 upstage/SOLAR-10.7B-v1.0 float16 2.12%
  120. 120 openbmb/Eurus-7b-kto bfloat16 2.01%
  121. 121 LeroyDyer/Mixtral_AI_Cyber_Boss float16 1.90%
  122. 122 HuggingFaceTB/SmolLM-1.7B-Instruct bfloat16 1.87%
  123. 123 Gemma 2B bfloat16 1.53%
  124. 124 meta-llama/Llama-2-7b-hf float16 1.42%
  125. 125 HuggingFaceTB/SmolLM-135M-Instruct bfloat16 1.39%
  126. 126 Yi 6B float16 1.21%
  127. 127 databricks/dolly-v2-3b bfloat16 1.11%
  128. 128 Gemma 7B bfloat16 0.68%
  129. 129 terrycraddock/Reflection-Llama-3.1-8B float16 -0.06%
  130. 130 OpenBuddy/openbuddy-zen-3b-v21.1-32k bfloat16 -0.23%
  131. 131 OpenBuddy/openbuddy-zen-3b-v21.2-32k bfloat16 -0.33%
  132. 132 Llama 2-70B bfloat16 -1.33%
  133. 133 Llama 3-8B bfloat16 -2.63%

Source: Official Open CoT Leaderboard leaderboard

Source: Official Open CoT Leaderboard leaderboard

#EntryScoreSettingDateLinkSource
1 DeepSeek-R1-Distill-Qwen-14B 17.6% bfloat16 Jan 2025 huggingface.co Official leaderboard
2 DeepSeek-R1-Distill-Qwen-32B 16.9% bfloat16 Jan 2025 huggingface.co Official leaderboard
3 meta-llama/Meta-Llama-3.1-8B-Instruct 8B 16.3% bfloat16 huggingface.co Official leaderboard
4 internlm/internlm2-chat-20b 19.9B 16.1% bfloat16 huggingface.co Official leaderboard
5 deepseek-ai/DeepSeek-R1-Distill-Llama-8B 8B 16.1% bfloat16 huggingface.co Official leaderboard
6 DeepSeek-R1-Distill-Llama-70B 15.3% bfloat16 Jan 2025 huggingface.co Official leaderboard
7 meta-llama/Meta-Llama-3-8B-Instruct 8B 14.8% bfloat16 huggingface.co Official leaderboard
8 Kukedlc/NeuralLLaMa-3-8b-ORPO-v0.3 8B 14.5% float16 huggingface.co Official leaderboard
9 mlabonne/Daredevil-8B-abliterated 8B 14.3% bfloat16 huggingface.co Official leaderboard
10 OpenBuddy/openbuddy-mistral2-7b-v20.2-32k 7.3B 14.1% bfloat16 huggingface.co Official leaderboard
11 meta-llama/Meta-Llama-3-70B-Instruct 70.6B 14% bfloat16 huggingface.co Official leaderboard
12 arcee-ai/Llama-3.1-SuperNova-Lite 8B 14% bfloat16 huggingface.co Official leaderboard
13 Llama-3.1-Nemotron-70B-Instruct 13.9% bfloat16 Oct 2024 huggingface.co Official leaderboard
14 01-ai/Yi-1.5-34B-Chat 34.4B 13.8% bfloat16 huggingface.co Official leaderboard
15 meta-llama/Meta-Llama-3.1-70B-Instruct 70.6B 13.4% bfloat16 huggingface.co Official leaderboard
16 Kukedlc/NeuralLLaMa-3-8b-DT-v0.1 8B 13.3% float16 huggingface.co Official leaderboard
17 ai21labs/AI21-Jamba-1.5-Mini 51.6B 13.1% bfloat16 huggingface.co Official leaderboard
18 Phi-3.5-MoE 13.1% bfloat16 Aug 2024 huggingface.co Official leaderboard
19 SOLAR-10.7B-Instruct-v1.0 12.8% float16 Dec 2023 huggingface.co Official leaderboard
20 mistralai/Ministral-8B-Instruct-2410 8B 12.8% bfloat16 huggingface.co Official leaderboard
21 CohereForAI/c4ai-command-r-plus-08-2024 103.8B 12.6% bfloat16 huggingface.co Official leaderboard
22 DeepSeek-R1-Distill-Qwen-7B 12.6% bfloat16 Jan 2025 huggingface.co Official leaderboard
23 Qwen/Qwen2.5-7B-Instruct 7.6B 12.4% bfloat16 huggingface.co Official leaderboard
24 NousResearch/Hermes-3-Llama-3.1-70B 70.6B 12.3% bfloat16 huggingface.co Official leaderboard
25 Qwen/Qwen2.5-14B-Instruct 14.8B 12.3% bfloat16 huggingface.co Official leaderboard
26 arcee-ai/SuperNova-Medius 14.8B 12.3% bfloat16 huggingface.co Official leaderboard
27 Aya Expanse 32B 12.3% float16 Oct 2024 huggingface.co Official leaderboard
28 internlm/internlm2-chat-7b 7.7B 11.9% bfloat16 huggingface.co Official leaderboard
29 Qwen/Qwen2.5-32B-Instruct 32.8B 11.8% bfloat16 huggingface.co Official leaderboard
30 OpenBuddy/openbuddy-yi1.5-9b-v21.1-32k 8.9B 11.8% bfloat16 huggingface.co Official leaderboard
31 vicgalle/Configurable-Yi-1.5-9B-Chat 8.8B 11.5% float16 huggingface.co Official leaderboard
32 phi-3-small 7.4B 11.4% bfloat16 Apr 2024 huggingface.co Official leaderboard
33 databricks/dbrx-instruct 131.6B 11.3% bfloat16 huggingface.co Official leaderboard
34 Aya Expanse 8B 11.2% float16 Oct 2024 huggingface.co Official leaderboard
35 phi-3-mini 3.8B 11.2% bfloat16 Apr 2024 huggingface.co Official leaderboard
36 DebateLabKIT/Llama-3.1-Argunaut-1-8B-SPIN 8B 11.1% bfloat16 huggingface.co Official leaderboard
37 mistralai/Mistral-Small-Instruct-2409 22.3B 10.9% bfloat16 huggingface.co Official leaderboard
38 mistralai/Mixtral-8x7B-Instruct-v0.1 46.7B 10.8% bfloat16 huggingface.co Official leaderboard
39 OpenBuddy/openbuddy-llama3-8b-v21.1-8k 8B 10.8% bfloat16 huggingface.co Official leaderboard
40 wenbopan/Faro-Yi-9B-DPO 8.8B 10.8% float16 huggingface.co Official leaderboard
41 Qwen/Qwen2.5-3B-Instruct 3.1B 10.3% bfloat16 huggingface.co Official leaderboard
42 mistralai/Mixtral-8x22B-Instruct-v0.1 140.6B 10.2% bfloat16 huggingface.co Official leaderboard
43 vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B 8B 9.93% float16 huggingface.co Official leaderboard
44 Yi-1.5-9B-Chat 9.75% bfloat16 May 2024 huggingface.co Official leaderboard
45 mistralai/Mistral-Nemo-Instruct-2407 12.3B 9.69% bfloat16 huggingface.co Official leaderboard
46 CohereForAI/aya-23-35B 35B 9.56% float16 huggingface.co Official leaderboard
47 Qwen2.5-72B 9.56% bfloat16 Sep 2024 huggingface.co Official leaderboard
48 01-ai/Yi-1.5-9B-Chat-16K 8.8B 9.44% bfloat16 huggingface.co Official leaderboard
49 unsloth/Phi-3.5-mini-instruct 3.8B 9.33% bfloat16 huggingface.co Official leaderboard
50 Llama 3-70B 9.33% bfloat16 Apr 2024 huggingface.co Official leaderboard
51 princeton-nlp/gemma-2-9b-it-SimPO 9.2B 9.06% bfloat16 huggingface.co Official leaderboard
52 openchat/openchat-3.5-0106 7.2B 8.57% bfloat16 huggingface.co Official leaderboard
53 meta-llama/Llama-3.2-3B-Instruct 3.2B 8.42% bfloat16 huggingface.co Official leaderboard
54 Tulu 2 DPO 70B 8.34% bfloat16 Nov 2023 huggingface.co Official leaderboard
55 google/gemma-2-9b-it 9.2B 8.32% bfloat16 huggingface.co Official leaderboard
56 Gemma 2 2B IT 8.16% bfloat16 Jul 2024 huggingface.co Official leaderboard
57 allenai/OLMo-7B-0724-Instruct-hf 6.9B 8.11% bfloat16 huggingface.co Official leaderboard
58 Phi-2 8.08% float16 Dec 2023 huggingface.co Official leaderboard
59 01-ai/Yi-34B-Chat 34.4B 8.02% bfloat16 huggingface.co Official leaderboard
60 Qwen2-7B-Instruct 7.86% bfloat16 Jun 2024 huggingface.co Official leaderboard
61 bunnycore/Mnemosyne-7B 7.2B 7.68% bfloat16 huggingface.co Official leaderboard
62 microsoft/Phi-3.5-mini-instruct 3.8B 7.59% bfloat16 huggingface.co Official leaderboard
63 allenai/tulu-2-70b 70B 7.52% bfloat16 huggingface.co Official leaderboard
64 ChavyvAkvar/habib-DPO-v3 7.2B 7.46% float16 huggingface.co Official leaderboard
65 Salesforce/LLaMA-3-8B-SFR-Iterative-DPO-R 8B 7.43% bfloat16 huggingface.co Official leaderboard
66 Mistral-7B-Instruct-v0.2 7.41% float16 Dec 2023 huggingface.co Official leaderboard
67 meta-llama/Llama-2-70b-chat-hf 69B 7.33% float16 huggingface.co Official leaderboard
68 Locutusque/Hercules-4.0-Mistral-v0.2-7B 7.2B 7.27% bfloat16 huggingface.co Official leaderboard
69 internlm/internlm2-7b 7B 7.20% bfloat16 huggingface.co Official leaderboard
70 Deci/DeciLM-7B-instruct 7B 7.17% float16 huggingface.co Official leaderboard
71 Locutusque/OpenCerebrum-1.0-7b-DPO 7.2B 7.00% bfloat16 huggingface.co Official leaderboard
72 google/gemma-7b-it 8.5B 6.91% bfloat16 huggingface.co Official leaderboard
73 ichigoberry/pandafish-2-7b-32k 7.2B 6.89% bfloat16 huggingface.co Official leaderboard
74 Qwen2-72B 6.81% bfloat16 Jun 2024 huggingface.co Official leaderboard
75 DeepSeek-R1-Distill-Qwen-1.5B 6.74% bfloat16 Jan 2025 huggingface.co Official leaderboard
76 Mixtral 8x7B 6.41% bfloat16 Dec 2023 huggingface.co Official leaderboard
77 mistralai/Mistral-7B-Instruct-v0.3 7.3B 6.35% bfloat16 huggingface.co Official leaderboard
78 vicgalle/Configurable-Janus-7B 7.2B 5.98% float16 huggingface.co Official leaderboard
79 phi-3-medium 14B 5.93% bfloat16 Apr 2024 huggingface.co Official leaderboard
80 internlm/internlm2-math-7b 7.7B 5.82% bfloat16 huggingface.co Official leaderboard
81 ibm/merlinite-7b 7.2B 5.80% float16 huggingface.co Official leaderboard
82 google/gemma-2-27b-it 27.2B 5.78% bfloat16 huggingface.co Official leaderboard
83 LeroyDyer/Mixtral_AI_CyberTron_DeepMind_III_UFT 7.2B 5.73% float16 huggingface.co Official leaderboard
84 internlm/internlm2-math-20b 19.9B 5.50% bfloat16 huggingface.co Official leaderboard
85 cognitivecomputations/Llama-3-8B-Instruct-abliterated-v2 8B 5.47% bfloat16 huggingface.co Official leaderboard
86 01-ai/Yi-1.5-9B-32K 8.8B 5.32% bfloat16 huggingface.co Official leaderboard
87 allenai/tulu-2-7b 7B 5.29% float16 huggingface.co Official leaderboard
88 CohereForAI/aya-23-8B 8B 5.25% float16 huggingface.co Official leaderboard
89 NousResearch/Hermes-2-Pro-Mistral-7B 7.2B 5.22% bfloat16 huggingface.co Official leaderboard
90 allenai/tulu-2-13b 13B 5.03% bfloat16 huggingface.co Official leaderboard
91 teknium/OpenHermes-2.5-Mistral-7B 7.2B 5.03% bfloat16 huggingface.co Official leaderboard
92 openbmb/Eurus-70b-sft 69B 4.93% float16 huggingface.co Official leaderboard
93 allenai/tulu-2-dpo-13b 13B 4.91% bfloat16 huggingface.co Official leaderboard
94 Intel/neural-chat-7b-v3-1 7.2B 4.87% float16 huggingface.co Official leaderboard
95 0-hero/Matter-0.2-7B-DPO 7.2B 4.70% bfloat16 huggingface.co Official leaderboard
96 DebateLabKIT/Llama-3.1-Argunaut-1-8B-SFT 8B 4.60% bfloat16 huggingface.co Official leaderboard
97 Deci/DeciLM-7B 7B 4.48% float16 huggingface.co Official leaderboard
98 SmolLM2 1.7B Instruct 4.48% bfloat16 Oct 2024 huggingface.co Official leaderboard
99 LeroyDyer/Mixtral_AI_Chat_1.0 7.2B 4.45% float16 huggingface.co Official leaderboard
100 allenai/tulu-2-dpo-7b 7B 4.14% bfloat16 huggingface.co Official leaderboard
101 BSC-LT/salamandra-7b-instruct 7.8B 4.04% bfloat16 huggingface.co Official leaderboard
102 01-ai/Yi-1.5-9B 8.8B 3.89% bfloat16 huggingface.co Official leaderboard
103 Mistral 7B v0.1 3.88% float16 Sep 2023 huggingface.co Official leaderboard
104 google/gemma-2b-it 2.5B 3.81% bfloat16 huggingface.co Official leaderboard
105 meta-llama/Llama-2-13b-chat-hf 13B 3.65% float16 huggingface.co Official leaderboard
106 Yi-34B 3.65% bfloat16 Nov 2023 huggingface.co Official leaderboard
107 DebateLabKIT/Llama-3.3-Argunaut-1-70B-SFT 70.6B 3.53% bfloat16 huggingface.co Official leaderboard
108 microsoft/Orca-2-7b 3.49% float16 huggingface.co Official leaderboard
109 DebateLabKIT/Llama-3.3-Argunaut-1-70B-SPIN 70.6B 3.37% bfloat16 huggingface.co Official leaderboard
110 cookinai/LlamaReflect-8B-CoT-safetensors 8B 3.30% bfloat16 huggingface.co Official leaderboard
111 microsoft/Orca-2-13b 3.13% float16 huggingface.co Official leaderboard
112 meta-llama/Llama-3.2-1B-Instruct 1.2B 3.07% bfloat16 huggingface.co Official leaderboard
113 NousResearch/Nous-Hermes-llama-2-7b 6.7B 2.87% bfloat16 huggingface.co Official leaderboard
114 wandb/gemma-2b-zephyr-dpo 2.5B 2.54% bfloat16 huggingface.co Official leaderboard
115 Kukedlc/NeuralSynthesis-7b-v0.4-slerp 7.2B 2.47% bfloat16 huggingface.co Official leaderboard
116 meta-llama/Llama-2-13b-hf 13B 2.46% float16 huggingface.co Official leaderboard
117 Zephyr 7B Beta 2.45% bfloat16 Oct 2023 huggingface.co Official leaderboard
118 mlabonne/AlphaMonarch-7B 7.2B 2.26% bfloat16 huggingface.co Official leaderboard
119 upstage/SOLAR-10.7B-v1.0 10.7B 2.12% float16 huggingface.co Official leaderboard
120 openbmb/Eurus-7b-kto 7.2B 2.01% bfloat16 huggingface.co Official leaderboard
121 LeroyDyer/Mixtral_AI_Cyber_Boss 7.2B 1.90% float16 huggingface.co Official leaderboard
122 HuggingFaceTB/SmolLM-1.7B-Instruct 1.7B 1.87% bfloat16 huggingface.co Official leaderboard
123 Gemma 2B 1.53% bfloat16 Feb 2024 huggingface.co Official leaderboard
124 meta-llama/Llama-2-7b-hf 6.7B 1.42% float16 huggingface.co Official leaderboard
125 HuggingFaceTB/SmolLM-135M-Instruct 0.1B 1.39% bfloat16 huggingface.co Official leaderboard
126 Yi 6B 1.21% float16 Nov 2023 huggingface.co Official leaderboard
127 databricks/dolly-v2-3b 3B 1.11% bfloat16 huggingface.co Official leaderboard
128 Gemma 7B 0.68% bfloat16 Feb 2024 huggingface.co Official leaderboard
129 terrycraddock/Reflection-Llama-3.1-8B 8B -0.06% float16 huggingface.co Official leaderboard
130 OpenBuddy/openbuddy-zen-3b-v21.1-32k 4.8B -0.23% bfloat16 huggingface.co Official leaderboard
131 OpenBuddy/openbuddy-zen-3b-v21.2-32k 4.8B -0.33% bfloat16 huggingface.co Official leaderboard
132 Llama 2-70B -1.33% bfloat16 Jul 2023 huggingface.co Official leaderboard
133 Llama 3-8B -2.63% bfloat16 Apr 2024 huggingface.co Official leaderboard
#ModelScoreSettingDateLinkSource
1 DeepSeek-R1-Distill-Qwen-14B 17.6% bfloat16 Jan 2025 huggingface.co Official leaderboard
2 DeepSeek-R1-Distill-Qwen-32B 16.9% bfloat16 Jan 2025 huggingface.co Official leaderboard
3 DeepSeek-R1-Distill-Llama-70B 15.3% bfloat16 Jan 2025 huggingface.co Official leaderboard
4 Llama-3.1-Nemotron-70B-Instruct 13.9% bfloat16 Oct 2024 huggingface.co Official leaderboard
5 Phi-3.5-MoE 13.1% bfloat16 Aug 2024 huggingface.co Official leaderboard
6 SOLAR-10.7B-Instruct-v1.0 12.8% float16 Dec 2023 huggingface.co Official leaderboard
7 DeepSeek-R1-Distill-Qwen-7B 12.6% bfloat16 Jan 2025 huggingface.co Official leaderboard
8 Aya Expanse 32B 12.3% float16 Oct 2024 huggingface.co Official leaderboard
9 phi-3-small 7.4B 11.4% bfloat16 Apr 2024 huggingface.co Official leaderboard
10 Aya Expanse 8B 11.2% float16 Oct 2024 huggingface.co Official leaderboard
11 phi-3-mini 3.8B 11.2% bfloat16 Apr 2024 huggingface.co Official leaderboard
12 Yi-1.5-9B-Chat 9.75% bfloat16 May 2024 huggingface.co Official leaderboard
13 Qwen2.5-72B 9.56% bfloat16 Sep 2024 huggingface.co Official leaderboard
14 Llama 3-70B 9.33% bfloat16 Apr 2024 huggingface.co Official leaderboard
15 Tulu 2 DPO 70B 8.34% bfloat16 Nov 2023 huggingface.co Official leaderboard
16 Gemma 2 2B IT 8.16% bfloat16 Jul 2024 huggingface.co Official leaderboard
17 Phi-2 8.08% float16 Dec 2023 huggingface.co Official leaderboard
18 Qwen2-7B-Instruct 7.86% bfloat16 Jun 2024 huggingface.co Official leaderboard
19 Mistral-7B-Instruct-v0.2 7.41% float16 Dec 2023 huggingface.co Official leaderboard
20 Qwen2-72B 6.81% bfloat16 Jun 2024 huggingface.co Official leaderboard
21 DeepSeek-R1-Distill-Qwen-1.5B 6.74% bfloat16 Jan 2025 huggingface.co Official leaderboard
22 Mixtral 8x7B 6.41% bfloat16 Dec 2023 huggingface.co Official leaderboard
23 phi-3-medium 14B 5.93% bfloat16 Apr 2024 huggingface.co Official leaderboard
24 SmolLM2 1.7B Instruct 4.48% bfloat16 Oct 2024 huggingface.co Official leaderboard
25 Mistral 7B v0.1 3.88% float16 Sep 2023 huggingface.co Official leaderboard
26 Yi-34B 3.65% bfloat16 Nov 2023 huggingface.co Official leaderboard
27 Zephyr 7B Beta 2.45% bfloat16 Oct 2023 huggingface.co Official leaderboard
28 Gemma 2B 1.53% bfloat16 Feb 2024 huggingface.co Official leaderboard
29 Yi 6B 1.21% float16 Nov 2023 huggingface.co Official leaderboard
30 Gemma 7B 0.68% bfloat16 Feb 2024 huggingface.co Official leaderboard
31 Llama 2-70B -1.33% bfloat16 Jul 2023 huggingface.co Official leaderboard
32 Llama 3-8B -2.63% bfloat16 Apr 2024 huggingface.co Official leaderboard

Rankings by part

We read 5 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as LogiQA Δ, LogiQA2 Δ and LSAT-ar Δ. DeepSeek-R1-Distill-Qwen-14B leads 1 of them, and 4 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.

LogiQA Δ

32 models we track have a score here.

  1. 1 DeepSeek-R1-Distill-Qwen-14B 13.7%
  2. 2 DeepSeek-R1-Distill-Llama-70B ↑1 12.6%
  3. 3 DeepSeek-R1-Distill-Qwen-7B ↑4 10.4%
  4. 4 DeepSeek-R1-Distill-Qwen-32B ↓2 10.1%
  5. 5 Phi-3.5-MoE 8.47%
  6. 6 Yi-1.5-9B-Chat ↑6 7.51%
  7. 7 Qwen2.5-72B ↑6 7.51%
  8. 8 Aya Expanse 8B ↑2 7.19%
  9. 9 Qwen2-7B-Instruct ↑9 6.55%
  10. 10 Phi-2 ↑7 6.23%

4 models have held number one on the official leaderboard since its release in January 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Upstage Microsoft NVIDIA DeepSeek
Own leaderboard Not tested since Jan 2025

Source: Official Open CoT Leaderboard leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) DeepSeek-R1-Distill-Qwen-14B Llama-3.1-Nemotron-70B-Instruct SOLAR-10.7B-Instruct-v1.0 Tulu 2 DPO 70B Mistral 7B v0.1 0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) DeepSeek-R1-Distill-Qwen-14B SOLAR-10.7B-Instruct-v1.0

Source: Official Open CoT Leaderboard leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Knowledge

All 332 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • MMLU In use 57-subject multiple-choice exam questions from elementary to professional level. Scored by accuracy. Seed1.8 92.3%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • HLE-Diamond In use Models answer 1,000 cleaned expert questions from Humanity's Last Exam, 500 on reasoning and 500 on knowledge, closed book. Graded on exact answers. Claude Fable 5.2 77.9%

Data audit

9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 22 months ago, in January 2025, for DeepSeek-R1-Distill-Llama-70B. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (133 entries).
  • Pass Models scored 32 models we track, plus 101 other entries on its leaderboard.
  • Warning Newest score The newest came 22 months ago, in January 2025, for DeepSeek-R1-Distill-Llama-70B.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166).
  • Pass Headroom The best is 17.6%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on huggingface.co: 32 rows for models we track and 101 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check