nocot-bench (NCRI)

GPT-6 Astra 159.0 leads since September 2026

Only in system cards

Reasoning problems answered without chain of thought (e.g., Brew multi-hop tasks). Measures forward-pass reasoning relevant to monitorability.

Made by: Neel Nanda et al.
Released: September 2026
Kind: Deception and alignment
Scored: Checked automatically
Answers: Short answer
Tasks: All public

Strongest model tested: GPT-6 Astra #3 on the capability index
Last in a system card: October 2026
Last model added: November 2026
Scores refreshed: November 12th, 2026

neelnanda-io/nocot-bench 28 stars

292

Entries on its own leaderboard, 185 models we track

System card reports it, from 1 lab

GPT-6 Astra leads with 159.0

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

Bars Table

292 entries. Scroll sideways for the rest.

1 GPT-6 Astra NCRI 15.2; no filler; placed 159.0
2 Gemini 4 Argon NCRI 15.2; no filler; placed_after_release 151.2
3 GPT-6.1 Sol NCRI 15.2; no filler; placed_after_release 149.0
4 GPT-6.1 Terra NCRI 15.2; no filler; placed_after_release 130.5
5 Claude Fable 5.1 NCRI 15.2; no filler; placed 128.0
6 Gemini 3.7 Flash NCRI 15.2; no filler; sealed 126.1
7 Gemini 3.8 Flash NCRI 15.2; no filler; placed 125.8
8 Claude Fable 5 NCRI 15.2; no filler; sealed 119.0
9 Claude Opus 5.5 NCRI 15.2; dots@512; placed_after_release 118.9
10 Gemini 3.1 Pro NCRI 15.2; no filler; sealed 115.3
11 Claude Opus 5 NCRI 15.2; no filler; sealed 115.1
12 GPT-6 Sol NCRI 15.2; no filler; placed_after_release 114.6
13 GPT-5.6 Sol NCRI 15.2; no filler; sealed 109.4
14 Claude Opus 4.6 NCRI 15.2; no filler; sealed 104.3
15 Claude Opus 4.7 NCRI 15.2; no filler; sealed 103.7
16 Gemini 3 Flash NCRI 15.2; no filler; sealed 103.2
17 Kimi K3.1 Pro NCRI 15.2; no filler; placed_after_release 102.8
18 Claude Opus 4.5 NCRI 15.2; no filler; sealed 102.7
19 Claude Opus 4.1 NCRI 15.2; no filler; sealed 102.6
20 Claude Opus 4.8 NCRI 15.2; no filler; sealed 102.0
21 Gemini 3.5 Flash NCRI 15.2; no filler; sealed 101.5
22 Claude Opus 4 NCRI 15.2; no filler; sealed 101.4
23 Gemini 3.6 Flash NCRI 15.2; no filler; sealed 101.0
24 GPT-5.5 NCRI 15.2; no filler; sealed 100.2
25 google/gemini-2.5-pro (snapshot unspecified) NCRI 15.2; no filler; sealed 99.8
26 Muse Spark 1.3 NCRI 15.2; no filler; placed 98.3
27 Muse Wonder NCRI 15.2; no filler; placed_after_release 97.7
28 Muse Spark 1.2 NCRI 15.2; no filler; sealed 97.1
29 Muse Spark 1.1 NCRI 15.2; no filler; sealed 96.8
30 Kimi K3 NCRI 15.2; no filler; sealed 96.8
31 Muse Spark Open NCRI 15.2; no filler; placed_after_release 95.7
32 Claude Sonnet 5.5 NCRI 15.2; no filler; placed_after_release 95.1
33 Tencent Hy4 NCRI 15.2; no filler; placed_after_release 94.0
34 GPT-5.4 NCRI 15.2; no filler; sealed 94.0
35 Claude Sonnet 4.5 NCRI 15.2; no filler; sealed 94.0
36 Claude Sonnet 4 NCRI 15.2; no filler; sealed 91.3
37 Claude Sonnet 5 NCRI 15.2; no filler; sealed 90.8
38 GPT-5.6 Terra NCRI 15.2; no filler; sealed 89.0
39 Claude Sonnet 4.6 NCRI 15.2; no filler; sealed 87.8
40 Qwen 3.6 Max (Preview) NCRI 15.2; no filler; sealed 87.0
41 Grok 4.6 NCRI 15.2; no filler; sealed 86.8
42 Gemini 3.1 Flash-Lite NCRI 15.2; no filler; sealed 86.5
43 Qwen3.8-2.4T-A95B NCRI 15.2; no filler; sealed 86.0
44 GPT-5.2 NCRI 15.2; no filler; sealed 85.8
45 Qwen3.7-Max NCRI 15.2; no filler; sealed 85.8
46 Qwen 3.8 Max NCRI 15.2; no filler; sealed 85.4
47 GPT-5.3 Codex NCRI 15.2; no filler; sealed 85.2
48 DeepSeek V4 Pro 0813 NCRI 15.2; no filler; sealed 84.1
49 DeepSeek-V4-Pro NCRI 15.2; no filler; sealed 84.1
50 Qwen3-Max NCRI 15.2; no filler; sealed 83.3
51 Qwen3.7-Plus NCRI 15.2; no filler; sealed 83.1
52 Tencent Hy4 preview NCRI 15.2; no filler; sealed 82.8
53 LongCat-2.0 NCRI 15.2; no filler; sealed 82.5
54 Qwen 3.6 Plus NCRI 15.2; no filler; sealed 82.2
55 Gemma 4 31B IT NCRI 15.2; no filler; sealed 81.7
56 DeepSeek V4.1 Flash 1017 NCRI 15.2; no filler; placed_after_release 81.6
57 DeepSeek V4.1 Flash NCRI 15.2; no filler; placed_after_release 81.3
58 Grok 4.5 NCRI 15.2; no filler; sealed 81.0
59 kwaipilot/kat-coder-pro-v2 NCRI 15.2; no filler; sealed 80.8
60 Gemini 3.5 Flash-Lite NCRI 15.2; no filler; sealed 80.6
61 qwen/qwen3.5-plus-02-15 NCRI 15.2; no filler; sealed 80.6
62 Kimi K2.5 NCRI 15.2; no filler; sealed 80.6
63 Qwen3.5 397B-A17B NCRI 15.2; no filler; sealed 80.0
64 Kimi K2.6 NCRI 15.2; no filler; sealed 79.2
65 local/deepseek-v4-flash-base NCRI 15.2; no filler; sealed 79.1
66 google/gemini-2.5-flash (snapshot unspecified) NCRI 15.2; no filler; sealed 78.8
67 nvidia/nemotron-3-ultra-550b-a55b NCRI 15.2; no filler; sealed 78.3
68 Inkling NCRI 15.2; no filler; sealed 78.3
69 DeepSeek-V4-Flash NCRI 15.2; no filler; sealed 78.1
70 moonshotai/kimi-k2 (snapshot unspecified) NCRI 15.2; no filler; sealed 78.0
71 DeepSeek-V3.1-Terminus NCRI 15.2; no filler; sealed 77.6
72 GLM-5.2 NCRI 15.2; no filler; sealed 77.3
73 GPT-6 Luna NCRI 15.2; no filler; placed_after_release 77.3
74 Qwen3-VL-235B-A22B-Instruct NCRI 15.2; no filler; sealed 77.3
75 MiMo-V2.5-Pro NCRI 15.2; no filler; sealed 77.1
76 DeepSeek-V3.2-Exp NCRI 15.2; no filler; sealed 76.7
77 GPT-5.6 Luna NCRI 15.2; no filler; sealed 76.6
78 deepseek/deepseek-chat-v3.1 NCRI 15.2; no filler; sealed 76.5
79 GLM-5.3-Flash NCRI 15.2; no filler; sealed 76.4
80 GLM-5 NCRI 15.2; no filler; sealed 76.2
81 DeepSeek V4 Flash 0731 NCRI 15.2; no filler; sealed 75.5
82 GLM-5.1 NCRI 15.2; no filler; sealed 75.2
83 GPT-4.1 NCRI 15.2; no filler; sealed 75.1
84 DeepSeek-V3.2 NCRI 15.2; no filler; sealed 75.0
85 qwen/qwen3-235b-a22b-2507 NCRI 15.2; no filler; sealed 74.6
86 GLM-5.3 NCRI 15.2; no filler; sealed 74.5
87 Qwen3.8 Flash NCRI 15.2; no filler; sealed 74.4
88 deepseek/deepseek-chat-v3-0324 NCRI 15.2; no filler; sealed 74.4
89 openai/gpt-4o (snapshot unspecified) NCRI 15.2; no filler; sealed 74.2
90 GPT-4o Aug 2024, NCRI 15.2; no filler; sealed 74.2
91 Claude Haiku 4.5 NCRI 15.2; no filler; sealed 74.2
92 Ling-2.6-1T NCRI 15.2; no filler; sealed 73.9
93 openai/gpt-4 (snapshot unspecified) NCRI 15.2; no filler; sealed 73.7
94 WizardLM-2 8x22B NCRI 15.2; no filler; sealed 73.2
95 Qwen3-Coder-480B-A35B NCRI 15.2; no filler; sealed 73.1
96 deepseek/deepseek-chat NCRI 15.2; no filler; sealed 72.9
97 GPT-4 Jun 2023, NCRI 15.2; no filler; sealed 72.8
98 MiniMax-M3 NCRI 15.2; no filler; sealed 72.6
99 Tencent Hy3 NCRI 15.2; no filler; sealed 72.5
100 Qwen3.5-Plus NCRI 15.2; no filler; sealed 72.3
101 Mercury 3 NCRI 15.2; no filler; placed_after_release 72.3
102 Tencent Hy3 preview NCRI 15.2; no filler; sealed 72.3
103 GPT-4o May 2024, NCRI 15.2; no filler; sealed 72.2
104 GLM-4.7 NCRI 15.2; no filler; sealed 71.3
105 openai/gpt-4-turbo (snapshot unspecified) NCRI 15.2; no filler; sealed 71.3
106 GLM-4.5 NCRI 15.2; no filler; sealed 71.3
107 Qwen3-Coder-Plus NCRI 15.2; no filler; sealed 71.2
108 GLM-4.6 NCRI 15.2; no filler; sealed 70.8
109 GPT-5.1 NCRI 15.2; no filler; sealed 70.6
110 Qwen3-235B-A22B NCRI 15.2; no filler; sealed 70.5
111 Llama 4 Maverick NCRI 15.2; no filler; sealed 70.5
112 GPT-5.4 Mini NCRI 15.2; no filler; sealed 70.5
113 GPT-4o Nov 2024, NCRI 15.2; no filler; sealed 70.3
114 Inkling-Small NCRI 15.2; no filler; sealed 70.2
115 DiffusionGemma 26B A4B NCRI 15.2; no filler; sealed 70.2
116 MiMo-V2.5 NCRI 15.2; no filler; sealed 69.9
117 google/gemma-4-26b-a4b-it NCRI 15.2; no filler; sealed 69.5
118 Qwen3.5-122B-A10B NCRI 15.2; no filler; sealed 69.4
119 Grok 4.3 NCRI 15.2; no filler; sealed 69.2
120 Qwen3-235B-A22B-Thinking Jul 2025, NCRI 15.2; no filler; sealed 68.9
121 Qwen3-Next-80B-A3B-Instruct NCRI 15.2; no filler; sealed 68.4
122 MiniMax-M3.1-Flash NCRI 15.2; no filler; placed_after_release 68.4
123 Claude Haiku 5.5 NCRI 15.2; no filler; placed_after_release 68.3
124 local/muse-glimmer-30b NCRI 15.2; no filler; sealed 67.9
125 Mistral Medium 3.5 NCRI 15.2; no filler; sealed 67.5
126 Qwen3.6 27B NCRI 15.2; no filler; sealed 67.4
127 Grok 4.20 NCRI 15.2; no filler; sealed 67.1
128 GPT-5 NCRI 15.2; no filler; sealed 67.0
129 baidu/ernie-4.5-vl-424b-a47b NCRI 15.2; no filler; sealed 66.9
130 GLM-4.5V NCRI 15.2; no filler; sealed 66.7
131 Qwen3.5-27B NCRI 15.2; no filler; sealed 66.6
132 Ling-3.0-flash NCRI 15.2; no filler; sealed 66.3
133 meta/muse-glimmer-30b NCRI 15.2; no filler; sealed 66.3
134 nvidia/nemotron-3-super-120b-a12b NCRI 15.2; no filler; sealed 66.2
135 Qwen 3.8 27B NCRI 15.2; no filler; sealed 66.1
136 Mercury 2.5 NCRI 15.2; no filler; placed 65.8
137 inclusionai/ling-2.6-flash NCRI 15.2; no filler; sealed 65.5
138 Qwen3-Coder-Next NCRI 15.2; no filler; sealed 65.5
139 Solar Pro 4 NCRI 15.2; no filler; sealed 65.3
140 Qwen2.5-VL-72B NCRI 15.2; no filler; sealed 65.2
141 GPT-4.1 mini NCRI 15.2; no filler; sealed 65.0
142 MiniMax-M2.5 NCRI 15.2; no filler; sealed 65.0
143 reflection/beam-1.0-501b-a23b NCRI 15.2; no filler; placed_after_release 64.8
144 nousresearch/hermes-3-llama-3.1-405b NCRI 15.2; no filler; sealed 64.6
145 local/qwen3.5-35b-a3b-base NCRI 15.2; no filler; sealed 64.5
146 google/gemini-2.5-flash-lite (snapshot unspecified) NCRI 15.2; no filler; sealed 64.5
147 MiniMax-M2.7 NCRI 15.2; no filler; sealed 64.0
148 Qwen3-VL-32B-Instruct NCRI 15.2; no filler; sealed 64.0
149 Qwen3-Next-80B-A3B NCRI 15.2; no filler; sealed 63.8
150 sao10k/l3.3-euryale-70b NCRI 15.2; no filler; sealed 63.8
151 mistralai/mistral-small-2603 NCRI 15.2; no filler; sealed 63.5
152 Qwen2.5-72B NCRI 15.2; no filler; sealed 63.5
153 GLM-4.6V NCRI 15.2; no filler; sealed 63.4
154 Qwen 3.6 Flash NCRI 15.2; no filler; sealed 63.3
155 Qwen3.7 Flash NCRI 15.2; no filler; sealed 63.3
156 Hermes 4 405B NCRI 15.2; no filler; sealed 62.8
157 meta-llama/llama-3.1-70b-instruct NCRI 15.2; no filler; sealed 62.8
158 local/qwen2.5-32b-instruct NCRI 15.2; no filler; sealed 62.7
159 Llama 3.3 70B Instruct NCRI 15.2; no filler; sealed 62.6
160 Laguna S 2.1 NCRI 15.2; no filler; sealed 62.5
161 GLM-4.5-Air NCRI 15.2; no filler; sealed 62.4
162 qwen/qwen3-30b-a3b-instruct-2507 NCRI 15.2; no filler; sealed 62.1
163 Qwen 3.6 35B-A3B NCRI 15.2; no filler; sealed 61.6
164 qwen/qwen3.5-flash-02-23 NCRI 15.2; no filler; sealed 61.4
165 amazon/nova-pro-v1 NCRI 15.2; no filler; sealed 61.3
166 Qwen2.5-32B NCRI 15.2; no filler; sealed 61.1
167 sao10k/l3.1-euryale-70b NCRI 15.2; no filler; sealed 61.0
168 Qwen2-72B NCRI 15.2; no filler; sealed 61.0
169 Qwen3.5-35B-A3B NCRI 15.2; no filler; sealed 60.1
170 Qwen2.5-Coder-32B-Instruct NCRI 15.2; no filler; sealed 60.0
171 Qwen3-VL-30B-A3B-Instruct NCRI 15.2; no filler; sealed 59.9
172 Gemma 4 12B NCRI 15.2; no filler; sealed 59.7
173 Qwen3-Coder-30B-A3B-Instruct NCRI 15.2; no filler; sealed 59.6
174 qwen/qwen3-coder-flash NCRI 15.2; no filler; sealed 59.5
175 Ripple-65B Preview NCRI 15.2; no filler; placed_after_release 59.3
176 Qwen3-32B NCRI 15.2; no filler; sealed 59.2
177 thedrummer/cydonia-24b-v4.1 NCRI 15.2; no filler; sealed 59.1
178 gpt-oss-120b NCRI 15.2; no filler; sealed 59.0
179 mistralai/ministral-14b-2512 NCRI 15.2; no filler; sealed 58.7
180 Phi-4 NCRI 15.2; no filler; sealed 58.3
181 nousresearch/hermes-4-70b NCRI 15.2; no filler; sealed 57.9
182 Llama 4 Scout NCRI 15.2; no filler; sealed 57.7
183 Llama 3.1-70B NCRI 15.2; no filler; sealed 57.6
184 Laguna XS 2.1 NCRI 15.2; no filler; sealed 57.5
185 local/gemma-3-27b-pt NCRI 15.2; no filler; sealed 57.5
186 Claude 3 Haiku NCRI 15.2; no filler; sealed 57.3
187 local/mistral-small-24b-base-2501 NCRI 15.2; no filler; sealed 56.2
188 Nemotron 3.5 Lightning 30B-A3B NCRI 15.2; no filler; sealed 56.1
189 Qwen3-VL-8B-Instruct NCRI 15.2; no filler; sealed 56.0
190 Gemma 3 27B NCRI 15.2; no filler; sealed 56.0
191 glm-4.7-flash NCRI 15.2; no filler; sealed 55.9
192 local/gemma-3-27b-it-hf NCRI 15.2; no filler; sealed 55.8
193 GPT-4o mini NCRI 15.2; no filler; sealed 55.7
194 mistralai/mistral-small-24b-instruct-2501 NCRI 15.2; no filler; sealed 55.4
195 openai/gpt-3.5-turbo (snapshot unspecified) NCRI 15.2; no filler; sealed 55.2
196 Mercury 2 NCRI 15.2; no filler; sealed 55.1
197 Qwen3-14B NCRI 15.2; no filler; sealed 55.0
198 gpt-oss-20b NCRI 15.2; no filler; sealed 54.8
199 Olmo 3.1 32B Instruct NCRI 15.2; no filler; placed 54.3
200 local/qwen3.5-9b-base NCRI 15.2; no filler; sealed 54.2
201 local/olmo-3-32b-dpo NCRI 15.2; no filler; sealed 54.0
202 mistralai/mistral-small-3.2-24b-instruct NCRI 15.2; no filler; sealed 53.8
203 local/olmo-3-32b-rl NCRI 15.2; no filler; sealed 53.8
204 local/olmo-3-32b-mid NCRI 15.2; no filler; sealed 53.3
205 Gemma 2 27B NCRI 15.2; no filler; sealed 53.3
206 local/olmo-3-32b-sft NCRI 15.2; no filler; sealed 52.8
207 google/gemma-2-27b-it NCRI 15.2; no filler; sealed 52.6
208 local/nanbeige42-r2 NCRI 15.2; no filler; placed_after_release 52.6
209 local/qwen1.5-110b-chat NCRI 15.2; no filler; sealed 52.6
210 Qwen3.5-9B NCRI 15.2; no filler; sealed 52.4
211 local/ouro26-r4 NCRI 15.2; no filler; placed_after_release 51.7
212 Qwen3-8B NCRI 15.2; no filler; sealed 51.4
213 mistralai/ministral-8b-2512 NCRI 15.2; no filler; sealed 51.0
214 GPT-5 mini NCRI 15.2; no filler; sealed 50.7
215 Gemma 4 E4B NCRI 15.2; no filler; sealed 50.6
216 Qwen3-30B-A3B NCRI 15.2; no filler; sealed 50.4
217 GPT-4.1 nano NCRI 15.2; no filler; sealed 50.2
218 local/gemma-3-12b-pt NCRI 15.2; no filler; sealed 49.9
219 Qwen1.5-72B NCRI 15.2; no filler; sealed 49.7
220 GPT-3.5 Turbo Instruct NCRI 15.2; no filler; sealed 49.6
221 GPT-5.4 Nano NCRI 15.2; no filler; sealed 49.4
222 local/gemma-3-12b-it NCRI 15.2; no filler; sealed 49.3
223 Hunyuan-A13B-Instruct NCRI 15.2; no filler; sealed 48.7
224 qwen/qwen-2.5-7b-instruct NCRI 15.2; no filler; sealed 48.4
225 local/gemma-2-9b-it NCRI 15.2; no filler; sealed 47.5
226 Nemotron 3 Nano 30B-A3B NCRI 15.2; no filler; sealed 47.3
227 local/ouro14-r4 NCRI 15.2; no filler; placed_after_release 47.3
228 google/gemma-3-12b-it NCRI 15.2; no filler; sealed 46.8
229 mistralai/ministral-3b-2512 NCRI 15.2; no filler; sealed 46.2
230 local/qwen3.5-4b-base NCRI 15.2; no filler; sealed 46.0
231 local/olmo-3-32b-pretrain NCRI 15.2; no filler; sealed 45.0
232 Gemma 2 9B NCRI 15.2; no filler; sealed 44.7
233 Qwen3.5-4B NCRI 15.2; no filler; sealed 44.3
234 Gemma 4 E2B NCRI 15.2; no filler; sealed 44.0
235 UI-TARS-1.5 NCRI 15.2; no filler; sealed 43.9
236 Mistral-Nemo-Base-2407 NCRI 15.2; no filler; sealed 43.2
237 thedrummer/unslopnemo-12b NCRI 15.2; no filler; sealed 42.8
238 Mistral NeMo NCRI 15.2; no filler; sealed 42.5
239 local/llama-3.1-8b-instruct-fp8 NCRI 15.2; no filler; sealed 41.8
240 meta-llama/llama-3.1-8b-instruct NCRI 15.2; no filler; sealed 41.6
241 local/llama-3.1-8b-instruct NCRI 15.2; no filler; sealed 40.9
242 local/olmo3-32b-step260000 NCRI 15.2; no filler; placed 40.8
243 local/llama-2-70b-hf NCRI 15.2; no filler; sealed 40.7
244 sao10k/l3-lunaris-8b NCRI 15.2; no filler; sealed 40.7
245 Llama 3.1-8B NCRI 15.2; no filler; sealed 40.6
246 local/llama-3-8b-instruct NCRI 15.2; no filler; sealed 39.2
247 Gemma 3n E4B NCRI 15.2; no filler; sealed 38.5
248 local/olmo3-32b-step360000 NCRI 15.2; no filler; placed 38.5
249 google/gemma-3-4b-it NCRI 15.2; no filler; sealed 37.9
250 LLaMA-65B NCRI 15.2; no filler; sealed 37.5
251 aion-labs/aion-rp-llama-3.1-8b NCRI 15.2; no filler; sealed 37.2
252 ibm-granite/granite-4.0-h-micro NCRI 15.2; no filler; sealed 37.2
253 local/olmo3-32b-step130000 NCRI 15.2; no filler; placed 36.2
254 Qwen3.5-2B NCRI 15.2; no filler; sealed 35.5
255 local/gemma-3-4b-it NCRI 15.2; no filler; sealed 35.3
256 local/olmo3-32b-step92000 NCRI 15.2; no filler; placed 34.5
257 local/olmo3-32b-step182000 NCRI 15.2; no filler; sealed 34.4
258 davinci-002 NCRI 15.2; no filler; sealed 32.6
259 local/olmo3-7b-step1413814 NCRI 15.2; no filler; sealed 32.0
260 local/olmo3-32b-step65000 NCRI 15.2; no filler; placed 31.4
261 GPT-5 nano NCRI 15.2; no filler; sealed 31.0
262 local/olmo3-32b-step46000 NCRI 15.2; no filler; sealed 29.5
263 local/olmo3-32b-step33000 NCRI 15.2; no filler; placed 27.4
264 local/gemma-7b-it NCRI 15.2; no filler; sealed 25.8
265 undi95/remm-slerp-l2-13b NCRI 15.2; no filler; sealed 25.7
266 Qwen3.5-0.8B NCRI 15.2; no filler; sealed 24.9
267 local/olmo3-7b-step360000 NCRI 15.2; no filler; sealed 24.6
268 local/gemma-2-2b NCRI 15.2; no filler; sealed 24.3
269 Gemma 2 2B IT NCRI 15.2; no filler; sealed 24.1
270 local/talkie-1930-13b-it NCRI 15.2; no filler; sealed 23.9
271 gryphe/mythomax-l2-13b NCRI 15.2; no filler; sealed 23.8
272 local/olmo3-7b-step714000 NCRI 15.2; no filler; sealed 23.2
273 local/olmo3-7b-step182000 NCRI 15.2; no filler; sealed 22.7
274 local/olmo3-32b-step16000 NCRI 15.2; no filler; placed 21.5
275 local/olmo3-32b-step23000 NCRI 15.2; no filler; placed 20.4
276 local/olmo3-7b-step92000 NCRI 15.2; no filler; sealed 18.9
277 local/llama-2-7b-hf NCRI 15.2; no filler; sealed 18.7
278 local/huginn-r12 NCRI 15.2; no filler; placed_after_release 16.8
279 local/gemma-3-1b-it NCRI 15.2; no filler; sealed 16.1
280 local/olmo3-32b-step12000 NCRI 15.2; no filler; placed 15.6
281 LLaMA-7B NCRI 15.2; no filler; sealed 15.0
282 local/gemma-2b-it NCRI 15.2; no filler; sealed 11.9
283 local/talkie-web-13b-base NCRI 15.2; no filler; sealed 10.6
284 openai/babbage-002 NCRI 15.2; no filler; sealed 6.6
285 local/olmo3-7b-step46000 NCRI 15.2; no filler; sealed 4.1
286 local/olmo3-7b-step23000 NCRI 15.2; no filler; sealed 3.1
287 local/olmo3-7b-step12000 NCRI 15.2; no filler; sealed -8.3
288 local/pythia-12b NCRI 15.2; no filler; sealed -8.6
289 local/olmo3-32b-step6000 NCRI 15.2; no filler; placed -9.9
290 local/olmo3-7b-step6000 NCRI 15.2; no filler; sealed -11.5
291 local/talkie-1930-13b-base NCRI 15.2; no filler; sealed -12.4
292 local/gemma-3-270m-it NCRI 15.2; no filler; sealed -23.1

Source: Official nocot-bench (NCRI) leaderboard

#	Entry	Score	Setting	Date	Source
1	GPT-6 Astra	159.0	NCRI 15.2; no filler; placed	Sep 2026	Official leaderboard
2	Gemini 4 Argon	151.2	NCRI 15.2; no filler; placed_after_release	Sep 2026	Official leaderboard
3	GPT-6.1 Sol	149.0	NCRI 15.2; no filler; placed_after_release	Sep 2026	Official leaderboard
4	GPT-6.1 Terra	130.5	NCRI 15.2; no filler; placed_after_release	Oct 2026	Official leaderboard
5	Claude Fable 5.1	128.0	NCRI 15.2; no filler; placed	Sep 2026	Official leaderboard
6	Gemini 3.7 Flash	126.1	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
7	Gemini 3.8 Flash	125.8	NCRI 15.2; no filler; placed	Sep 2026	Official leaderboard
8	Claude Fable 5	119.0	NCRI 15.2; no filler; sealed	Jun 2026	Official leaderboard
9	Claude Opus 5.5	118.9	NCRI 15.2; dots@512; placed_after_release	Sep 2026	Official leaderboard
10	Gemini 3.1 Pro	115.3	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
11	Claude Opus 5	115.1	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
12	GPT-6 Sol	114.6	NCRI 15.2; no filler; placed_after_release	Sep 2026	Official leaderboard
13	GPT-5.6 Sol	109.4	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
14	Claude Opus 4.6	104.3	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
15	Claude Opus 4.7	103.7	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
16	Gemini 3 Flash	103.2	NCRI 15.2; no filler; sealed	Dec 2025	Official leaderboard
17	Kimi K3.1 Pro	102.8	NCRI 15.2; no filler; placed_after_release	Oct 2026	Official leaderboard
18	Claude Opus 4.5	102.7	NCRI 15.2; no filler; sealed	Nov 2025	Official leaderboard
19	Claude Opus 4.1	102.6	NCRI 15.2; no filler; sealed	Aug 2025	Official leaderboard
20	Claude Opus 4.8	102.0	NCRI 15.2; no filler; sealed	May 2026	Official leaderboard
21	Gemini 3.5 Flash	101.5	NCRI 15.2; no filler; sealed	May 2026	Official leaderboard
22	Claude Opus 4	101.4	NCRI 15.2; no filler; sealed	May 2025	Official leaderboard
23	Gemini 3.6 Flash	101.0	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
24	GPT-5.5	100.2	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
25	google/gemini-2.5-pro (snapshot unspecified)	99.8	NCRI 15.2; no filler; sealed		Official leaderboard
26	Muse Spark 1.3	98.3	NCRI 15.2; no filler; placed	Sep 2026	Official leaderboard
27	Muse Wonder	97.7	NCRI 15.2; no filler; placed_after_release	Oct 2026	Official leaderboard
28	Muse Spark 1.2	97.1	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
29	Muse Spark 1.1	96.8	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
30	Kimi K3	96.8	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
31	Muse Spark Open	95.7	NCRI 15.2; no filler; placed_after_release	Oct 2026	Official leaderboard
32	Claude Sonnet 5.5	95.1	NCRI 15.2; no filler; placed_after_release	Sep 2026	Official leaderboard
33	Tencent Hy4	94.0	NCRI 15.2; no filler; placed_after_release	Nov 2026	Official leaderboard
34	GPT-5.4	94.0	NCRI 15.2; no filler; sealed	Mar 2026	Official leaderboard
35	Claude Sonnet 4.5	94.0	NCRI 15.2; no filler; sealed	Sep 2025	Official leaderboard
36	Claude Sonnet 4	91.3	NCRI 15.2; no filler; sealed	May 2025	Official leaderboard
37	Claude Sonnet 5	90.8	NCRI 15.2; no filler; sealed	Jun 2026	Official leaderboard
38	GPT-5.6 Terra	89.0	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
39	Claude Sonnet 4.6	87.8	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
40	Qwen 3.6 Max (Preview)	87.0	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
41	Grok 4.6	86.8	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
42	Gemini 3.1 Flash-Lite	86.5	NCRI 15.2; no filler; sealed	Mar 2026	Official leaderboard
43	Qwen3.8-2.4T-A95B	86.0	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
44	GPT-5.2	85.8	NCRI 15.2; no filler; sealed	Dec 2025	Official leaderboard
45	Qwen3.7-Max	85.8	NCRI 15.2; no filler; sealed	May 2026	Official leaderboard
46	Qwen 3.8 Max	85.4	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
47	GPT-5.3 Codex	85.2	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
48	DeepSeek V4 Pro 0813	84.1	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
49	DeepSeek-V4-Pro	84.1	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
50	Qwen3-Max	83.3	NCRI 15.2; no filler; sealed	Sep 2025	Official leaderboard
51	Qwen3.7-Plus	83.1	NCRI 15.2; no filler; sealed	Jun 2026	Official leaderboard
52	Tencent Hy4 preview	82.8	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
53	LongCat-2.0	82.5	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
54	Qwen 3.6 Plus	82.2	NCRI 15.2; no filler; sealed	Mar 2026	Official leaderboard
55	Gemma 4 31B IT	81.7	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
56	DeepSeek V4.1 Flash 1017	81.6	NCRI 15.2; no filler; placed_after_release	Oct 2026	Official leaderboard
57	DeepSeek V4.1 Flash	81.3	NCRI 15.2; no filler; placed_after_release	Sep 2026	Official leaderboard
58	Grok 4.5	81.0	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
59	kwaipilot/kat-coder-pro-v2	80.8	NCRI 15.2; no filler; sealed		Official leaderboard
60	Gemini 3.5 Flash-Lite	80.6	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
61	qwen/qwen3.5-plus-02-15	80.6	NCRI 15.2; no filler; sealed		Official leaderboard
62	Kimi K2.5	80.6	NCRI 15.2; no filler; sealed	Jan 2026	Official leaderboard
63	Qwen3.5 397B-A17B	80.0	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
64	Kimi K2.6	79.2	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
65	local/deepseek-v4-flash-base	79.1	NCRI 15.2; no filler; sealed		Official leaderboard
66	google/gemini-2.5-flash (snapshot unspecified)	78.8	NCRI 15.2; no filler; sealed		Official leaderboard
67	nvidia/nemotron-3-ultra-550b-a55b	78.3	NCRI 15.2; no filler; sealed		Official leaderboard
68	Inkling	78.3	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
69	DeepSeek-V4-Flash	78.1	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
70	moonshotai/kimi-k2 (snapshot unspecified)	78.0	NCRI 15.2; no filler; sealed		Official leaderboard
71	DeepSeek-V3.1-Terminus	77.6	NCRI 15.2; no filler; sealed	Sep 2025	Official leaderboard
72	GLM-5.2	77.3	NCRI 15.2; no filler; sealed	Jun 2026	Official leaderboard
73	GPT-6 Luna	77.3	NCRI 15.2; no filler; placed_after_release	Sep 2026	Official leaderboard
74	Qwen3-VL-235B-A22B-Instruct	77.3	NCRI 15.2; no filler; sealed	Sep 2025	Official leaderboard
75	MiMo-V2.5-Pro	77.1	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
76	DeepSeek-V3.2-Exp	76.7	NCRI 15.2; no filler; sealed	Sep 2025	Official leaderboard
77	GPT-5.6 Luna	76.6	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
78	deepseek/deepseek-chat-v3.1	76.5	NCRI 15.2; no filler; sealed		Official leaderboard
79	GLM-5.3-Flash	76.4	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
80	GLM-5	76.2	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
81	DeepSeek V4 Flash 0731	75.5	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
82	GLM-5.1	75.2	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
83	GPT-4.1	75.1	NCRI 15.2; no filler; sealed	Apr 2025	Official leaderboard
84	DeepSeek-V3.2	75.0	NCRI 15.2; no filler; sealed	Dec 2025	Official leaderboard
85	qwen/qwen3-235b-a22b-2507	74.6	NCRI 15.2; no filler; sealed		Official leaderboard
86	GLM-5.3	74.5	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
87	Qwen3.8 Flash	74.4	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
88	deepseek/deepseek-chat-v3-0324	74.4	NCRI 15.2; no filler; sealed		Official leaderboard
89	openai/gpt-4o (snapshot unspecified)	74.2	NCRI 15.2; no filler; sealed		Official leaderboard
90	GPT-4o Aug 2024	74.2	NCRI 15.2; no filler; sealed	Aug 2024	Official leaderboard
91	Claude Haiku 4.5	74.2	NCRI 15.2; no filler; sealed	Oct 2025	Official leaderboard
92	Ling-2.6-1T	73.9	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
93	openai/gpt-4 (snapshot unspecified)	73.7	NCRI 15.2; no filler; sealed		Official leaderboard
94	WizardLM-2 8x22B	73.2	NCRI 15.2; no filler; sealed	Apr 2024	Official leaderboard
95	Qwen3-Coder-480B-A35B	73.1	NCRI 15.2; no filler; sealed	Jul 2025	Official leaderboard
96	deepseek/deepseek-chat	72.9	NCRI 15.2; no filler; sealed		Official leaderboard
97	GPT-4 Jun 2023	72.8	NCRI 15.2; no filler; sealed	Jun 2023	Official leaderboard
98	MiniMax-M3	72.6	NCRI 15.2; no filler; sealed	Jun 2026	Official leaderboard
99	Tencent Hy3	72.5	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
100	Qwen3.5-Plus	72.3	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
101	Mercury 3	72.3	NCRI 15.2; no filler; placed_after_release	Oct 2026	Official leaderboard
102	Tencent Hy3 preview	72.3	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
103	GPT-4o May 2024	72.2	NCRI 15.2; no filler; sealed	May 2024	Official leaderboard
104	GLM-4.7	71.3	NCRI 15.2; no filler; sealed	Dec 2025	Official leaderboard
105	openai/gpt-4-turbo (snapshot unspecified)	71.3	NCRI 15.2; no filler; sealed		Official leaderboard
106	GLM-4.5	71.3	NCRI 15.2; no filler; sealed	Aug 2025	Official leaderboard
107	Qwen3-Coder-Plus	71.2	NCRI 15.2; no filler; sealed	Sep 2025	Official leaderboard
108	GLM-4.6	70.8	NCRI 15.2; no filler; sealed	Sep 2025	Official leaderboard
109	GPT-5.1	70.6	NCRI 15.2; no filler; sealed	Nov 2025	Official leaderboard
110	Qwen3-235B-A22B	70.5	NCRI 15.2; no filler; sealed	Apr 2025	Official leaderboard
111	Llama 4 Maverick	70.5	NCRI 15.2; no filler; sealed	Apr 2025	Official leaderboard
112	GPT-5.4 Mini	70.5	NCRI 15.2; no filler; sealed	Mar 2026	Official leaderboard
113	GPT-4o Nov 2024	70.3	NCRI 15.2; no filler; sealed	Nov 2024	Official leaderboard
114	Inkling-Small	70.2	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
115	DiffusionGemma 26B A4B	70.2	NCRI 15.2; no filler; sealed	Jun 2026	Official leaderboard
116	MiMo-V2.5	69.9	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
117	google/gemma-4-26b-a4b-it	69.5	NCRI 15.2; no filler; sealed		Official leaderboard
118	Qwen3.5-122B-A10B	69.4	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
119	Grok 4.3	69.2	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
120	Qwen3-235B-A22B-Thinking Jul 2025	68.9	NCRI 15.2; no filler; sealed	Jul 2025	Official leaderboard
121	Qwen3-Next-80B-A3B-Instruct	68.4	NCRI 15.2; no filler; sealed	Sep 2025	Official leaderboard
122	MiniMax-M3.1-Flash	68.4	NCRI 15.2; no filler; placed_after_release	Oct 2026	Official leaderboard
123	Claude Haiku 5.5	68.3	NCRI 15.2; no filler; placed_after_release	Oct 2026	Official leaderboard
124	local/muse-glimmer-30b	67.9	NCRI 15.2; no filler; sealed		Official leaderboard
125	Mistral Medium 3.5	67.5	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
126	Qwen3.6 27B	67.4	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
127	Grok 4.20	67.1	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
128	GPT-5	67.0	NCRI 15.2; no filler; sealed	Aug 2025	Official leaderboard
129	baidu/ernie-4.5-vl-424b-a47b	66.9	NCRI 15.2; no filler; sealed		Official leaderboard
130	GLM-4.5V	66.7	NCRI 15.2; no filler; sealed	Aug 2025	Official leaderboard
131	Qwen3.5-27B	66.6	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
132	Ling-3.0-flash	66.3	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
133	meta/muse-glimmer-30b	66.3	NCRI 15.2; no filler; sealed		Official leaderboard
134	nvidia/nemotron-3-super-120b-a12b	66.2	NCRI 15.2; no filler; sealed		Official leaderboard
135	Qwen 3.8 27B	66.1	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
136	Mercury 2.5	65.8	NCRI 15.2; no filler; placed	Sep 2026	Official leaderboard
137	inclusionai/ling-2.6-flash	65.5	NCRI 15.2; no filler; sealed		Official leaderboard
138	Qwen3-Coder-Next	65.5	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
139	Solar Pro 4	65.3	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
140	Qwen2.5-VL-72B	65.2	NCRI 15.2; no filler; sealed	Jan 2025	Official leaderboard
141	GPT-4.1 mini	65.0	NCRI 15.2; no filler; sealed	Apr 2025	Official leaderboard
142	MiniMax-M2.5	65.0	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
143	reflection/beam-1.0-501b-a23b	64.8	NCRI 15.2; no filler; placed_after_release		Official leaderboard
144	nousresearch/hermes-3-llama-3.1-405b	64.6	NCRI 15.2; no filler; sealed		Official leaderboard
145	local/qwen3.5-35b-a3b-base	64.5	NCRI 15.2; no filler; sealed		Official leaderboard
146	google/gemini-2.5-flash-lite (snapshot unspecified)	64.5	NCRI 15.2; no filler; sealed		Official leaderboard
147	MiniMax-M2.7	64.0	NCRI 15.2; no filler; sealed	Mar 2026	Official leaderboard
148	Qwen3-VL-32B-Instruct	64.0	NCRI 15.2; no filler; sealed	Oct 2025	Official leaderboard
149	Qwen3-Next-80B-A3B	63.8	NCRI 15.2; no filler; sealed	Sep 2025	Official leaderboard
150	sao10k/l3.3-euryale-70b	63.8	NCRI 15.2; no filler; sealed		Official leaderboard
151	mistralai/mistral-small-2603	63.5	NCRI 15.2; no filler; sealed		Official leaderboard
152	Qwen2.5-72B	63.5	NCRI 15.2; no filler; sealed	Sep 2024	Official leaderboard
153	GLM-4.6V	63.4	NCRI 15.2; no filler; sealed	Dec 2025	Official leaderboard
154	Qwen 3.6 Flash	63.3	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
155	Qwen3.7 Flash	63.3	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
156	Hermes 4 405B	62.8	NCRI 15.2; no filler; sealed	Aug 2025	Official leaderboard
157	meta-llama/llama-3.1-70b-instruct	62.8	NCRI 15.2; no filler; sealed		Official leaderboard
158	local/qwen2.5-32b-instruct	62.7	NCRI 15.2; no filler; sealed		Official leaderboard
159	Llama 3.3 70B Instruct	62.6	NCRI 15.2; no filler; sealed	Dec 2024	Official leaderboard
160	Laguna S 2.1	62.5	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
161	GLM-4.5-Air	62.4	NCRI 15.2; no filler; sealed	Jul 2025	Official leaderboard
162	qwen/qwen3-30b-a3b-instruct-2507	62.1	NCRI 15.2; no filler; sealed		Official leaderboard
163	Qwen 3.6 35B-A3B	61.6	NCRI 15.2; no filler; sealed	Apr 2026	Official leaderboard
164	qwen/qwen3.5-flash-02-23	61.4	NCRI 15.2; no filler; sealed		Official leaderboard
165	amazon/nova-pro-v1	61.3	NCRI 15.2; no filler; sealed		Official leaderboard
166	Qwen2.5-32B	61.1	NCRI 15.2; no filler; sealed	Sep 2024	Official leaderboard
167	sao10k/l3.1-euryale-70b	61.0	NCRI 15.2; no filler; sealed		Official leaderboard
168	Qwen2-72B	61.0	NCRI 15.2; no filler; sealed	Jun 2024	Official leaderboard
169	Qwen3.5-35B-A3B	60.1	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
170	Qwen2.5-Coder-32B-Instruct	60.0	NCRI 15.2; no filler; sealed	Nov 2024	Official leaderboard
171	Qwen3-VL-30B-A3B-Instruct	59.9	NCRI 15.2; no filler; sealed	Oct 2025	Official leaderboard
172	Gemma 4 12B	59.7	NCRI 15.2; no filler; sealed	Jun 2026	Official leaderboard
173	Qwen3-Coder-30B-A3B-Instruct	59.6	NCRI 15.2; no filler; sealed	Jul 2025	Official leaderboard
174	qwen/qwen3-coder-flash	59.5	NCRI 15.2; no filler; sealed		Official leaderboard
175	Ripple-65B Preview	59.3	NCRI 15.2; no filler; placed_after_release	Nov 2026	Official leaderboard
176	Qwen3-32B	59.2	NCRI 15.2; no filler; sealed	Apr 2025	Official leaderboard
177	thedrummer/cydonia-24b-v4.1	59.1	NCRI 15.2; no filler; sealed		Official leaderboard
178	gpt-oss-120b	59.0	NCRI 15.2; no filler; sealed	Aug 2025	Official leaderboard
179	mistralai/ministral-14b-2512	58.7	NCRI 15.2; no filler; sealed		Official leaderboard
180	Phi-4	58.3	NCRI 15.2; no filler; sealed	Dec 2024	Official leaderboard
181	nousresearch/hermes-4-70b	57.9	NCRI 15.2; no filler; sealed		Official leaderboard
182	Llama 4 Scout	57.7	NCRI 15.2; no filler; sealed	Apr 2025	Official leaderboard
183	Llama 3.1-70B	57.6	NCRI 15.2; no filler; sealed	Jul 2024	Official leaderboard
184	Laguna XS 2.1	57.5	NCRI 15.2; no filler; sealed	Jul 2026	Official leaderboard
185	local/gemma-3-27b-pt	57.5	NCRI 15.2; no filler; sealed		Official leaderboard
186	Claude 3 Haiku	57.3	NCRI 15.2; no filler; sealed	Mar 2024	Official leaderboard
187	local/mistral-small-24b-base-2501	56.2	NCRI 15.2; no filler; sealed		Official leaderboard
188	Nemotron 3.5 Lightning 30B-A3B	56.1	NCRI 15.2; no filler; sealed	Aug 2026	Official leaderboard
189	Qwen3-VL-8B-Instruct	56.0	NCRI 15.2; no filler; sealed	Oct 2025	Official leaderboard
190	Gemma 3 27B	56.0	NCRI 15.2; no filler; sealed	Mar 2025	Official leaderboard
191	glm-4.7-flash	55.9	NCRI 15.2; no filler; sealed	Jan 2026	Official leaderboard
192	local/gemma-3-27b-it-hf	55.8	NCRI 15.2; no filler; sealed		Official leaderboard
193	GPT-4o mini	55.7	NCRI 15.2; no filler; sealed	Jul 2024	Official leaderboard
194	mistralai/mistral-small-24b-instruct-2501	55.4	NCRI 15.2; no filler; sealed		Official leaderboard
195	openai/gpt-3.5-turbo (snapshot unspecified)	55.2	NCRI 15.2; no filler; sealed		Official leaderboard
196	Mercury 2	55.1	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
197	Qwen3-14B	55.0	NCRI 15.2; no filler; sealed	Apr 2025	Official leaderboard
198	gpt-oss-20b	54.8	NCRI 15.2; no filler; sealed	Aug 2025	Official leaderboard
199	Olmo 3.1 32B Instruct	54.3	NCRI 15.2; no filler; placed	Dec 2025	Official leaderboard
200	local/qwen3.5-9b-base	54.2	NCRI 15.2; no filler; sealed		Official leaderboard
201	local/olmo-3-32b-dpo	54.0	NCRI 15.2; no filler; sealed		Official leaderboard
202	mistralai/mistral-small-3.2-24b-instruct	53.8	NCRI 15.2; no filler; sealed		Official leaderboard
203	local/olmo-3-32b-rl	53.8	NCRI 15.2; no filler; sealed		Official leaderboard
204	local/olmo-3-32b-mid	53.3	NCRI 15.2; no filler; sealed		Official leaderboard
205	Gemma 2 27B	53.3	NCRI 15.2; no filler; sealed	Jun 2024	Official leaderboard
206	local/olmo-3-32b-sft	52.8	NCRI 15.2; no filler; sealed		Official leaderboard
207	google/gemma-2-27b-it	52.6	NCRI 15.2; no filler; sealed		Official leaderboard
208	local/nanbeige42-r2	52.6	NCRI 15.2; no filler; placed_after_release		Official leaderboard
209	local/qwen1.5-110b-chat	52.6	NCRI 15.2; no filler; sealed		Official leaderboard
210	Qwen3.5-9B	52.4	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
211	local/ouro26-r4	51.7	NCRI 15.2; no filler; placed_after_release		Official leaderboard
212	Qwen3-8B	51.4	NCRI 15.2; no filler; sealed	Apr 2025	Official leaderboard
213	mistralai/ministral-8b-2512	51.0	NCRI 15.2; no filler; sealed		Official leaderboard
214	GPT-5 mini	50.7	NCRI 15.2; no filler; sealed	Aug 2025	Official leaderboard
215	Gemma 4 E4B	50.6	NCRI 15.2; no filler; sealed	Mar 2026	Official leaderboard
216	Qwen3-30B-A3B	50.4	NCRI 15.2; no filler; sealed	Apr 2025	Official leaderboard
217	GPT-4.1 nano	50.2	NCRI 15.2; no filler; sealed	Apr 2025	Official leaderboard
218	local/gemma-3-12b-pt	49.9	NCRI 15.2; no filler; sealed		Official leaderboard
219	Qwen1.5-72B	49.7	NCRI 15.2; no filler; sealed	Feb 2024	Official leaderboard
220	GPT-3.5 Turbo Instruct	49.6	NCRI 15.2; no filler; sealed	Sep 2023	Official leaderboard
221	GPT-5.4 Nano	49.4	NCRI 15.2; no filler; sealed	Mar 2026	Official leaderboard
222	local/gemma-3-12b-it	49.3	NCRI 15.2; no filler; sealed		Official leaderboard
223	Hunyuan-A13B-Instruct	48.7	NCRI 15.2; no filler; sealed	Jun 2025	Official leaderboard
224	qwen/qwen-2.5-7b-instruct	48.4	NCRI 15.2; no filler; sealed		Official leaderboard
225	local/gemma-2-9b-it	47.5	NCRI 15.2; no filler; sealed		Official leaderboard
226	Nemotron 3 Nano 30B-A3B	47.3	NCRI 15.2; no filler; sealed	Dec 2025	Official leaderboard
227	local/ouro14-r4	47.3	NCRI 15.2; no filler; placed_after_release		Official leaderboard
228	google/gemma-3-12b-it	46.8	NCRI 15.2; no filler; sealed		Official leaderboard
229	mistralai/ministral-3b-2512	46.2	NCRI 15.2; no filler; sealed		Official leaderboard
230	local/qwen3.5-4b-base	46.0	NCRI 15.2; no filler; sealed		Official leaderboard
231	local/olmo-3-32b-pretrain	45.0	NCRI 15.2; no filler; sealed		Official leaderboard
232	Gemma 2 9B	44.7	NCRI 15.2; no filler; sealed	Jun 2024	Official leaderboard
233	Qwen3.5-4B	44.3	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
234	Gemma 4 E2B	44.0	NCRI 15.2; no filler; sealed	Mar 2026	Official leaderboard
235	UI-TARS-1.5	43.9	NCRI 15.2; no filler; sealed	Apr 2025	Official leaderboard
236	Mistral-Nemo-Base-2407	43.2	NCRI 15.2; no filler; sealed	Jul 2024	Official leaderboard
237	thedrummer/unslopnemo-12b	42.8	NCRI 15.2; no filler; sealed		Official leaderboard
238	Mistral NeMo	42.5	NCRI 15.2; no filler; sealed	Jul 2024	Official leaderboard
239	local/llama-3.1-8b-instruct-fp8	41.8	NCRI 15.2; no filler; sealed		Official leaderboard
240	meta-llama/llama-3.1-8b-instruct	41.6	NCRI 15.2; no filler; sealed		Official leaderboard
241	local/llama-3.1-8b-instruct	40.9	NCRI 15.2; no filler; sealed		Official leaderboard
242	local/olmo3-32b-step260000	40.8	NCRI 15.2; no filler; placed		Official leaderboard
243	local/llama-2-70b-hf	40.7	NCRI 15.2; no filler; sealed		Official leaderboard
244	sao10k/l3-lunaris-8b	40.7	NCRI 15.2; no filler; sealed		Official leaderboard
245	Llama 3.1-8B	40.6	NCRI 15.2; no filler; sealed	Jul 2024	Official leaderboard
246	local/llama-3-8b-instruct	39.2	NCRI 15.2; no filler; sealed		Official leaderboard
247	Gemma 3n E4B	38.5	NCRI 15.2; no filler; sealed	Jun 2025	Official leaderboard
248	local/olmo3-32b-step360000	38.5	NCRI 15.2; no filler; placed		Official leaderboard
249	google/gemma-3-4b-it	37.9	NCRI 15.2; no filler; sealed		Official leaderboard
250	LLaMA-65B	37.5	NCRI 15.2; no filler; sealed	Feb 2023	Official leaderboard
251	aion-labs/aion-rp-llama-3.1-8b	37.2	NCRI 15.2; no filler; sealed		Official leaderboard
252	ibm-granite/granite-4.0-h-micro	37.2	NCRI 15.2; no filler; sealed		Official leaderboard
253	local/olmo3-32b-step130000	36.2	NCRI 15.2; no filler; placed		Official leaderboard
254	Qwen3.5-2B	35.5	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
255	local/gemma-3-4b-it	35.3	NCRI 15.2; no filler; sealed		Official leaderboard
256	local/olmo3-32b-step92000	34.5	NCRI 15.2; no filler; placed		Official leaderboard
257	local/olmo3-32b-step182000	34.4	NCRI 15.2; no filler; sealed		Official leaderboard
258	davinci-002	32.6	NCRI 15.2; no filler; sealed	Mar 2022	Official leaderboard
259	local/olmo3-7b-step1413814	32.0	NCRI 15.2; no filler; sealed		Official leaderboard
260	local/olmo3-32b-step65000	31.4	NCRI 15.2; no filler; placed		Official leaderboard
261	GPT-5 nano	31.0	NCRI 15.2; no filler; sealed	Aug 2025	Official leaderboard
262	local/olmo3-32b-step46000	29.5	NCRI 15.2; no filler; sealed		Official leaderboard
263	local/olmo3-32b-step33000	27.4	NCRI 15.2; no filler; placed		Official leaderboard
264	local/gemma-7b-it	25.8	NCRI 15.2; no filler; sealed		Official leaderboard
265	undi95/remm-slerp-l2-13b	25.7	NCRI 15.2; no filler; sealed		Official leaderboard
266	Qwen3.5-0.8B	24.9	NCRI 15.2; no filler; sealed	Feb 2026	Official leaderboard
267	local/olmo3-7b-step360000	24.6	NCRI 15.2; no filler; sealed		Official leaderboard
268	local/gemma-2-2b	24.3	NCRI 15.2; no filler; sealed		Official leaderboard
269	Gemma 2 2B IT	24.1	NCRI 15.2; no filler; sealed	Jul 2024	Official leaderboard
270	local/talkie-1930-13b-it	23.9	NCRI 15.2; no filler; sealed		Official leaderboard
271	gryphe/mythomax-l2-13b	23.8	NCRI 15.2; no filler; sealed		Official leaderboard
272	local/olmo3-7b-step714000	23.2	NCRI 15.2; no filler; sealed		Official leaderboard
273	local/olmo3-7b-step182000	22.7	NCRI 15.2; no filler; sealed		Official leaderboard
274	local/olmo3-32b-step16000	21.5	NCRI 15.2; no filler; placed		Official leaderboard
275	local/olmo3-32b-step23000	20.4	NCRI 15.2; no filler; placed		Official leaderboard
276	local/olmo3-7b-step92000	18.9	NCRI 15.2; no filler; sealed		Official leaderboard
277	local/llama-2-7b-hf	18.7	NCRI 15.2; no filler; sealed		Official leaderboard
278	local/huginn-r12	16.8	NCRI 15.2; no filler; placed_after_release		Official leaderboard
279	local/gemma-3-1b-it	16.1	NCRI 15.2; no filler; sealed		Official leaderboard
280	local/olmo3-32b-step12000	15.6	NCRI 15.2; no filler; placed		Official leaderboard
281	LLaMA-7B	15.0	NCRI 15.2; no filler; sealed	Feb 2023	Official leaderboard
282	local/gemma-2b-it	11.9	NCRI 15.2; no filler; sealed		Official leaderboard
283	local/talkie-web-13b-base	10.6	NCRI 15.2; no filler; sealed		Official leaderboard
284	openai/babbage-002	6.6	NCRI 15.2; no filler; sealed		Official leaderboard
285	local/olmo3-7b-step46000	4.1	NCRI 15.2; no filler; sealed		Official leaderboard
286	local/olmo3-7b-step23000	3.1	NCRI 15.2; no filler; sealed		Official leaderboard
287	local/olmo3-7b-step12000	-8.3	NCRI 15.2; no filler; sealed		Official leaderboard
288	local/pythia-12b	-8.6	NCRI 15.2; no filler; sealed		Official leaderboard
289	local/olmo3-32b-step6000	-9.9	NCRI 15.2; no filler; placed		Official leaderboard
290	local/olmo3-7b-step6000	-11.5	NCRI 15.2; no filler; sealed		Official leaderboard
291	local/talkie-1930-13b-base	-12.4	NCRI 15.2; no filler; sealed		Official leaderboard
292	local/gemma-3-270m-it	-23.1	NCRI 15.2; no filler; sealed		Official leaderboard

Labs report scores for 1 model in their cards. We plot every source in the charts below.

2 models have held number one on the official leaderboard since its release in September 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic OpenAI

Own leaderboard 2 leaders

GPT-6 Astra 159.0

Source: Official nocot-bench (NCRI) leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card

Sources: Official nocot-bench (NCRI) leaderboard and labs’ model cards

Labs report it only in system cards

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark. Some system cards do, below.

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

Date	Document	Score	As the document writes it	Section
Oct 13, 2026	Gemini 4 Argon system card	150.3	No Chain of Thought Reasoning Index (NCRI, Nanda 2026), 150.3 without filler tokens (152.1 with filler tokens); about 5.5 serial arithmetic steps at 50% (3.4.5.3, Figures 3.23 and 3.24)	Monitorability

One lab claim next to the leaderboard

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. Every benchmark.

Model	Lab says	Board says	Gap
Gemini 4 Argon	150.3 No Chain of Thought Reasoning Index (NCRI, Nanda 2026), 150.3 without filler tokens (152.1 with filler tokens); about 5.5 serial arithmetic steps at 50% (3.4.5.3, Figures 3.23 and 3.24)	151.2 Leaderboard, NCRI 15.2; no filler; placed_after_release	−0.9