nocot-bench (NCRI)
The official leaderboard hasn’t scored a model released after Tencent Hy4 (November 2026). Labs report their own numbers for one newer model: GPT-6.2 Astra at 171.0.
Reasoning problems answered without chain of thought (e.g., Brew multi-hop tasks). Measures forward-pass reasoning relevant to monitorability.
- Made by
- Neel Nanda et al.
- Released
- September 2026
- Kind
- Deception and alignment
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- All public
- Measures
- Capability
- Risk
- Loss of control
- Language
- English
- Interaction
- Single turn
- Items from
- Another benchmark
- Refresh
- New versions replace it
- Run by
- Anyone
- In views
- Loss of control
- Strongest model tested
- GPT-6.2 Astra #1 on the capability index
- Last in a system card
- November 2026
- Last model added
- November 2026
- Scores refreshed
- November 20th, 2026
GPT-6 Astra leads with 173.9 GPT-6 Astra leads with 173.9
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
GPT-6 Astra leads with 173.9, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
- 173.9 GPT-6 Astra
- 156.6 GPT-6.1 Sol
- 154.1
Gemini 4 Argon
- 149.6 GPT-6.1 Terra
- 129.7
Gemini 3.8 Flash
- 129.6 Claude Fable 5.1
- 127.6
Gemini 3.7 Flash
- 123.9
Gemini 3.1 Pro
- 122.2 GPT-6 Sol
- 121.5 Claude Fable 5
- 119.5 GPT-5.6 Sol
- 118.9 Claude Opus 5.5
- 115.8 Claude Opus 5
- 111.0 GPT-5.5
- 110.9
Gemini 3.6 Flash
- 110.8 Claude Opus 4.6
- 110.5
Gemini 3 Flash
- 110.5 Claude Opus 4.5
- 109.0 Claude Opus 4
- 108.8
Muse Spark 1.1
- 107.5 Claude Opus 4.1
- 107.5
Gemini 3.5 Flash
- 105.6 Kimi K3
- 103.9 Claude Opus 4.7
- 102.9 google/gemini-2.5-pro (snapshot unspecified)
- 102.8 Kimi K3.1 Pro
- 102.3
Muse Spark 1.2
- 102.2 Claude Opus 4.8
- 100.8
Muse Spark 1.3
- 100.8 Claude Sonnet 4.5
- 97.7
Muse Wonder
- 96.5 Claude Sonnet 4
- 95.7
Muse Spark Open
- 95.1 GPT-5.6 Terra
- 95.1 Claude Sonnet 5.5
- 94.1 GPT-5.4
- 94.0
Tencent Hy4
- 91.1 Claude Sonnet 5
- 90.0 Claude Sonnet 4.6
- 87.0
Qwen 3.6 Max (Preview)
- 86.8 Grok 4.6
- 86.5
Gemini 3.1 Flash-Lite
- 86.0
Qwen3.8-2.4T-A95B
- 85.8 GPT-5.2
- 85.8
Qwen3.7-Max
- 85.4
Qwen 3.8 Max
- 85.2 GPT-5.3 Codex
- 84.1
DeepSeek V4 Pro 0813
- 84.1
DeepSeek-V4-Pro
- 83.3
Qwen3-Max
- 83.1
Qwen3.7-Plus
- 82.8
Tencent Hy4 preview
- 82.5
LongCat-2.0
- 82.2
Qwen 3.6 Plus
- 81.7
Gemma 4 31B IT
- 81.6
DeepSeek V4.1 Flash 1017
- 81.3
DeepSeek V4.1 Flash
- 81.0 Grok 4.5
- 80.8 kwaipilot/kat-coder-pro-v2
- 80.6
Gemini 3.5 Flash-Lite
- 80.6
Qwen3.5-Plus
- 80.6 Kimi K2.5
- 80.0
Qwen3.5 397B-A17B
- 79.2 Kimi K2.6
- 79.1
DeepSeek-V4-Flash-Base
- 78.8 google/gemini-2.5-flash (snapshot unspecified)
- 78.3 nvidia/nemotron-3-ultra-550b-a55b
- 78.3
Inkling
- 78.1
DeepSeek-V4-Flash
- 78.0 moonshotai/kimi-k2 (snapshot unspecified)
- 77.6
DeepSeek-V3.1-Terminus
- 77.3 GLM-5.2
- 77.3 GPT-6 Luna
- 77.3
Qwen3-VL-235B-A22B-Instruct
- 77.1 MiMo-V2.5-Pro
- 76.7
DeepSeek-V3.2-Exp
- 76.6 GPT-5.6 Luna
- 76.5 deepseek/deepseek-chat-v3.1
- 76.4 GLM-5.3-Flash
- 76.2 GLM-5
- 75.5
DeepSeek V4 Flash 0731
- 75.2 GLM-5.1
- 75.1 GPT-4.1
- 75.0
DeepSeek-V3.2
- 74.6
Qwen3-235B-A22B-Instruct
- 74.5 GLM-5.3
- 74.4
Qwen3.8 Flash
- 74.4 deepseek/deepseek-chat-v3-0324
- 74.2 openai/gpt-4o (snapshot unspecified)
- 74.2 GPT-4o
- 74.2 Claude Haiku 4.5
- 73.9
Ling-2.6-1T
- 73.7 openai/gpt-4 (snapshot unspecified)
- 73.2
WizardLM-2 8x22B
- 73.1
Qwen3-Coder-480B-A35B
- 72.9 deepseek/deepseek-chat
- 72.8 GPT-4
- 72.6
MiniMax-M3
- 72.5
Tencent Hy3
- 72.3 Mercury 3
- 72.3
Tencent Hy3 preview
- 72.2 GPT-4o
- 71.3 GLM-4.7
- 71.3 openai/gpt-4-turbo (snapshot unspecified)
- 71.3 GLM-4.5
- 71.2 Qwen3-Coder-Plus
- 70.8 GLM-4.6
- 70.6 GPT-5.1
- 70.5
Qwen3-235B-A22B
- 70.5
Llama 4 Maverick
- 70.5 GPT-5.4 Mini
- 70.3 GPT-4o
- 70.2
Inkling-Small
- 70.2
DiffusionGemma 26B A4B
- 69.9 MiMo-V2.5
- 69.5
Gemma 4 26B A4B
- 69.4
Qwen3.5-122B-A10B
- 69.2 Grok 4.3
- 68.9
Qwen3-235B-A22B-Thinking
- 68.4
Qwen3-Next-80B-A3B-Instruct
- 68.4
MiniMax-M3.1-Flash
- 68.3 Claude Haiku 5.5
- 67.9
Muse Glimmer
- 67.5
Mistral Medium 3.5
- 67.4
Qwen3.6 27B
- 67.1 Grok 4.20
- 67.0 GPT-5
- 66.9 baidu/ernie-4.5-vl-424b-a47b
- 66.7 GLM-4.5V
- 66.6
Qwen3.5-27B
- 66.3
Ling-3.0-flash
- 66.2 nvidia/nemotron-3-super-120b-a12b
- 66.1
Qwen 3.8 27B
- 65.8 Mercury 2.5
- 65.5 inclusionai/ling-2.6-flash
- 65.5
Qwen3-Coder-Next
- 65.3
Solar Pro 4
- 65.2
Qwen2.5-VL-72B
- 65.0 GPT-4.1 mini
- 65.0
MiniMax-M2.5
- 64.8 reflection/beam-1.0-501b-a23b
- 64.6 nousresearch/hermes-3-llama-3.1-405b
- 64.5
Qwen3.5-35B-A3B-Base
- 64.5 google/gemini-2.5-flash-lite (snapshot unspecified)
- 64.0
MiniMax-M2.7
- 64.0
Qwen3-VL-32B-Instruct
- 63.8
Qwen3-Next-80B-A3B
- 63.8 sao10k/l3.3-euryale-70b
- 63.5 mistralai/mistral-small-2603
- 63.5
Qwen2.5-72B
- 63.4 GLM-4.6V
- 63.3
Qwen 3.6 Flash
- 63.3
Qwen3.7 Flash
- 62.8 Hermes 4 405B
- 62.8 meta-llama/llama-3.1-70b-instruct
- 62.7 local/qwen2.5-32b-instruct
- 62.6
Llama 3.3 70B Instruct
- 62.5
Laguna S 2.1
- 62.4 GLM-4.5-Air
- 62.1
Qwen3-30B-A3B-Instruct
- 61.6
Qwen 3.6 35B-A3B
- 61.4
Qwen3.5-Flash
- 61.3 amazon/nova-pro-v1
- 61.1
Qwen2.5-32B
- 61.0 sao10k/l3.1-euryale-70b
- 61.0
Qwen2-72B
- 60.1
Qwen3.5-35B-A3B
- 60.0
Qwen2.5-Coder-32B-Instruct
- 59.9
Qwen3-VL-30B-A3B-Instruct
- 59.7
Gemma 4 12B
- 59.6
Qwen3-Coder-30B-A3B-Instruct
- 59.5
Qwen3-Coder-Flash
- 59.3 Ripple-65B Preview
- 59.2
Qwen3-32B
- 59.1 thedrummer/cydonia-24b-v4.1
- 59.0 gpt-oss-120b
- 58.7 mistralai/ministral-14b-2512
- 58.3
Phi-4
- 57.9 nousresearch/hermes-4-70b
- 57.7
Llama 4 Scout
- 57.6
Llama 3.1-70B
- 57.5
Laguna XS 2.1
- 57.5
Gemma 3 27B PT
- 57.3 Claude 3 Haiku
- 56.2 local/mistral-small-24b-base-2501
- 56.1
Nemotron 3.5 Lightning 30B-A3B
- 56.0
Qwen3-VL-8B-Instruct
- 56.0
Gemma 3 27B
- 55.9 glm-4.7-flash
- 55.7 GPT-4o mini
- 55.4 mistralai/mistral-small-24b-instruct-2501
- 55.2 openai/gpt-3.5-turbo (snapshot unspecified)
- 55.1 Mercury 2
- 55.0
Qwen3-14B
- 54.8 gpt-oss-20b
- 54.3
Olmo 3.1 32B Instruct
- 54.2
Qwen3.5-9B-Base
- 54.0 local/olmo-3-32b-dpo
- 53.8 mistralai/mistral-small-3.2-24b-instruct
- 53.8 local/olmo-3-32b-rl
- 53.3 local/olmo-3-32b-mid
- 53.3
Gemma 2 27B
- 52.8 local/olmo-3-32b-sft
- 52.6 google/gemma-2-27b-it
- 52.6 local/nanbeige42-r2
- 52.6 local/qwen1.5-110b-chat
- 52.4
Qwen3.5-9B
- 51.7 local/ouro26-r4
- 51.4
Qwen3-8B
- 51.0 mistralai/ministral-8b-2512
- 50.7 GPT-5 mini
- 50.6
Gemma 4 E4B
- 50.4
Qwen3-30B-A3B
- 50.2 GPT-4.1 nano
- 49.9
Gemma 3 12B PT
- 49.7
Qwen1.5-72B
- 49.6 GPT-3.5 Turbo Instruct
- 49.4 GPT-5.4 Nano
- 49.3
Gemma 3 12B
- 48.7
Hunyuan-A13B-Instruct
- 48.4 qwen/qwen-2.5-7b-instruct
- 47.5 local/gemma-2-9b-it
- 47.3
Nemotron 3 Nano 30B-A3B
- 47.3 local/ouro14-r4
- 46.2 mistralai/ministral-3b-2512
- 46.0
Qwen3.5-4B-Base
- 45.0 local/olmo-3-32b-pretrain
- 44.7
Gemma 2 9B
- 44.3
Qwen3.5-4B
- 44.0
Gemma 4 E2B
- 43.9
UI-TARS-1.5
- 43.2
Mistral-Nemo-Base-2407
- 42.8 thedrummer/unslopnemo-12b
- 42.5
Mistral NeMo
- 41.8 local/llama-3.1-8b-instruct-fp8
- 41.6 meta-llama/llama-3.1-8b-instruct
- 40.9 local/llama-3.1-8b-instruct
- 40.8 local/olmo3-32b-step260000
- 40.7 local/llama-2-70b-hf
- 40.7 sao10k/l3-lunaris-8b
- 40.6
Llama 3.1-8B
- 39.2 local/llama-3-8b-instruct
- 38.5
Gemma 3n E4B
- 38.5 local/olmo3-32b-step360000
- 37.9
Gemma 3 4B
- 37.5
LLaMA-65B
- 37.2 aion-labs/aion-rp-llama-3.1-8b
- 37.2 ibm-granite/granite-4.0-h-micro
- 36.2 local/olmo3-32b-step130000
- 35.5
Qwen3.5-2B
- 34.5 local/olmo3-32b-step92000
- 34.4 local/olmo3-32b-step182000
- 32.6 davinci-002
- 32.0 local/olmo3-7b-step1413814
- 31.4 local/olmo3-32b-step65000
- 31.0 GPT-5 nano
- 29.5 local/olmo3-32b-step46000
- 27.4 local/olmo3-32b-step33000
- 25.8 local/gemma-7b-it
- 25.7 undi95/remm-slerp-l2-13b
- 24.9
Qwen3.5-0.8B
- 24.6 local/olmo3-7b-step360000
- 24.3 local/gemma-2-2b
- 24.1
Gemma 2 2B IT
- 23.9 local/talkie-1930-13b-it
- 23.8 gryphe/mythomax-l2-13b
- 23.2 local/olmo3-7b-step714000
- 22.7 local/olmo3-7b-step182000
- 21.5 local/olmo3-32b-step16000
- 20.4 local/olmo3-32b-step23000
- 18.9 local/olmo3-7b-step92000
- 18.7 local/llama-2-7b-hf
- 16.8 local/huginn-r12
- 16.1
Gemma 3 1B
- 15.6 local/olmo3-32b-step12000
- 15.0
LLaMA-7B
- 11.9 local/gemma-2b-it
- 10.6 local/talkie-web-13b-base
- 6.6 openai/babbage-002
- 4.1 local/olmo3-7b-step46000
- 3.1 local/olmo3-7b-step23000
- -8.3 local/olmo3-7b-step12000
- -8.6 local/pythia-12b
- -9.9 local/olmo3-32b-step6000
- -11.5 local/olmo3-7b-step6000
- -12.4 local/talkie-1930-13b-base
- -23.1
Gemma 3 270M
- 1 GPT-6 Astra NCRI 15.2; dots@2048; placed 173.9
- 2 GPT-6.1 Sol NCRI 15.2; dots@128; placed_after_release 156.6
- 3
Gemini 4 Argon NCRI 15.2; qrep@512; placed_after_release 154.1
- 4 GPT-6.1 Terra NCRI 15.2; qrep@512; placed_after_release 149.6
- 5
Gemini 3.8 Flash NCRI 15.2; dots@128; placed 129.7
- 6 Claude Fable 5.1 NCRI 15.2; dots@128; placed 129.6
- 7
Gemini 3.7 Flash NCRI 15.2; dots@32; sealed 127.6
- 8
Gemini 3.1 Pro NCRI 15.2; count@512; sealed 123.9
- 9 GPT-6 Sol NCRI 15.2; dots@128; placed_after_release 122.2
- 10 Claude Fable 5 NCRI 15.2; dots@32; sealed 121.5
- 11 GPT-5.6 Sol NCRI 15.2; qrep@512; sealed 119.5
- 12 Claude Opus 5.5 NCRI 15.2; dots@512; placed_after_release 118.9
- 13 Claude Opus 5 NCRI 15.2; dots@128; sealed 115.8
- 14 GPT-5.5 NCRI 15.2; qrep@512; sealed 111.0
- 15
Gemini 3.6 Flash NCRI 15.2; count@512; sealed 110.9
- 16 Claude Opus 4.6 NCRI 15.2; qrep@512; sealed 110.8
- 17
Gemini 3 Flash NCRI 15.2; count@512; sealed 110.5
- 18 Claude Opus 4.5 NCRI 15.2; dots@512; sealed 110.5
- 19 Claude Opus 4 NCRI 15.2; dots@512; sealed 109.0
- 20
Muse Spark 1.1 NCRI 15.2; qrep@512; sealed 108.8
- 21 Claude Opus 4.1 NCRI 15.2; dots@128; sealed 107.5
- 22
Gemini 3.5 Flash NCRI 15.2; dots@512; sealed 107.5
- 23 Kimi K3 NCRI 15.2; qrep@512; sealed 105.6
- 24 Claude Opus 4.7 NCRI 15.2; dots@128; sealed 103.9
- 25 google/gemini-2.5-pro (snapshot unspecified) NCRI 15.2; dots@512; sealed 102.9
- 26 Kimi K3.1 Pro NCRI 15.2; no filler; placed_after_release 102.8
- 27
Muse Spark 1.2 NCRI 15.2; dots@128; sealed 102.3
- 28 Claude Opus 4.8 NCRI 15.2; dots@128; sealed 102.2
- 29
Muse Spark 1.3 NCRI 15.2; dots@2048; placed 100.8
- 30 Claude Sonnet 4.5 NCRI 15.2; qrep@512; sealed 100.8
- 31
Muse Wonder NCRI 15.2; no filler; placed_after_release 97.7
- 32 Claude Sonnet 4 NCRI 15.2; count@512; sealed 96.5
- 33
Muse Spark Open NCRI 15.2; no filler; placed_after_release 95.7
- 34 GPT-5.6 Terra NCRI 15.2; qrep@512; sealed 95.1
- 35 Claude Sonnet 5.5 NCRI 15.2; no filler; placed_after_release 95.1
- 36 GPT-5.4 NCRI 15.2; dots@128; sealed 94.1
- 37
Tencent Hy4 NCRI 15.2; no filler; placed_after_release 94.0
- 38 Claude Sonnet 5 NCRI 15.2; dots@128; sealed 91.1
- 39 Claude Sonnet 4.6 NCRI 15.2; dots@512; sealed 90.0
- 40
Qwen 3.6 Max (Preview) NCRI 15.2; no filler; sealed 87.0
- 41 Grok 4.6 NCRI 15.2; no filler; sealed 86.8
- 42
Gemini 3.1 Flash-Lite NCRI 15.2; no filler; sealed 86.5
- 43
Qwen3.8-2.4T-A95B NCRI 15.2; no filler; sealed 86.0
- 44 GPT-5.2 NCRI 15.2; no filler; sealed 85.8
- 45
Qwen3.7-Max NCRI 15.2; no filler; sealed 85.8
- 46
Qwen 3.8 Max NCRI 15.2; no filler; sealed 85.4
- 47 GPT-5.3 Codex NCRI 15.2; no filler; sealed 85.2
- 48
DeepSeek V4 Pro 0813 NCRI 15.2; no filler; sealed 84.1
- 49
DeepSeek-V4-Pro NCRI 15.2; no filler; sealed 84.1
- 50
Qwen3-Max NCRI 15.2; no filler; sealed 83.3
- 51
Qwen3.7-Plus NCRI 15.2; no filler; sealed 83.1
- 52
Tencent Hy4 preview NCRI 15.2; no filler; sealed 82.8
- 53
LongCat-2.0 NCRI 15.2; no filler; sealed 82.5
- 54
Qwen 3.6 Plus NCRI 15.2; no filler; sealed 82.2
- 55
Gemma 4 31B IT NCRI 15.2; no filler; sealed 81.7
- 56
DeepSeek V4.1 Flash 1017 NCRI 15.2; no filler; placed_after_release 81.6
- 57
DeepSeek V4.1 Flash NCRI 15.2; no filler; placed_after_release 81.3
- 58 Grok 4.5 NCRI 15.2; no filler; sealed 81.0
- 59 kwaipilot/kat-coder-pro-v2 NCRI 15.2; no filler; sealed 80.8
- 60
Gemini 3.5 Flash-Lite NCRI 15.2; no filler; sealed 80.6
- 61
Qwen3.5-Plus NCRI 15.2; no filler; sealed 80.6
- 62 Kimi K2.5 NCRI 15.2; no filler; sealed 80.6
- 63
Qwen3.5 397B-A17B NCRI 15.2; no filler; sealed 80.0
- 64 Kimi K2.6 NCRI 15.2; no filler; sealed 79.2
- 65
DeepSeek-V4-Flash-Base NCRI 15.2; no filler; sealed 79.1
- 66 google/gemini-2.5-flash (snapshot unspecified) NCRI 15.2; no filler; sealed 78.8
- 67 nvidia/nemotron-3-ultra-550b-a55b NCRI 15.2; no filler; sealed 78.3
- 68
Inkling NCRI 15.2; no filler; sealed 78.3
- 69
DeepSeek-V4-Flash NCRI 15.2; no filler; sealed 78.1
- 70 moonshotai/kimi-k2 (snapshot unspecified) NCRI 15.2; no filler; sealed 78.0
- 71
DeepSeek-V3.1-Terminus NCRI 15.2; no filler; sealed 77.6
- 72 GLM-5.2 NCRI 15.2; no filler; sealed 77.3
- 73 GPT-6 Luna NCRI 15.2; no filler; placed_after_release 77.3
- 74
Qwen3-VL-235B-A22B-Instruct NCRI 15.2; no filler; sealed 77.3
- 75 MiMo-V2.5-Pro NCRI 15.2; no filler; sealed 77.1
- 76
DeepSeek-V3.2-Exp NCRI 15.2; no filler; sealed 76.7
- 77 GPT-5.6 Luna NCRI 15.2; no filler; sealed 76.6
- 78 deepseek/deepseek-chat-v3.1 NCRI 15.2; no filler; sealed 76.5
- 79 GLM-5.3-Flash NCRI 15.2; no filler; sealed 76.4
- 80 GLM-5 NCRI 15.2; no filler; sealed 76.2
- 81
DeepSeek V4 Flash 0731 NCRI 15.2; no filler; sealed 75.5
- 82 GLM-5.1 NCRI 15.2; no filler; sealed 75.2
- 83 GPT-4.1 NCRI 15.2; no filler; sealed 75.1
- 84
DeepSeek-V3.2 NCRI 15.2; no filler; sealed 75.0
- 85
Qwen3-235B-A22B-Instruct Jul 2025, NCRI 15.2; no filler; sealed 74.6
- 86 GLM-5.3 NCRI 15.2; no filler; sealed 74.5
- 87
Qwen3.8 Flash NCRI 15.2; no filler; sealed 74.4
- 88 deepseek/deepseek-chat-v3-0324 NCRI 15.2; no filler; sealed 74.4
- 89 openai/gpt-4o (snapshot unspecified) NCRI 15.2; no filler; sealed 74.2
- 90 GPT-4o Aug 2024, NCRI 15.2; no filler; sealed 74.2
- 91 Claude Haiku 4.5 NCRI 15.2; no filler; sealed 74.2
- 92
Ling-2.6-1T NCRI 15.2; no filler; sealed 73.9
- 93 openai/gpt-4 (snapshot unspecified) NCRI 15.2; no filler; sealed 73.7
- 94
WizardLM-2 8x22B NCRI 15.2; no filler; sealed 73.2
- 95
Qwen3-Coder-480B-A35B NCRI 15.2; no filler; sealed 73.1
- 96 deepseek/deepseek-chat NCRI 15.2; no filler; sealed 72.9
- 97 GPT-4 Jun 2023, NCRI 15.2; no filler; sealed 72.8
- 98
MiniMax-M3 NCRI 15.2; no filler; sealed 72.6
- 99
Tencent Hy3 NCRI 15.2; no filler; sealed 72.5
- 100 Mercury 3 NCRI 15.2; no filler; placed_after_release 72.3
- 101
Tencent Hy3 preview NCRI 15.2; no filler; sealed 72.3
- 102 GPT-4o May 2024, NCRI 15.2; no filler; sealed 72.2
- 103 GLM-4.7 NCRI 15.2; no filler; sealed 71.3
- 104 openai/gpt-4-turbo (snapshot unspecified) NCRI 15.2; no filler; sealed 71.3
- 105 GLM-4.5 NCRI 15.2; no filler; sealed 71.3
- 106 Qwen3-Coder-Plus NCRI 15.2; no filler; sealed 71.2
- 107 GLM-4.6 NCRI 15.2; no filler; sealed 70.8
- 108 GPT-5.1 NCRI 15.2; no filler; sealed 70.6
- 109
Qwen3-235B-A22B NCRI 15.2; no filler; sealed 70.5
- 110
Llama 4 Maverick NCRI 15.2; no filler; sealed 70.5
- 111 GPT-5.4 Mini NCRI 15.2; no filler; sealed 70.5
- 112 GPT-4o Nov 2024, NCRI 15.2; no filler; sealed 70.3
- 113
Inkling-Small NCRI 15.2; no filler; sealed 70.2
- 114
DiffusionGemma 26B A4B NCRI 15.2; no filler; sealed 70.2
- 115 MiMo-V2.5 NCRI 15.2; no filler; sealed 69.9
- 116
Gemma 4 26B A4B NCRI 15.2; no filler; sealed 69.5
- 117
Qwen3.5-122B-A10B NCRI 15.2; no filler; sealed 69.4
- 118 Grok 4.3 NCRI 15.2; no filler; sealed 69.2
- 119
Qwen3-235B-A22B-Thinking Jul 2025, NCRI 15.2; no filler; sealed 68.9
- 120
Qwen3-Next-80B-A3B-Instruct NCRI 15.2; no filler; sealed 68.4
- 121
MiniMax-M3.1-Flash NCRI 15.2; no filler; placed_after_release 68.4
- 122 Claude Haiku 5.5 NCRI 15.2; no filler; placed_after_release 68.3
- 123
Muse Glimmer NCRI 15.2; no filler; sealed 67.9
- 124
Mistral Medium 3.5 NCRI 15.2; no filler; sealed 67.5
- 125
Qwen3.6 27B NCRI 15.2; no filler; sealed 67.4
- 126 Grok 4.20 NCRI 15.2; no filler; sealed 67.1
- 127 GPT-5 NCRI 15.2; no filler; sealed 67.0
- 128 baidu/ernie-4.5-vl-424b-a47b NCRI 15.2; no filler; sealed 66.9
- 129 GLM-4.5V NCRI 15.2; no filler; sealed 66.7
- 130
Qwen3.5-27B NCRI 15.2; no filler; sealed 66.6
- 131
Ling-3.0-flash NCRI 15.2; no filler; sealed 66.3
- 132 nvidia/nemotron-3-super-120b-a12b NCRI 15.2; no filler; sealed 66.2
- 133
Qwen 3.8 27B NCRI 15.2; no filler; sealed 66.1
- 134 Mercury 2.5 NCRI 15.2; no filler; placed 65.8
- 135 inclusionai/ling-2.6-flash NCRI 15.2; no filler; sealed 65.5
- 136
Qwen3-Coder-Next NCRI 15.2; no filler; sealed 65.5
- 137
Solar Pro 4 NCRI 15.2; no filler; sealed 65.3
- 138
Qwen2.5-VL-72B NCRI 15.2; no filler; sealed 65.2
- 139 GPT-4.1 mini NCRI 15.2; no filler; sealed 65.0
- 140
MiniMax-M2.5 NCRI 15.2; no filler; sealed 65.0
- 141 reflection/beam-1.0-501b-a23b NCRI 15.2; no filler; placed_after_release 64.8
- 142 nousresearch/hermes-3-llama-3.1-405b NCRI 15.2; no filler; sealed 64.6
- 143
Qwen3.5-35B-A3B-Base NCRI 15.2; no filler; sealed 64.5
- 144 google/gemini-2.5-flash-lite (snapshot unspecified) NCRI 15.2; no filler; sealed 64.5
- 145
MiniMax-M2.7 NCRI 15.2; no filler; sealed 64.0
- 146
Qwen3-VL-32B-Instruct NCRI 15.2; no filler; sealed 64.0
- 147
Qwen3-Next-80B-A3B NCRI 15.2; no filler; sealed 63.8
- 148 sao10k/l3.3-euryale-70b NCRI 15.2; no filler; sealed 63.8
- 149 mistralai/mistral-small-2603 NCRI 15.2; no filler; sealed 63.5
- 150
Qwen2.5-72B NCRI 15.2; no filler; sealed 63.5
- 151 GLM-4.6V NCRI 15.2; no filler; sealed 63.4
- 152
Qwen 3.6 Flash NCRI 15.2; no filler; sealed 63.3
- 153
Qwen3.7 Flash NCRI 15.2; no filler; sealed 63.3
- 154 Hermes 4 405B NCRI 15.2; no filler; sealed 62.8
- 155 meta-llama/llama-3.1-70b-instruct NCRI 15.2; no filler; sealed 62.8
- 156 local/qwen2.5-32b-instruct NCRI 15.2; no filler; sealed 62.7
- 157
Llama 3.3 70B Instruct NCRI 15.2; no filler; sealed 62.6
- 158
Laguna S 2.1 NCRI 15.2; no filler; sealed 62.5
- 159 GLM-4.5-Air NCRI 15.2; no filler; sealed 62.4
- 160
Qwen3-30B-A3B-Instruct Jul 2025, NCRI 15.2; no filler; sealed 62.1
- 161
Qwen 3.6 35B-A3B NCRI 15.2; no filler; sealed 61.6
- 162
Qwen3.5-Flash NCRI 15.2; no filler; sealed 61.4
- 163 amazon/nova-pro-v1 NCRI 15.2; no filler; sealed 61.3
- 164
Qwen2.5-32B NCRI 15.2; no filler; sealed 61.1
- 165 sao10k/l3.1-euryale-70b NCRI 15.2; no filler; sealed 61.0
- 166
Qwen2-72B NCRI 15.2; no filler; sealed 61.0
- 167
Qwen3.5-35B-A3B NCRI 15.2; no filler; sealed 60.1
- 168
Qwen2.5-Coder-32B-Instruct NCRI 15.2; no filler; sealed 60.0
- 169
Qwen3-VL-30B-A3B-Instruct NCRI 15.2; no filler; sealed 59.9
- 170
Gemma 4 12B NCRI 15.2; no filler; sealed 59.7
- 171
Qwen3-Coder-30B-A3B-Instruct NCRI 15.2; no filler; sealed 59.6
- 172
Qwen3-Coder-Flash NCRI 15.2; no filler; sealed 59.5
- 173 Ripple-65B Preview NCRI 15.2; no filler; placed_after_release 59.3
- 174
Qwen3-32B NCRI 15.2; no filler; sealed 59.2
- 175 thedrummer/cydonia-24b-v4.1 NCRI 15.2; no filler; sealed 59.1
- 176 gpt-oss-120b NCRI 15.2; no filler; sealed 59.0
- 177 mistralai/ministral-14b-2512 NCRI 15.2; no filler; sealed 58.7
- 178
Phi-4 NCRI 15.2; no filler; sealed 58.3
- 179 nousresearch/hermes-4-70b NCRI 15.2; no filler; sealed 57.9
- 180
Llama 4 Scout NCRI 15.2; no filler; sealed 57.7
- 181
Llama 3.1-70B NCRI 15.2; no filler; sealed 57.6
- 182
Laguna XS 2.1 NCRI 15.2; no filler; sealed 57.5
- 183
Gemma 3 27B PT NCRI 15.2; no filler; sealed 57.5
- 184 Claude 3 Haiku NCRI 15.2; no filler; sealed 57.3
- 185 local/mistral-small-24b-base-2501 NCRI 15.2; no filler; sealed 56.2
- 186
Nemotron 3.5 Lightning 30B-A3B NCRI 15.2; no filler; sealed 56.1
- 187
Qwen3-VL-8B-Instruct NCRI 15.2; no filler; sealed 56.0
- 188
Gemma 3 27B NCRI 15.2; no filler; sealed 56.0
- 189 glm-4.7-flash NCRI 15.2; no filler; sealed 55.9
- 190 GPT-4o mini NCRI 15.2; no filler; sealed 55.7
- 191 mistralai/mistral-small-24b-instruct-2501 NCRI 15.2; no filler; sealed 55.4
- 192 openai/gpt-3.5-turbo (snapshot unspecified) NCRI 15.2; no filler; sealed 55.2
- 193 Mercury 2 NCRI 15.2; no filler; sealed 55.1
- 194
Qwen3-14B NCRI 15.2; no filler; sealed 55.0
- 195 gpt-oss-20b NCRI 15.2; no filler; sealed 54.8
- 196
Olmo 3.1 32B Instruct NCRI 15.2; no filler; placed 54.3
- 197
Qwen3.5-9B-Base NCRI 15.2; no filler; sealed 54.2
- 198 local/olmo-3-32b-dpo NCRI 15.2; no filler; sealed 54.0
- 199 mistralai/mistral-small-3.2-24b-instruct NCRI 15.2; no filler; sealed 53.8
- 200 local/olmo-3-32b-rl NCRI 15.2; no filler; sealed 53.8
- 201 local/olmo-3-32b-mid NCRI 15.2; no filler; sealed 53.3
- 202
Gemma 2 27B NCRI 15.2; no filler; sealed 53.3
- 203 local/olmo-3-32b-sft NCRI 15.2; no filler; sealed 52.8
- 204 google/gemma-2-27b-it NCRI 15.2; no filler; sealed 52.6
- 205 local/nanbeige42-r2 NCRI 15.2; no filler; placed_after_release 52.6
- 206 local/qwen1.5-110b-chat NCRI 15.2; no filler; sealed 52.6
- 207
Qwen3.5-9B NCRI 15.2; no filler; sealed 52.4
- 208 local/ouro26-r4 NCRI 15.2; no filler; placed_after_release 51.7
- 209
Qwen3-8B NCRI 15.2; no filler; sealed 51.4
- 210 mistralai/ministral-8b-2512 NCRI 15.2; no filler; sealed 51.0
- 211 GPT-5 mini NCRI 15.2; no filler; sealed 50.7
- 212
Gemma 4 E4B NCRI 15.2; no filler; sealed 50.6
- 213
Qwen3-30B-A3B NCRI 15.2; no filler; sealed 50.4
- 214 GPT-4.1 nano NCRI 15.2; no filler; sealed 50.2
- 215
Gemma 3 12B PT NCRI 15.2; no filler; sealed 49.9
- 216
Qwen1.5-72B NCRI 15.2; no filler; sealed 49.7
- 217 GPT-3.5 Turbo Instruct NCRI 15.2; no filler; sealed 49.6
- 218 GPT-5.4 Nano NCRI 15.2; no filler; sealed 49.4
- 219
Gemma 3 12B NCRI 15.2; no filler; sealed 49.3
- 220
Hunyuan-A13B-Instruct NCRI 15.2; no filler; sealed 48.7
- 221 qwen/qwen-2.5-7b-instruct NCRI 15.2; no filler; sealed 48.4
- 222 local/gemma-2-9b-it NCRI 15.2; no filler; sealed 47.5
- 223
Nemotron 3 Nano 30B-A3B NCRI 15.2; no filler; sealed 47.3
- 224 local/ouro14-r4 NCRI 15.2; no filler; placed_after_release 47.3
- 225 mistralai/ministral-3b-2512 NCRI 15.2; no filler; sealed 46.2
- 226
Qwen3.5-4B-Base NCRI 15.2; no filler; sealed 46.0
- 227 local/olmo-3-32b-pretrain NCRI 15.2; no filler; sealed 45.0
- 228
Gemma 2 9B NCRI 15.2; no filler; sealed 44.7
- 229
Qwen3.5-4B NCRI 15.2; no filler; sealed 44.3
- 230
Gemma 4 E2B NCRI 15.2; no filler; sealed 44.0
- 231
UI-TARS-1.5 NCRI 15.2; no filler; sealed 43.9
- 232
Mistral-Nemo-Base-2407 NCRI 15.2; no filler; sealed 43.2
- 233 thedrummer/unslopnemo-12b NCRI 15.2; no filler; sealed 42.8
- 234
Mistral NeMo NCRI 15.2; no filler; sealed 42.5
- 235 local/llama-3.1-8b-instruct-fp8 NCRI 15.2; no filler; sealed 41.8
- 236 meta-llama/llama-3.1-8b-instruct NCRI 15.2; no filler; sealed 41.6
- 237 local/llama-3.1-8b-instruct NCRI 15.2; no filler; sealed 40.9
- 238 local/olmo3-32b-step260000 NCRI 15.2; no filler; placed 40.8
- 239 local/llama-2-70b-hf NCRI 15.2; no filler; sealed 40.7
- 240 sao10k/l3-lunaris-8b NCRI 15.2; no filler; sealed 40.7
- 241
Llama 3.1-8B NCRI 15.2; no filler; sealed 40.6
- 242 local/llama-3-8b-instruct NCRI 15.2; no filler; sealed 39.2
- 243
Gemma 3n E4B NCRI 15.2; no filler; sealed 38.5
- 244 local/olmo3-32b-step360000 NCRI 15.2; no filler; placed 38.5
- 245
Gemma 3 4B NCRI 15.2; no filler; sealed 37.9
- 246
LLaMA-65B NCRI 15.2; no filler; sealed 37.5
- 247 aion-labs/aion-rp-llama-3.1-8b NCRI 15.2; no filler; sealed 37.2
- 248 ibm-granite/granite-4.0-h-micro NCRI 15.2; no filler; sealed 37.2
- 249 local/olmo3-32b-step130000 NCRI 15.2; no filler; placed 36.2
- 250
Qwen3.5-2B NCRI 15.2; no filler; sealed 35.5
- 251 local/olmo3-32b-step92000 NCRI 15.2; no filler; placed 34.5
- 252 local/olmo3-32b-step182000 NCRI 15.2; no filler; sealed 34.4
- 253 davinci-002 NCRI 15.2; no filler; sealed 32.6
- 254 local/olmo3-7b-step1413814 NCRI 15.2; no filler; sealed 32.0
- 255 local/olmo3-32b-step65000 NCRI 15.2; no filler; placed 31.4
- 256 GPT-5 nano NCRI 15.2; no filler; sealed 31.0
- 257 local/olmo3-32b-step46000 NCRI 15.2; no filler; sealed 29.5
- 258 local/olmo3-32b-step33000 NCRI 15.2; no filler; placed 27.4
- 259 local/gemma-7b-it NCRI 15.2; no filler; sealed 25.8
- 260 undi95/remm-slerp-l2-13b NCRI 15.2; no filler; sealed 25.7
- 261
Qwen3.5-0.8B NCRI 15.2; no filler; sealed 24.9
- 262 local/olmo3-7b-step360000 NCRI 15.2; no filler; sealed 24.6
- 263 local/gemma-2-2b NCRI 15.2; no filler; sealed 24.3
- 264
Gemma 2 2B IT NCRI 15.2; no filler; sealed 24.1
- 265 local/talkie-1930-13b-it NCRI 15.2; no filler; sealed 23.9
- 266 gryphe/mythomax-l2-13b NCRI 15.2; no filler; sealed 23.8
- 267 local/olmo3-7b-step714000 NCRI 15.2; no filler; sealed 23.2
- 268 local/olmo3-7b-step182000 NCRI 15.2; no filler; sealed 22.7
- 269 local/olmo3-32b-step16000 NCRI 15.2; no filler; placed 21.5
- 270 local/olmo3-32b-step23000 NCRI 15.2; no filler; placed 20.4
- 271 local/olmo3-7b-step92000 NCRI 15.2; no filler; sealed 18.9
- 272 local/llama-2-7b-hf NCRI 15.2; no filler; sealed 18.7
- 273 local/huginn-r12 NCRI 15.2; no filler; placed_after_release 16.8
- 274
Gemma 3 1B NCRI 15.2; no filler; sealed 16.1
- 275 local/olmo3-32b-step12000 NCRI 15.2; no filler; placed 15.6
- 276
LLaMA-7B NCRI 15.2; no filler; sealed 15.0
- 277 local/gemma-2b-it NCRI 15.2; no filler; sealed 11.9
- 278 local/talkie-web-13b-base NCRI 15.2; no filler; sealed 10.6
- 279 openai/babbage-002 NCRI 15.2; no filler; sealed 6.6
- 280 local/olmo3-7b-step46000 NCRI 15.2; no filler; sealed 4.1
- 281 local/olmo3-7b-step23000 NCRI 15.2; no filler; sealed 3.1
- 282 local/olmo3-7b-step12000 NCRI 15.2; no filler; sealed -8.3
- 283 local/pythia-12b NCRI 15.2; no filler; sealed -8.6
- 284 local/olmo3-32b-step6000 NCRI 15.2; no filler; placed -9.9
- 285 local/olmo3-7b-step6000 NCRI 15.2; no filler; sealed -11.5
- 286 local/talkie-1930-13b-base NCRI 15.2; no filler; sealed -12.4
- 287
Gemma 3 270M NCRI 15.2; no filler; sealed -23.1
Source: Official nocot-bench (NCRI) leaderboard
- 173.9 GPT-6 Astra
- 156.6 GPT-6.1 Sol
- 154.1
Gemini 4 Argon
- 149.6 GPT-6.1 Terra
- 129.7
Gemini 3.8 Flash
- 129.6 Claude Fable 5.1
- 127.6
Gemini 3.7 Flash
- 123.9
Gemini 3.1 Pro
- 122.2 GPT-6 Sol
- 121.5 Claude Fable 5
- 119.5 GPT-5.6 Sol
- 118.9 Claude Opus 5.5
- 115.8 Claude Opus 5
- 111.0 GPT-5.5
- 110.9
Gemini 3.6 Flash
- 110.8 Claude Opus 4.6
- 110.5
Gemini 3 Flash
- 110.5 Claude Opus 4.5
- 109.0 Claude Opus 4
- 108.8
Muse Spark 1.1
- 107.5 Claude Opus 4.1
- 107.5
Gemini 3.5 Flash
- 105.6 Kimi K3
- 103.9 Claude Opus 4.7
- 102.8 Kimi K3.1 Pro
- 102.3
Muse Spark 1.2
- 102.2 Claude Opus 4.8
- 100.8
Muse Spark 1.3
- 100.8 Claude Sonnet 4.5
- 97.7
Muse Wonder
- 96.5 Claude Sonnet 4
- 95.7
Muse Spark Open
- 95.1 GPT-5.6 Terra
- 95.1 Claude Sonnet 5.5
- 94.1 GPT-5.4
- 94.0
Tencent Hy4
- 91.1 Claude Sonnet 5
- 90.0 Claude Sonnet 4.6
- 87.0
Qwen 3.6 Max (Preview)
- 86.8 Grok 4.6
- 86.5
Gemini 3.1 Flash-Lite
- 86.0
Qwen3.8-2.4T-A95B
- 85.8 GPT-5.2
- 85.8
Qwen3.7-Max
- 85.4
Qwen 3.8 Max
- 85.2 GPT-5.3 Codex
- 84.1
DeepSeek V4 Pro 0813
- 84.1
DeepSeek-V4-Pro
- 83.3
Qwen3-Max
- 83.1
Qwen3.7-Plus
- 82.8
Tencent Hy4 preview
- 82.5
LongCat-2.0
- 82.2
Qwen 3.6 Plus
- 81.7
Gemma 4 31B IT
- 81.6
DeepSeek V4.1 Flash 1017
- 81.3
DeepSeek V4.1 Flash
- 81.0 Grok 4.5
- 80.6
Gemini 3.5 Flash-Lite
- 80.6
Qwen3.5-Plus
- 80.6 Kimi K2.5
- 80.0
Qwen3.5 397B-A17B
- 79.2 Kimi K2.6
- 79.1
DeepSeek-V4-Flash-Base
- 78.3
Inkling
- 78.1
DeepSeek-V4-Flash
- 77.6
DeepSeek-V3.1-Terminus
- 77.3 GLM-5.2
- 77.3 GPT-6 Luna
- 77.3
Qwen3-VL-235B-A22B-Instruct
- 77.1 MiMo-V2.5-Pro
- 76.7
DeepSeek-V3.2-Exp
- 76.6 GPT-5.6 Luna
- 76.4 GLM-5.3-Flash
- 76.2 GLM-5
- 75.5
DeepSeek V4 Flash 0731
- 75.2 GLM-5.1
- 75.1 GPT-4.1
- 75.0
DeepSeek-V3.2
- 74.6
Qwen3-235B-A22B-Instruct
- 74.5 GLM-5.3
- 74.4
Qwen3.8 Flash
- 74.2 GPT-4o
- 74.2 Claude Haiku 4.5
- 73.9
Ling-2.6-1T
- 73.2
WizardLM-2 8x22B
- 73.1
Qwen3-Coder-480B-A35B
- 72.8 GPT-4
- 72.6
MiniMax-M3
- 72.5
Tencent Hy3
- 72.3 Mercury 3
- 72.3
Tencent Hy3 preview
- 72.2 GPT-4o
- 71.3 GLM-4.7
- 71.3 GLM-4.5
- 71.2 Qwen3-Coder-Plus
- 70.8 GLM-4.6
- 70.6 GPT-5.1
- 70.5
Qwen3-235B-A22B
- 70.5
Llama 4 Maverick
- 70.5 GPT-5.4 Mini
- 70.3 GPT-4o
- 70.2
Inkling-Small
- 70.2
DiffusionGemma 26B A4B
- 69.9 MiMo-V2.5
- 69.5
Gemma 4 26B A4B
- 69.4
Qwen3.5-122B-A10B
- 69.2 Grok 4.3
- 68.9
Qwen3-235B-A22B-Thinking
- 68.4
Qwen3-Next-80B-A3B-Instruct
- 68.4
MiniMax-M3.1-Flash
- 68.3 Claude Haiku 5.5
- 67.9
Muse Glimmer
- 67.5
Mistral Medium 3.5
- 67.4
Qwen3.6 27B
- 67.1 Grok 4.20
- 67.0 GPT-5
- 66.7 GLM-4.5V
- 66.6
Qwen3.5-27B
- 66.3
Ling-3.0-flash
- 66.1
Qwen 3.8 27B
- 65.8 Mercury 2.5
- 65.5
Qwen3-Coder-Next
- 65.3
Solar Pro 4
- 65.2
Qwen2.5-VL-72B
- 65.0 GPT-4.1 mini
- 65.0
MiniMax-M2.5
- 64.5
Qwen3.5-35B-A3B-Base
- 64.0
MiniMax-M2.7
- 64.0
Qwen3-VL-32B-Instruct
- 63.8
Qwen3-Next-80B-A3B
- 63.5
Qwen2.5-72B
- 63.4 GLM-4.6V
- 63.3
Qwen 3.6 Flash
- 63.3
Qwen3.7 Flash
- 62.8 Hermes 4 405B
- 62.6
Llama 3.3 70B Instruct
- 62.5
Laguna S 2.1
- 62.4 GLM-4.5-Air
- 62.1
Qwen3-30B-A3B-Instruct
- 61.6
Qwen 3.6 35B-A3B
- 61.4
Qwen3.5-Flash
- 61.1
Qwen2.5-32B
- 61.0
Qwen2-72B
- 60.1
Qwen3.5-35B-A3B
- 60.0
Qwen2.5-Coder-32B-Instruct
- 59.9
Qwen3-VL-30B-A3B-Instruct
- 59.7
Gemma 4 12B
- 59.6
Qwen3-Coder-30B-A3B-Instruct
- 59.5
Qwen3-Coder-Flash
- 59.3 Ripple-65B Preview
- 59.2
Qwen3-32B
- 59.0 gpt-oss-120b
- 58.3
Phi-4
- 57.7
Llama 4 Scout
- 57.6
Llama 3.1-70B
- 57.5
Laguna XS 2.1
- 57.5
Gemma 3 27B PT
- 57.3 Claude 3 Haiku
- 56.1
Nemotron 3.5 Lightning 30B-A3B
- 56.0
Qwen3-VL-8B-Instruct
- 56.0
Gemma 3 27B
- 55.9 glm-4.7-flash
- 55.7 GPT-4o mini
- 55.1 Mercury 2
- 55.0
Qwen3-14B
- 54.8 gpt-oss-20b
- 54.3
Olmo 3.1 32B Instruct
- 54.2
Qwen3.5-9B-Base
- 53.3
Gemma 2 27B
- 52.4
Qwen3.5-9B
- 51.4
Qwen3-8B
- 50.7 GPT-5 mini
- 50.6
Gemma 4 E4B
- 50.4
Qwen3-30B-A3B
- 50.2 GPT-4.1 nano
- 49.9
Gemma 3 12B PT
- 49.7
Qwen1.5-72B
- 49.6 GPT-3.5 Turbo Instruct
- 49.4 GPT-5.4 Nano
- 49.3
Gemma 3 12B
- 48.7
Hunyuan-A13B-Instruct
- 47.3
Nemotron 3 Nano 30B-A3B
- 46.0
Qwen3.5-4B-Base
- 44.7
Gemma 2 9B
- 44.3
Qwen3.5-4B
- 44.0
Gemma 4 E2B
- 43.9
UI-TARS-1.5
- 43.2
Mistral-Nemo-Base-2407
- 42.5
Mistral NeMo
- 40.6
Llama 3.1-8B
- 38.5
Gemma 3n E4B
- 37.9
Gemma 3 4B
- 37.5
LLaMA-65B
- 35.5
Qwen3.5-2B
- 32.6 davinci-002
- 31.0 GPT-5 nano
- 24.9
Qwen3.5-0.8B
- 24.1
Gemma 2 2B IT
- 16.1
Gemma 3 1B
- 15.0
LLaMA-7B
- -23.1
Gemma 3 270M
- 1 GPT-6 Astra NCRI 15.2; dots@2048; placed 173.9
- 2 GPT-6.1 Sol NCRI 15.2; dots@128; placed_after_release 156.6
- 3
Gemini 4 Argon NCRI 15.2; qrep@512; placed_after_release 154.1
- 4 GPT-6.1 Terra NCRI 15.2; qrep@512; placed_after_release 149.6
- 5
Gemini 3.8 Flash NCRI 15.2; dots@128; placed 129.7
- 6 Claude Fable 5.1 NCRI 15.2; dots@128; placed 129.6
- 7
Gemini 3.7 Flash NCRI 15.2; dots@32; sealed 127.6
- 8
Gemini 3.1 Pro NCRI 15.2; count@512; sealed 123.9
- 9 GPT-6 Sol NCRI 15.2; dots@128; placed_after_release 122.2
- 10 Claude Fable 5 NCRI 15.2; dots@32; sealed 121.5
- 11 GPT-5.6 Sol NCRI 15.2; qrep@512; sealed 119.5
- 12 Claude Opus 5.5 NCRI 15.2; dots@512; placed_after_release 118.9
- 13 Claude Opus 5 NCRI 15.2; dots@128; sealed 115.8
- 14 GPT-5.5 NCRI 15.2; qrep@512; sealed 111.0
- 15
Gemini 3.6 Flash NCRI 15.2; count@512; sealed 110.9
- 16 Claude Opus 4.6 NCRI 15.2; qrep@512; sealed 110.8
- 17
Gemini 3 Flash NCRI 15.2; count@512; sealed 110.5
- 18 Claude Opus 4.5 NCRI 15.2; dots@512; sealed 110.5
- 19 Claude Opus 4 NCRI 15.2; dots@512; sealed 109.0
- 20
Muse Spark 1.1 NCRI 15.2; qrep@512; sealed 108.8
- 21 Claude Opus 4.1 NCRI 15.2; dots@128; sealed 107.5
- 22
Gemini 3.5 Flash NCRI 15.2; dots@512; sealed 107.5
- 23 Kimi K3 NCRI 15.2; qrep@512; sealed 105.6
- 24 Claude Opus 4.7 NCRI 15.2; dots@128; sealed 103.9
- 25 Kimi K3.1 Pro NCRI 15.2; no filler; placed_after_release 102.8
- 26
Muse Spark 1.2 NCRI 15.2; dots@128; sealed 102.3
- 27 Claude Opus 4.8 NCRI 15.2; dots@128; sealed 102.2
- 28
Muse Spark 1.3 NCRI 15.2; dots@2048; placed 100.8
- 29 Claude Sonnet 4.5 NCRI 15.2; qrep@512; sealed 100.8
- 30
Muse Wonder NCRI 15.2; no filler; placed_after_release 97.7
- 31 Claude Sonnet 4 NCRI 15.2; count@512; sealed 96.5
- 32
Muse Spark Open NCRI 15.2; no filler; placed_after_release 95.7
- 33 GPT-5.6 Terra NCRI 15.2; qrep@512; sealed 95.1
- 34 Claude Sonnet 5.5 NCRI 15.2; no filler; placed_after_release 95.1
- 35 GPT-5.4 NCRI 15.2; dots@128; sealed 94.1
- 36
Tencent Hy4 NCRI 15.2; no filler; placed_after_release 94.0
- 37 Claude Sonnet 5 NCRI 15.2; dots@128; sealed 91.1
- 38 Claude Sonnet 4.6 NCRI 15.2; dots@512; sealed 90.0
- 39
Qwen 3.6 Max (Preview) NCRI 15.2; no filler; sealed 87.0
- 40 Grok 4.6 NCRI 15.2; no filler; sealed 86.8
- 41
Gemini 3.1 Flash-Lite NCRI 15.2; no filler; sealed 86.5
- 42
Qwen3.8-2.4T-A95B NCRI 15.2; no filler; sealed 86.0
- 43 GPT-5.2 NCRI 15.2; no filler; sealed 85.8
- 44
Qwen3.7-Max NCRI 15.2; no filler; sealed 85.8
- 45
Qwen 3.8 Max NCRI 15.2; no filler; sealed 85.4
- 46 GPT-5.3 Codex NCRI 15.2; no filler; sealed 85.2
- 47
DeepSeek V4 Pro 0813 NCRI 15.2; no filler; sealed 84.1
- 48
DeepSeek-V4-Pro NCRI 15.2; no filler; sealed 84.1
- 49
Qwen3-Max NCRI 15.2; no filler; sealed 83.3
- 50
Qwen3.7-Plus NCRI 15.2; no filler; sealed 83.1
- 51
Tencent Hy4 preview NCRI 15.2; no filler; sealed 82.8
- 52
LongCat-2.0 NCRI 15.2; no filler; sealed 82.5
- 53
Qwen 3.6 Plus NCRI 15.2; no filler; sealed 82.2
- 54
Gemma 4 31B IT NCRI 15.2; no filler; sealed 81.7
- 55
DeepSeek V4.1 Flash 1017 NCRI 15.2; no filler; placed_after_release 81.6
- 56
DeepSeek V4.1 Flash NCRI 15.2; no filler; placed_after_release 81.3
- 57 Grok 4.5 NCRI 15.2; no filler; sealed 81.0
- 58
Gemini 3.5 Flash-Lite NCRI 15.2; no filler; sealed 80.6
- 59
Qwen3.5-Plus NCRI 15.2; no filler; sealed 80.6
- 60 Kimi K2.5 NCRI 15.2; no filler; sealed 80.6
- 61
Qwen3.5 397B-A17B NCRI 15.2; no filler; sealed 80.0
- 62 Kimi K2.6 NCRI 15.2; no filler; sealed 79.2
- 63
DeepSeek-V4-Flash-Base NCRI 15.2; no filler; sealed 79.1
- 64
Inkling NCRI 15.2; no filler; sealed 78.3
- 65
DeepSeek-V4-Flash NCRI 15.2; no filler; sealed 78.1
- 66
DeepSeek-V3.1-Terminus NCRI 15.2; no filler; sealed 77.6
- 67 GLM-5.2 NCRI 15.2; no filler; sealed 77.3
- 68 GPT-6 Luna NCRI 15.2; no filler; placed_after_release 77.3
- 69
Qwen3-VL-235B-A22B-Instruct NCRI 15.2; no filler; sealed 77.3
- 70 MiMo-V2.5-Pro NCRI 15.2; no filler; sealed 77.1
- 71
DeepSeek-V3.2-Exp NCRI 15.2; no filler; sealed 76.7
- 72 GPT-5.6 Luna NCRI 15.2; no filler; sealed 76.6
- 73 GLM-5.3-Flash NCRI 15.2; no filler; sealed 76.4
- 74 GLM-5 NCRI 15.2; no filler; sealed 76.2
- 75
DeepSeek V4 Flash 0731 NCRI 15.2; no filler; sealed 75.5
- 76 GLM-5.1 NCRI 15.2; no filler; sealed 75.2
- 77 GPT-4.1 NCRI 15.2; no filler; sealed 75.1
- 78
DeepSeek-V3.2 NCRI 15.2; no filler; sealed 75.0
- 79
Qwen3-235B-A22B-Instruct Jul 2025, NCRI 15.2; no filler; sealed 74.6
- 80 GLM-5.3 NCRI 15.2; no filler; sealed 74.5
- 81
Qwen3.8 Flash NCRI 15.2; no filler; sealed 74.4
- 82 GPT-4o Aug 2024, NCRI 15.2; no filler; sealed 74.2
- 83 Claude Haiku 4.5 NCRI 15.2; no filler; sealed 74.2
- 84
Ling-2.6-1T NCRI 15.2; no filler; sealed 73.9
- 85
WizardLM-2 8x22B NCRI 15.2; no filler; sealed 73.2
- 86
Qwen3-Coder-480B-A35B NCRI 15.2; no filler; sealed 73.1
- 87 GPT-4 Jun 2023, NCRI 15.2; no filler; sealed 72.8
- 88
MiniMax-M3 NCRI 15.2; no filler; sealed 72.6
- 89
Tencent Hy3 NCRI 15.2; no filler; sealed 72.5
- 90 Mercury 3 NCRI 15.2; no filler; placed_after_release 72.3
- 91
Tencent Hy3 preview NCRI 15.2; no filler; sealed 72.3
- 92 GPT-4o May 2024, NCRI 15.2; no filler; sealed 72.2
- 93 GLM-4.7 NCRI 15.2; no filler; sealed 71.3
- 94 GLM-4.5 NCRI 15.2; no filler; sealed 71.3
- 95 Qwen3-Coder-Plus NCRI 15.2; no filler; sealed 71.2
- 96 GLM-4.6 NCRI 15.2; no filler; sealed 70.8
- 97 GPT-5.1 NCRI 15.2; no filler; sealed 70.6
- 98
Qwen3-235B-A22B NCRI 15.2; no filler; sealed 70.5
- 99
Llama 4 Maverick NCRI 15.2; no filler; sealed 70.5
- 100 GPT-5.4 Mini NCRI 15.2; no filler; sealed 70.5
- 101 GPT-4o Nov 2024, NCRI 15.2; no filler; sealed 70.3
- 102
Inkling-Small NCRI 15.2; no filler; sealed 70.2
- 103
DiffusionGemma 26B A4B NCRI 15.2; no filler; sealed 70.2
- 104 MiMo-V2.5 NCRI 15.2; no filler; sealed 69.9
- 105
Gemma 4 26B A4B NCRI 15.2; no filler; sealed 69.5
- 106
Qwen3.5-122B-A10B NCRI 15.2; no filler; sealed 69.4
- 107 Grok 4.3 NCRI 15.2; no filler; sealed 69.2
- 108
Qwen3-235B-A22B-Thinking Jul 2025, NCRI 15.2; no filler; sealed 68.9
- 109
Qwen3-Next-80B-A3B-Instruct NCRI 15.2; no filler; sealed 68.4
- 110
MiniMax-M3.1-Flash NCRI 15.2; no filler; placed_after_release 68.4
- 111 Claude Haiku 5.5 NCRI 15.2; no filler; placed_after_release 68.3
- 112
Muse Glimmer NCRI 15.2; no filler; sealed 67.9
- 113
Mistral Medium 3.5 NCRI 15.2; no filler; sealed 67.5
- 114
Qwen3.6 27B NCRI 15.2; no filler; sealed 67.4
- 115 Grok 4.20 NCRI 15.2; no filler; sealed 67.1
- 116 GPT-5 NCRI 15.2; no filler; sealed 67.0
- 117 GLM-4.5V NCRI 15.2; no filler; sealed 66.7
- 118
Qwen3.5-27B NCRI 15.2; no filler; sealed 66.6
- 119
Ling-3.0-flash NCRI 15.2; no filler; sealed 66.3
- 120
Qwen 3.8 27B NCRI 15.2; no filler; sealed 66.1
- 121 Mercury 2.5 NCRI 15.2; no filler; placed 65.8
- 122
Qwen3-Coder-Next NCRI 15.2; no filler; sealed 65.5
- 123
Solar Pro 4 NCRI 15.2; no filler; sealed 65.3
- 124
Qwen2.5-VL-72B NCRI 15.2; no filler; sealed 65.2
- 125 GPT-4.1 mini NCRI 15.2; no filler; sealed 65.0
- 126
MiniMax-M2.5 NCRI 15.2; no filler; sealed 65.0
- 127
Qwen3.5-35B-A3B-Base NCRI 15.2; no filler; sealed 64.5
- 128
MiniMax-M2.7 NCRI 15.2; no filler; sealed 64.0
- 129
Qwen3-VL-32B-Instruct NCRI 15.2; no filler; sealed 64.0
- 130
Qwen3-Next-80B-A3B NCRI 15.2; no filler; sealed 63.8
- 131
Qwen2.5-72B NCRI 15.2; no filler; sealed 63.5
- 132 GLM-4.6V NCRI 15.2; no filler; sealed 63.4
- 133
Qwen 3.6 Flash NCRI 15.2; no filler; sealed 63.3
- 134
Qwen3.7 Flash NCRI 15.2; no filler; sealed 63.3
- 135 Hermes 4 405B NCRI 15.2; no filler; sealed 62.8
- 136
Llama 3.3 70B Instruct NCRI 15.2; no filler; sealed 62.6
- 137
Laguna S 2.1 NCRI 15.2; no filler; sealed 62.5
- 138 GLM-4.5-Air NCRI 15.2; no filler; sealed 62.4
- 139
Qwen3-30B-A3B-Instruct Jul 2025, NCRI 15.2; no filler; sealed 62.1
- 140
Qwen 3.6 35B-A3B NCRI 15.2; no filler; sealed 61.6
- 141
Qwen3.5-Flash NCRI 15.2; no filler; sealed 61.4
- 142
Qwen2.5-32B NCRI 15.2; no filler; sealed 61.1
- 143
Qwen2-72B NCRI 15.2; no filler; sealed 61.0
- 144
Qwen3.5-35B-A3B NCRI 15.2; no filler; sealed 60.1
- 145
Qwen2.5-Coder-32B-Instruct NCRI 15.2; no filler; sealed 60.0
- 146
Qwen3-VL-30B-A3B-Instruct NCRI 15.2; no filler; sealed 59.9
- 147
Gemma 4 12B NCRI 15.2; no filler; sealed 59.7
- 148
Qwen3-Coder-30B-A3B-Instruct NCRI 15.2; no filler; sealed 59.6
- 149
Qwen3-Coder-Flash NCRI 15.2; no filler; sealed 59.5
- 150 Ripple-65B Preview NCRI 15.2; no filler; placed_after_release 59.3
- 151
Qwen3-32B NCRI 15.2; no filler; sealed 59.2
- 152 gpt-oss-120b NCRI 15.2; no filler; sealed 59.0
- 153
Phi-4 NCRI 15.2; no filler; sealed 58.3
- 154
Llama 4 Scout NCRI 15.2; no filler; sealed 57.7
- 155
Llama 3.1-70B NCRI 15.2; no filler; sealed 57.6
- 156
Laguna XS 2.1 NCRI 15.2; no filler; sealed 57.5
- 157
Gemma 3 27B PT NCRI 15.2; no filler; sealed 57.5
- 158 Claude 3 Haiku NCRI 15.2; no filler; sealed 57.3
- 159
Nemotron 3.5 Lightning 30B-A3B NCRI 15.2; no filler; sealed 56.1
- 160
Qwen3-VL-8B-Instruct NCRI 15.2; no filler; sealed 56.0
- 161
Gemma 3 27B NCRI 15.2; no filler; sealed 56.0
- 162 glm-4.7-flash NCRI 15.2; no filler; sealed 55.9
- 163 GPT-4o mini NCRI 15.2; no filler; sealed 55.7
- 164 Mercury 2 NCRI 15.2; no filler; sealed 55.1
- 165
Qwen3-14B NCRI 15.2; no filler; sealed 55.0
- 166 gpt-oss-20b NCRI 15.2; no filler; sealed 54.8
- 167
Olmo 3.1 32B Instruct NCRI 15.2; no filler; placed 54.3
- 168
Qwen3.5-9B-Base NCRI 15.2; no filler; sealed 54.2
- 169
Gemma 2 27B NCRI 15.2; no filler; sealed 53.3
- 170
Qwen3.5-9B NCRI 15.2; no filler; sealed 52.4
- 171
Qwen3-8B NCRI 15.2; no filler; sealed 51.4
- 172 GPT-5 mini NCRI 15.2; no filler; sealed 50.7
- 173
Gemma 4 E4B NCRI 15.2; no filler; sealed 50.6
- 174
Qwen3-30B-A3B NCRI 15.2; no filler; sealed 50.4
- 175 GPT-4.1 nano NCRI 15.2; no filler; sealed 50.2
- 176
Gemma 3 12B PT NCRI 15.2; no filler; sealed 49.9
- 177
Qwen1.5-72B NCRI 15.2; no filler; sealed 49.7
- 178 GPT-3.5 Turbo Instruct NCRI 15.2; no filler; sealed 49.6
- 179 GPT-5.4 Nano NCRI 15.2; no filler; sealed 49.4
- 180
Gemma 3 12B NCRI 15.2; no filler; sealed 49.3
- 181
Hunyuan-A13B-Instruct NCRI 15.2; no filler; sealed 48.7
- 182
Nemotron 3 Nano 30B-A3B NCRI 15.2; no filler; sealed 47.3
- 183
Qwen3.5-4B-Base NCRI 15.2; no filler; sealed 46.0
- 184
Gemma 2 9B NCRI 15.2; no filler; sealed 44.7
- 185
Qwen3.5-4B NCRI 15.2; no filler; sealed 44.3
- 186
Gemma 4 E2B NCRI 15.2; no filler; sealed 44.0
- 187
UI-TARS-1.5 NCRI 15.2; no filler; sealed 43.9
- 188
Mistral-Nemo-Base-2407 NCRI 15.2; no filler; sealed 43.2
- 189
Mistral NeMo NCRI 15.2; no filler; sealed 42.5
- 190
Llama 3.1-8B NCRI 15.2; no filler; sealed 40.6
- 191
Gemma 3n E4B NCRI 15.2; no filler; sealed 38.5
- 192
Gemma 3 4B NCRI 15.2; no filler; sealed 37.9
- 193
LLaMA-65B NCRI 15.2; no filler; sealed 37.5
- 194
Qwen3.5-2B NCRI 15.2; no filler; sealed 35.5
- 195 davinci-002 NCRI 15.2; no filler; sealed 32.6
- 196 GPT-5 nano NCRI 15.2; no filler; sealed 31.0
- 197
Qwen3.5-0.8B NCRI 15.2; no filler; sealed 24.9
- 198
Gemma 2 2B IT NCRI 15.2; no filler; sealed 24.1
- 199
Gemma 3 1B NCRI 15.2; no filler; sealed 16.1
- 200
LLaMA-7B NCRI 15.2; no filler; sealed 15.0
- 201
Gemma 3 270M NCRI 15.2; no filler; sealed -23.1
Source: Official nocot-bench (NCRI) leaderboard
- 173.9 GPT-6 Astra
- 171.0 GPT-6.2 Astra
- 156.6 GPT-6.1 Sol
- 154.1
Gemini 4 Argon
- 149.6 GPT-6.1 Terra
- 129.7
Gemini 3.8 Flash
- 129.6 Claude Fable 5.1
- 127.6
Gemini 3.7 Flash
- 123.9
Gemini 3.1 Pro
- 122.2 GPT-6 Sol
- 121.5 Claude Fable 5
- 119.5 GPT-5.6 Sol
- 118.9 Claude Opus 5.5
- 115.8 Claude Opus 5
- 111.0 GPT-5.5
- 110.9
Gemini 3.6 Flash
- 110.8 Claude Opus 4.6
- 110.5
Gemini 3 Flash
- 110.5 Claude Opus 4.5
- 109.0 Claude Opus 4
- 108.8
Muse Spark 1.1
- 107.5 Claude Opus 4.1
- 107.5
Gemini 3.5 Flash
- 105.6 Kimi K3
- 103.9 Claude Opus 4.7
- 102.8 Kimi K3.1 Pro
- 102.3
Muse Spark 1.2
- 102.2 Claude Opus 4.8
- 100.8
Muse Spark 1.3
- 100.8 Claude Sonnet 4.5
- 97.7
Muse Wonder
- 96.5 Claude Sonnet 4
- 95.7
Muse Spark Open
- 95.1 GPT-5.6 Terra
- 95.1 Claude Sonnet 5.5
- 94.1 GPT-5.4
- 94.0
Tencent Hy4
- 91.1 Claude Sonnet 5
- 90.0 Claude Sonnet 4.6
- 87.0
Qwen 3.6 Max (Preview)
- 86.8 Grok 4.6
- 86.5
Gemini 3.1 Flash-Lite
- 86.0
Qwen3.8-2.4T-A95B
- 85.8 GPT-5.2
- 85.8
Qwen3.7-Max
- 85.4
Qwen 3.8 Max
- 85.2 GPT-5.3 Codex
- 84.1
DeepSeek V4 Pro 0813
- 84.1
DeepSeek-V4-Pro
- 83.3
Qwen3-Max
- 83.1
Qwen3.7-Plus
- 82.8
Tencent Hy4 preview
- 82.5
LongCat-2.0
- 82.2
Qwen 3.6 Plus
- 81.7
Gemma 4 31B IT
- 81.6
DeepSeek V4.1 Flash 1017
- 81.3
DeepSeek V4.1 Flash
- 81.0 Grok 4.5
- 80.6
Gemini 3.5 Flash-Lite
- 80.6
Qwen3.5-Plus
- 80.6 Kimi K2.5
- 80.0
Qwen3.5 397B-A17B
- 79.2 Kimi K2.6
- 79.1
DeepSeek-V4-Flash-Base
- 78.3
Inkling
- 78.1
DeepSeek-V4-Flash
- 77.6
DeepSeek-V3.1-Terminus
- 77.3 GLM-5.2
- 77.3 GPT-6 Luna
- 77.3
Qwen3-VL-235B-A22B-Instruct
- 77.1 MiMo-V2.5-Pro
- 76.7
DeepSeek-V3.2-Exp
- 76.6 GPT-5.6 Luna
- 76.4 GLM-5.3-Flash
- 76.2 GLM-5
- 75.5
DeepSeek V4 Flash 0731
- 75.2 GLM-5.1
- 75.1 GPT-4.1
- 75.0
DeepSeek-V3.2
- 74.6
Qwen3-235B-A22B-Instruct
- 74.5 GLM-5.3
- 74.4
Qwen3.8 Flash
- 74.2 GPT-4o
- 74.2 Claude Haiku 4.5
- 73.9
Ling-2.6-1T
- 73.2
WizardLM-2 8x22B
- 73.1
Qwen3-Coder-480B-A35B
- 72.8 GPT-4
- 72.6
MiniMax-M3
- 72.5
Tencent Hy3
- 72.3 Mercury 3
- 72.3
Tencent Hy3 preview
- 72.2 GPT-4o
- 71.3 GLM-4.7
- 71.3 GLM-4.5
- 71.2 Qwen3-Coder-Plus
- 70.8 GLM-4.6
- 70.6 GPT-5.1
- 70.5
Qwen3-235B-A22B
- 70.5
Llama 4 Maverick
- 70.5 GPT-5.4 Mini
- 70.3 GPT-4o
- 70.2
Inkling-Small
- 70.2
DiffusionGemma 26B A4B
- 69.9 MiMo-V2.5
- 69.5
Gemma 4 26B A4B
- 69.4
Qwen3.5-122B-A10B
- 69.2 Grok 4.3
- 68.9
Qwen3-235B-A22B-Thinking
- 68.4
Qwen3-Next-80B-A3B-Instruct
- 68.4
MiniMax-M3.1-Flash
- 68.3 Claude Haiku 5.5
- 67.9
Muse Glimmer
- 67.5
Mistral Medium 3.5
- 67.4
Qwen3.6 27B
- 67.1 Grok 4.20
- 67.0 GPT-5
- 66.7 GLM-4.5V
- 66.6
Qwen3.5-27B
- 66.3
Ling-3.0-flash
- 66.1
Qwen 3.8 27B
- 65.8 Mercury 2.5
- 65.5
Qwen3-Coder-Next
- 65.3
Solar Pro 4
- 65.2
Qwen2.5-VL-72B
- 65.0 GPT-4.1 mini
- 65.0
MiniMax-M2.5
- 64.5
Qwen3.5-35B-A3B-Base
- 64.0
MiniMax-M2.7
- 64.0
Qwen3-VL-32B-Instruct
- 63.8
Qwen3-Next-80B-A3B
- 63.5
Qwen2.5-72B
- 63.4 GLM-4.6V
- 63.3
Qwen 3.6 Flash
- 63.3
Qwen3.7 Flash
- 62.8 Hermes 4 405B
- 62.6
Llama 3.3 70B Instruct
- 62.5
Laguna S 2.1
- 62.4 GLM-4.5-Air
- 62.1
Qwen3-30B-A3B-Instruct
- 61.6
Qwen 3.6 35B-A3B
- 61.4
Qwen3.5-Flash
- 61.1
Qwen2.5-32B
- 61.0
Qwen2-72B
- 60.1
Qwen3.5-35B-A3B
- 60.0
Qwen2.5-Coder-32B-Instruct
- 59.9
Qwen3-VL-30B-A3B-Instruct
- 59.7
Gemma 4 12B
- 59.6
Qwen3-Coder-30B-A3B-Instruct
- 59.5
Qwen3-Coder-Flash
- 59.3 Ripple-65B Preview
- 59.2
Qwen3-32B
- 59.0 gpt-oss-120b
- 58.3
Phi-4
- 57.7
Llama 4 Scout
- 57.6
Llama 3.1-70B
- 57.5
Laguna XS 2.1
- 57.5
Gemma 3 27B PT
- 57.3 Claude 3 Haiku
- 56.1
Nemotron 3.5 Lightning 30B-A3B
- 56.0
Qwen3-VL-8B-Instruct
- 56.0
Gemma 3 27B
- 55.9 glm-4.7-flash
- 55.7 GPT-4o mini
- 55.1 Mercury 2
- 55.0
Qwen3-14B
- 54.8 gpt-oss-20b
- 54.3
Olmo 3.1 32B Instruct
- 54.2
Qwen3.5-9B-Base
- 53.3
Gemma 2 27B
- 52.4
Qwen3.5-9B
- 51.4
Qwen3-8B
- 50.7 GPT-5 mini
- 50.6
Gemma 4 E4B
- 50.4
Qwen3-30B-A3B
- 50.2 GPT-4.1 nano
- 49.9
Gemma 3 12B PT
- 49.7
Qwen1.5-72B
- 49.6 GPT-3.5 Turbo Instruct
- 49.4 GPT-5.4 Nano
- 49.3
Gemma 3 12B
- 48.7
Hunyuan-A13B-Instruct
- 47.3
Nemotron 3 Nano 30B-A3B
- 46.0
Qwen3.5-4B-Base
- 44.7
Gemma 2 9B
- 44.3
Qwen3.5-4B
- 44.0
Gemma 4 E2B
- 43.9
UI-TARS-1.5
- 43.2
Mistral-Nemo-Base-2407
- 42.5
Mistral NeMo
- 40.6
Llama 3.1-8B
- 38.5
Gemma 3n E4B
- 37.9
Gemma 3 4B
- 37.5
LLaMA-65B
- 35.5
Qwen3.5-2B
- 32.6 davinci-002
- 31.0 GPT-5 nano
- 24.9
Qwen3.5-0.8B
- 24.1
Gemma 2 2B IT
- 16.1
Gemma 3 1B
- 15.0
LLaMA-7B
- -23.1
Gemma 3 270M
- 1 GPT-6 Astra NCRI 15.2; dots@2048; placed 173.9
- 2 GPT-6.2 Astra 171.0
- 3 GPT-6.1 Sol NCRI 15.2; dots@128; placed_after_release 156.6
- 4
Gemini 4 Argon NCRI 15.2; qrep@512; placed_after_release 154.1
- 5 GPT-6.1 Terra NCRI 15.2; qrep@512; placed_after_release 149.6
- 6
Gemini 3.8 Flash NCRI 15.2; dots@128; placed 129.7
- 7 Claude Fable 5.1 NCRI 15.2; dots@128; placed 129.6
- 8
Gemini 3.7 Flash NCRI 15.2; dots@32; sealed 127.6
- 9
Gemini 3.1 Pro NCRI 15.2; count@512; sealed 123.9
- 10 GPT-6 Sol NCRI 15.2; dots@128; placed_after_release 122.2
- 11 Claude Fable 5 NCRI 15.2; dots@32; sealed 121.5
- 12 GPT-5.6 Sol NCRI 15.2; qrep@512; sealed 119.5
- 13 Claude Opus 5.5 NCRI 15.2; dots@512; placed_after_release 118.9
- 14 Claude Opus 5 NCRI 15.2; dots@128; sealed 115.8
- 15 GPT-5.5 NCRI 15.2; qrep@512; sealed 111.0
- 16
Gemini 3.6 Flash NCRI 15.2; count@512; sealed 110.9
- 17 Claude Opus 4.6 NCRI 15.2; qrep@512; sealed 110.8
- 18
Gemini 3 Flash NCRI 15.2; count@512; sealed 110.5
- 19 Claude Opus 4.5 NCRI 15.2; dots@512; sealed 110.5
- 20 Claude Opus 4 NCRI 15.2; dots@512; sealed 109.0
- 21
Muse Spark 1.1 NCRI 15.2; qrep@512; sealed 108.8
- 22 Claude Opus 4.1 NCRI 15.2; dots@128; sealed 107.5
- 23
Gemini 3.5 Flash NCRI 15.2; dots@512; sealed 107.5
- 24 Kimi K3 NCRI 15.2; qrep@512; sealed 105.6
- 25 Claude Opus 4.7 NCRI 15.2; dots@128; sealed 103.9
- 26 Kimi K3.1 Pro NCRI 15.2; no filler; placed_after_release 102.8
- 27
Muse Spark 1.2 NCRI 15.2; dots@128; sealed 102.3
- 28 Claude Opus 4.8 NCRI 15.2; dots@128; sealed 102.2
- 29
Muse Spark 1.3 NCRI 15.2; dots@2048; placed 100.8
- 30 Claude Sonnet 4.5 NCRI 15.2; qrep@512; sealed 100.8
- 31
Muse Wonder NCRI 15.2; no filler; placed_after_release 97.7
- 32 Claude Sonnet 4 NCRI 15.2; count@512; sealed 96.5
- 33
Muse Spark Open NCRI 15.2; no filler; placed_after_release 95.7
- 34 GPT-5.6 Terra NCRI 15.2; qrep@512; sealed 95.1
- 35 Claude Sonnet 5.5 NCRI 15.2; no filler; placed_after_release 95.1
- 36 GPT-5.4 NCRI 15.2; dots@128; sealed 94.1
- 37
Tencent Hy4 NCRI 15.2; no filler; placed_after_release 94.0
- 38 Claude Sonnet 5 NCRI 15.2; dots@128; sealed 91.1
- 39 Claude Sonnet 4.6 NCRI 15.2; dots@512; sealed 90.0
- 40
Qwen 3.6 Max (Preview) NCRI 15.2; no filler; sealed 87.0
- 41 Grok 4.6 NCRI 15.2; no filler; sealed 86.8
- 42
Gemini 3.1 Flash-Lite NCRI 15.2; no filler; sealed 86.5
- 43
Qwen3.8-2.4T-A95B NCRI 15.2; no filler; sealed 86.0
- 44 GPT-5.2 NCRI 15.2; no filler; sealed 85.8
- 45
Qwen3.7-Max NCRI 15.2; no filler; sealed 85.8
- 46
Qwen 3.8 Max NCRI 15.2; no filler; sealed 85.4
- 47 GPT-5.3 Codex NCRI 15.2; no filler; sealed 85.2
- 48
DeepSeek V4 Pro 0813 NCRI 15.2; no filler; sealed 84.1
- 49
DeepSeek-V4-Pro NCRI 15.2; no filler; sealed 84.1
- 50
Qwen3-Max NCRI 15.2; no filler; sealed 83.3
- 51
Qwen3.7-Plus NCRI 15.2; no filler; sealed 83.1
- 52
Tencent Hy4 preview NCRI 15.2; no filler; sealed 82.8
- 53
LongCat-2.0 NCRI 15.2; no filler; sealed 82.5
- 54
Qwen 3.6 Plus NCRI 15.2; no filler; sealed 82.2
- 55
Gemma 4 31B IT NCRI 15.2; no filler; sealed 81.7
- 56
DeepSeek V4.1 Flash 1017 NCRI 15.2; no filler; placed_after_release 81.6
- 57
DeepSeek V4.1 Flash NCRI 15.2; no filler; placed_after_release 81.3
- 58 Grok 4.5 NCRI 15.2; no filler; sealed 81.0
- 59
Gemini 3.5 Flash-Lite NCRI 15.2; no filler; sealed 80.6
- 60
Qwen3.5-Plus NCRI 15.2; no filler; sealed 80.6
- 61 Kimi K2.5 NCRI 15.2; no filler; sealed 80.6
- 62
Qwen3.5 397B-A17B NCRI 15.2; no filler; sealed 80.0
- 63 Kimi K2.6 NCRI 15.2; no filler; sealed 79.2
- 64
DeepSeek-V4-Flash-Base NCRI 15.2; no filler; sealed 79.1
- 65
Inkling NCRI 15.2; no filler; sealed 78.3
- 66
DeepSeek-V4-Flash NCRI 15.2; no filler; sealed 78.1
- 67
DeepSeek-V3.1-Terminus NCRI 15.2; no filler; sealed 77.6
- 68 GLM-5.2 NCRI 15.2; no filler; sealed 77.3
- 69 GPT-6 Luna NCRI 15.2; no filler; placed_after_release 77.3
- 70
Qwen3-VL-235B-A22B-Instruct NCRI 15.2; no filler; sealed 77.3
- 71 MiMo-V2.5-Pro NCRI 15.2; no filler; sealed 77.1
- 72
DeepSeek-V3.2-Exp NCRI 15.2; no filler; sealed 76.7
- 73 GPT-5.6 Luna NCRI 15.2; no filler; sealed 76.6
- 74 GLM-5.3-Flash NCRI 15.2; no filler; sealed 76.4
- 75 GLM-5 NCRI 15.2; no filler; sealed 76.2
- 76
DeepSeek V4 Flash 0731 NCRI 15.2; no filler; sealed 75.5
- 77 GLM-5.1 NCRI 15.2; no filler; sealed 75.2
- 78 GPT-4.1 NCRI 15.2; no filler; sealed 75.1
- 79
DeepSeek-V3.2 NCRI 15.2; no filler; sealed 75.0
- 80
Qwen3-235B-A22B-Instruct Jul 2025, NCRI 15.2; no filler; sealed 74.6
- 81 GLM-5.3 NCRI 15.2; no filler; sealed 74.5
- 82
Qwen3.8 Flash NCRI 15.2; no filler; sealed 74.4
- 83 GPT-4o Aug 2024, NCRI 15.2; no filler; sealed 74.2
- 84 Claude Haiku 4.5 NCRI 15.2; no filler; sealed 74.2
- 85
Ling-2.6-1T NCRI 15.2; no filler; sealed 73.9
- 86
WizardLM-2 8x22B NCRI 15.2; no filler; sealed 73.2
- 87
Qwen3-Coder-480B-A35B NCRI 15.2; no filler; sealed 73.1
- 88 GPT-4 Jun 2023, NCRI 15.2; no filler; sealed 72.8
- 89
MiniMax-M3 NCRI 15.2; no filler; sealed 72.6
- 90
Tencent Hy3 NCRI 15.2; no filler; sealed 72.5
- 91 Mercury 3 NCRI 15.2; no filler; placed_after_release 72.3
- 92
Tencent Hy3 preview NCRI 15.2; no filler; sealed 72.3
- 93 GPT-4o May 2024, NCRI 15.2; no filler; sealed 72.2
- 94 GLM-4.7 NCRI 15.2; no filler; sealed 71.3
- 95 GLM-4.5 NCRI 15.2; no filler; sealed 71.3
- 96 Qwen3-Coder-Plus NCRI 15.2; no filler; sealed 71.2
- 97 GLM-4.6 NCRI 15.2; no filler; sealed 70.8
- 98 GPT-5.1 NCRI 15.2; no filler; sealed 70.6
- 99
Qwen3-235B-A22B NCRI 15.2; no filler; sealed 70.5
- 100
Llama 4 Maverick NCRI 15.2; no filler; sealed 70.5
- 101 GPT-5.4 Mini NCRI 15.2; no filler; sealed 70.5
- 102 GPT-4o Nov 2024, NCRI 15.2; no filler; sealed 70.3
- 103
Inkling-Small NCRI 15.2; no filler; sealed 70.2
- 104
DiffusionGemma 26B A4B NCRI 15.2; no filler; sealed 70.2
- 105 MiMo-V2.5 NCRI 15.2; no filler; sealed 69.9
- 106
Gemma 4 26B A4B NCRI 15.2; no filler; sealed 69.5
- 107
Qwen3.5-122B-A10B NCRI 15.2; no filler; sealed 69.4
- 108 Grok 4.3 NCRI 15.2; no filler; sealed 69.2
- 109
Qwen3-235B-A22B-Thinking Jul 2025, NCRI 15.2; no filler; sealed 68.9
- 110
Qwen3-Next-80B-A3B-Instruct NCRI 15.2; no filler; sealed 68.4
- 111
MiniMax-M3.1-Flash NCRI 15.2; no filler; placed_after_release 68.4
- 112 Claude Haiku 5.5 NCRI 15.2; no filler; placed_after_release 68.3
- 113
Muse Glimmer NCRI 15.2; no filler; sealed 67.9
- 114
Mistral Medium 3.5 NCRI 15.2; no filler; sealed 67.5
- 115
Qwen3.6 27B NCRI 15.2; no filler; sealed 67.4
- 116 Grok 4.20 NCRI 15.2; no filler; sealed 67.1
- 117 GPT-5 NCRI 15.2; no filler; sealed 67.0
- 118 GLM-4.5V NCRI 15.2; no filler; sealed 66.7
- 119
Qwen3.5-27B NCRI 15.2; no filler; sealed 66.6
- 120
Ling-3.0-flash NCRI 15.2; no filler; sealed 66.3
- 121
Qwen 3.8 27B NCRI 15.2; no filler; sealed 66.1
- 122 Mercury 2.5 NCRI 15.2; no filler; placed 65.8
- 123
Qwen3-Coder-Next NCRI 15.2; no filler; sealed 65.5
- 124
Solar Pro 4 NCRI 15.2; no filler; sealed 65.3
- 125
Qwen2.5-VL-72B NCRI 15.2; no filler; sealed 65.2
- 126 GPT-4.1 mini NCRI 15.2; no filler; sealed 65.0
- 127
MiniMax-M2.5 NCRI 15.2; no filler; sealed 65.0
- 128
Qwen3.5-35B-A3B-Base NCRI 15.2; no filler; sealed 64.5
- 129
MiniMax-M2.7 NCRI 15.2; no filler; sealed 64.0
- 130
Qwen3-VL-32B-Instruct NCRI 15.2; no filler; sealed 64.0
- 131
Qwen3-Next-80B-A3B NCRI 15.2; no filler; sealed 63.8
- 132
Qwen2.5-72B NCRI 15.2; no filler; sealed 63.5
- 133 GLM-4.6V NCRI 15.2; no filler; sealed 63.4
- 134
Qwen 3.6 Flash NCRI 15.2; no filler; sealed 63.3
- 135
Qwen3.7 Flash NCRI 15.2; no filler; sealed 63.3
- 136 Hermes 4 405B NCRI 15.2; no filler; sealed 62.8
- 137
Llama 3.3 70B Instruct NCRI 15.2; no filler; sealed 62.6
- 138
Laguna S 2.1 NCRI 15.2; no filler; sealed 62.5
- 139 GLM-4.5-Air NCRI 15.2; no filler; sealed 62.4
- 140
Qwen3-30B-A3B-Instruct Jul 2025, NCRI 15.2; no filler; sealed 62.1
- 141
Qwen 3.6 35B-A3B NCRI 15.2; no filler; sealed 61.6
- 142
Qwen3.5-Flash NCRI 15.2; no filler; sealed 61.4
- 143
Qwen2.5-32B NCRI 15.2; no filler; sealed 61.1
- 144
Qwen2-72B NCRI 15.2; no filler; sealed 61.0
- 145
Qwen3.5-35B-A3B NCRI 15.2; no filler; sealed 60.1
- 146
Qwen2.5-Coder-32B-Instruct NCRI 15.2; no filler; sealed 60.0
- 147
Qwen3-VL-30B-A3B-Instruct NCRI 15.2; no filler; sealed 59.9
- 148
Gemma 4 12B NCRI 15.2; no filler; sealed 59.7
- 149
Qwen3-Coder-30B-A3B-Instruct NCRI 15.2; no filler; sealed 59.6
- 150
Qwen3-Coder-Flash NCRI 15.2; no filler; sealed 59.5
- 151 Ripple-65B Preview NCRI 15.2; no filler; placed_after_release 59.3
- 152
Qwen3-32B NCRI 15.2; no filler; sealed 59.2
- 153 gpt-oss-120b NCRI 15.2; no filler; sealed 59.0
- 154
Phi-4 NCRI 15.2; no filler; sealed 58.3
- 155
Llama 4 Scout NCRI 15.2; no filler; sealed 57.7
- 156
Llama 3.1-70B NCRI 15.2; no filler; sealed 57.6
- 157
Laguna XS 2.1 NCRI 15.2; no filler; sealed 57.5
- 158
Gemma 3 27B PT NCRI 15.2; no filler; sealed 57.5
- 159 Claude 3 Haiku NCRI 15.2; no filler; sealed 57.3
- 160
Nemotron 3.5 Lightning 30B-A3B NCRI 15.2; no filler; sealed 56.1
- 161
Qwen3-VL-8B-Instruct NCRI 15.2; no filler; sealed 56.0
- 162
Gemma 3 27B NCRI 15.2; no filler; sealed 56.0
- 163 glm-4.7-flash NCRI 15.2; no filler; sealed 55.9
- 164 GPT-4o mini NCRI 15.2; no filler; sealed 55.7
- 165 Mercury 2 NCRI 15.2; no filler; sealed 55.1
- 166
Qwen3-14B NCRI 15.2; no filler; sealed 55.0
- 167 gpt-oss-20b NCRI 15.2; no filler; sealed 54.8
- 168
Olmo 3.1 32B Instruct NCRI 15.2; no filler; placed 54.3
- 169
Qwen3.5-9B-Base NCRI 15.2; no filler; sealed 54.2
- 170
Gemma 2 27B NCRI 15.2; no filler; sealed 53.3
- 171
Qwen3.5-9B NCRI 15.2; no filler; sealed 52.4
- 172
Qwen3-8B NCRI 15.2; no filler; sealed 51.4
- 173 GPT-5 mini NCRI 15.2; no filler; sealed 50.7
- 174
Gemma 4 E4B NCRI 15.2; no filler; sealed 50.6
- 175
Qwen3-30B-A3B NCRI 15.2; no filler; sealed 50.4
- 176 GPT-4.1 nano NCRI 15.2; no filler; sealed 50.2
- 177
Gemma 3 12B PT NCRI 15.2; no filler; sealed 49.9
- 178
Qwen1.5-72B NCRI 15.2; no filler; sealed 49.7
- 179 GPT-3.5 Turbo Instruct NCRI 15.2; no filler; sealed 49.6
- 180 GPT-5.4 Nano NCRI 15.2; no filler; sealed 49.4
- 181
Gemma 3 12B NCRI 15.2; no filler; sealed 49.3
- 182
Hunyuan-A13B-Instruct NCRI 15.2; no filler; sealed 48.7
- 183
Nemotron 3 Nano 30B-A3B NCRI 15.2; no filler; sealed 47.3
- 184
Qwen3.5-4B-Base NCRI 15.2; no filler; sealed 46.0
- 185
Gemma 2 9B NCRI 15.2; no filler; sealed 44.7
- 186
Qwen3.5-4B NCRI 15.2; no filler; sealed 44.3
- 187
Gemma 4 E2B NCRI 15.2; no filler; sealed 44.0
- 188
UI-TARS-1.5 NCRI 15.2; no filler; sealed 43.9
- 189
Mistral-Nemo-Base-2407 NCRI 15.2; no filler; sealed 43.2
- 190
Mistral NeMo NCRI 15.2; no filler; sealed 42.5
- 191
Llama 3.1-8B NCRI 15.2; no filler; sealed 40.6
- 192
Gemma 3n E4B NCRI 15.2; no filler; sealed 38.5
- 193
Gemma 3 4B NCRI 15.2; no filler; sealed 37.9
- 194
LLaMA-65B NCRI 15.2; no filler; sealed 37.5
- 195
Qwen3.5-2B NCRI 15.2; no filler; sealed 35.5
- 196 davinci-002 NCRI 15.2; no filler; sealed 32.6
- 197 GPT-5 nano NCRI 15.2; no filler; sealed 31.0
- 198
Qwen3.5-0.8B NCRI 15.2; no filler; sealed 24.9
- 199
Gemma 2 2B IT NCRI 15.2; no filler; sealed 24.1
- 200
Gemma 3 1B NCRI 15.2; no filler; sealed 16.1
- 201
LLaMA-7B NCRI 15.2; no filler; sealed 15.0
- 202
Gemma 3 270M NCRI 15.2; no filler; sealed -23.1
Sources: Official nocot-bench (NCRI) leaderboard and labs’ model cards
| # | Entry | Score ± 95% CI | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 173.9 | NCRI 15.2; dots@2048; placed | Sep 2026 | Official leaderboard |
| 2 | GPT-6.1 Sol | 156.6 | NCRI 15.2; dots@128; placed_after_release | Sep 2026 | Official leaderboard |
| 3 | | 154.1 | NCRI 15.2; qrep@512; placed_after_release | Sep 2026 | Official leaderboard |
| 4 | GPT-6.1 Terra | 149.6 | NCRI 15.2; qrep@512; placed_after_release | Oct 2026 | Official leaderboard |
| 5 | | 129.7 | NCRI 15.2; dots@128; placed | Sep 2026 | Official leaderboard |
| 6 | Claude Fable 5.1 | 129.6 | NCRI 15.2; dots@128; placed | Sep 2026 | Official leaderboard |
| 7 | | 127.6 | NCRI 15.2; dots@32; sealed | Aug 2026 | Official leaderboard |
| 8 | | 123.9 | NCRI 15.2; count@512; sealed | Feb 2026 | Official leaderboard |
| 9 | GPT-6 Sol | 122.2 | NCRI 15.2; dots@128; placed_after_release | Sep 2026 | Official leaderboard |
| 10 | Claude Fable 5 | 121.5 | NCRI 15.2; dots@32; sealed | Jun 2026 | Official leaderboard |
| 11 | GPT-5.6 Sol | 119.5 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard |
| 12 | Claude Opus 5.5 | 118.9 116.8 to 121.1 | NCRI 15.2; dots@512; placed_after_release | Sep 2026 | Official leaderboard |
| 13 | Claude Opus 5 | 115.8 | NCRI 15.2; dots@128; sealed | Jul 2026 | Official leaderboard |
| 14 | GPT-5.5 | 111.0 | NCRI 15.2; qrep@512; sealed | Apr 2026 | Official leaderboard |
| 15 | | 110.9 | NCRI 15.2; count@512; sealed | Jul 2026 | Official leaderboard |
| 16 | Claude Opus 4.6 | 110.8 | NCRI 15.2; qrep@512; sealed | Feb 2026 | Official leaderboard |
| 17 | | 110.5 | NCRI 15.2; count@512; sealed | Dec 2025 | Official leaderboard |
| 18 | Claude Opus 4.5 | 110.5 | NCRI 15.2; dots@512; sealed | Nov 2025 | Official leaderboard |
| 19 | Claude Opus 4 | 109.0 | NCRI 15.2; dots@512; sealed | May 2025 | Official leaderboard |
| 20 | | 108.8 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard |
| 21 | Claude Opus 4.1 | 107.5 | NCRI 15.2; dots@128; sealed | Aug 2025 | Official leaderboard |
| 22 | | 107.5 | NCRI 15.2; dots@512; sealed | May 2026 | Official leaderboard |
| 23 | Kimi K3 | 105.6 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard |
| 24 | Claude Opus 4.7 | 103.9 | NCRI 15.2; dots@128; sealed | Apr 2026 | Official leaderboard |
| 25 | google/gemini-2.5-pro (snapshot unspecified) | 102.9 | NCRI 15.2; dots@512; sealed | Official leaderboard | |
| 26 | Kimi K3.1 Pro | 102.8 100.8 to 105 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 27 | | 102.3 | NCRI 15.2; dots@128; sealed | Aug 2026 | Official leaderboard |
| 28 | Claude Opus 4.8 | 102.2 | NCRI 15.2; dots@128; sealed | May 2026 | Official leaderboard |
| 29 | | 100.8 | NCRI 15.2; dots@2048; placed | Sep 2026 | Official leaderboard |
| 30 | Claude Sonnet 4.5 | 100.8 | NCRI 15.2; qrep@512; sealed | Sep 2025 | Official leaderboard |
| 31 | | 97.7 95.8 to 99.8 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 32 | Claude Sonnet 4 | 96.5 | NCRI 15.2; count@512; sealed | May 2025 | Official leaderboard |
| 33 | | 95.7 ±2.1 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 34 | GPT-5.6 Terra | 95.1 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard |
| 35 | Claude Sonnet 5.5 | 95.1 92.6 to 97.7 | NCRI 15.2; no filler; placed_after_release | Sep 2026 | Official leaderboard |
| 36 | GPT-5.4 | 94.1 | NCRI 15.2; dots@128; sealed | Mar 2026 | Official leaderboard |
| 37 | | 94.0 ±2.1 | NCRI 15.2; no filler; placed_after_release | Nov 2026 | Official leaderboard |
| 38 | Claude Sonnet 5 | 91.1 | NCRI 15.2; dots@128; sealed | Jun 2026 | Official leaderboard |
| 39 | Claude Sonnet 4.6 | 90.0 | NCRI 15.2; dots@512; sealed | Feb 2026 | Official leaderboard |
| 40 | | 87.0 84.5 to 89 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 41 | Grok 4.6 | 86.8 84.1 to 88.9 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 42 | | 86.5 83.8 to 88.7 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 43 | | 86.0 83.3 to 88.1 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 44 | GPT-5.2 | 85.8 83.1 to 88 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 45 | | 85.8 83.2 to 88.1 | NCRI 15.2; no filler; sealed | May 2026 | Official leaderboard |
| 46 | | 85.4 82.6 to 87.4 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 47 | GPT-5.3 Codex | 85.2 82.5 to 87.3 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 48 | | 84.1 81.2 to 86.1 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 49 | | 84.1 81.3 to 86.1 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 50 | | 83.3 80.7 to 85.5 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 51 | | 83.1 80.2 to 85.2 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard |
| 52 | | 82.8 79.9 to 85 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 53 | | 82.5 79.9 to 84.6 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 54 | | 82.2 79.3 to 84.3 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 55 | | 81.7 78.6 to 83.9 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 56 | | 81.6 ±2.2 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 57 | | 81.3 79 to 83.5 | NCRI 15.2; no filler; placed_after_release | Sep 2026 | Official leaderboard |
| 58 | Grok 4.5 | 81.0 78.1 to 83.3 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 59 | kwaipilot/kat-coder-pro-v2 | 80.8 78 to 82.8 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 60 | | 80.6 77.9 to 82.7 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 61 | | 80.6 77.8 to 82.5 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 62 | Kimi K2.5 | 80.6 77.7 to 82.8 | NCRI 15.2; no filler; sealed | Jan 2026 | Official leaderboard |
| 63 | | 80.0 77.1 to 81.9 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 64 | Kimi K2.6 | 79.2 76.3 to 81.3 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 65 | | 79.1 76.1 to 81.2 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 66 | google/gemini-2.5-flash (snapshot unspecified) | 78.8 76 to 80.7 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 67 | nvidia/nemotron-3-ultra-550b-a55b | 78.3 75.4 to 80.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 68 | | 78.3 75.4 to 80.2 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 69 | | 78.1 75.2 to 80.1 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 70 | moonshotai/kimi-k2 (snapshot unspecified) | 78.0 75 to 80.1 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 71 | | 77.6 74.7 to 79.9 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 72 | GLM-5.2 | 77.3 74 to 79.6 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard |
| 73 | GPT-6 Luna | 77.3 74.7 to 79.1 | NCRI 15.2; no filler; placed_after_release | Sep 2026 | Official leaderboard |
| 74 | | 77.3 74.3 to 79.3 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 75 | MiMo-V2.5-Pro | 77.1 74.2 to 79.2 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 76 | | 76.7 73.8 to 79 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 77 | GPT-5.6 Luna | 76.6 73.5 to 78.7 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 78 | deepseek/deepseek-chat-v3.1 | 76.5 73.5 to 78.8 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 79 | GLM-5.3-Flash | 76.4 73.2 to 78.6 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 80 | GLM-5 | 76.2 73.2 to 78.3 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 81 | | 75.5 72.6 to 77.5 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 82 | GLM-5.1 | 75.2 71.8 to 77.7 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 83 | GPT-4.1 | 75.1 72 to 77.4 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 84 | | 75.0 72 to 77.2 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 85 | | 74.6 71.4 to 76.7 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 86 | GLM-5.3 | 74.5 71.5 to 76.8 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 87 | | 74.4 71.3 to 76.8 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 88 | deepseek/deepseek-chat-v3-0324 | 74.4 71.5 to 76.5 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 89 | openai/gpt-4o (snapshot unspecified) | 74.2 71.3 to 76.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 90 | GPT-4o Aug 2024 | 74.2 71.2 to 76.4 | NCRI 15.2; no filler; sealed | Aug 2024 | Official leaderboard |
| 91 | Claude Haiku 4.5 | 74.2 71.1 to 76.4 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard |
| 92 | | 73.9 71 to 76.3 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 93 | openai/gpt-4 (snapshot unspecified) | 73.7 70.6 to 75.8 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 94 | | 73.2 70.2 to 75.3 | NCRI 15.2; no filler; sealed | Apr 2024 | Official leaderboard |
| 95 | | 73.1 69.7 to 75.5 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 96 | deepseek/deepseek-chat | 72.9 70.1 to 75.1 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 97 | GPT-4 Jun 2023 | 72.8 69.7 to 75 | NCRI 15.2; no filler; sealed | Jun 2023 | Official leaderboard |
| 98 | | 72.6 69.2 to 74.7 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard |
| 99 | | 72.5 69.3 to 74.7 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 100 | Mercury 3 | 72.3 69.9 to 74.5 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 101 | | 72.3 69 to 74.5 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 102 | GPT-4o May 2024 | 72.2 69.1 to 74.3 | NCRI 15.2; no filler; sealed | May 2024 | Official leaderboard |
| 103 | GLM-4.7 | 71.3 68 to 73.5 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 104 | openai/gpt-4-turbo (snapshot unspecified) | 71.3 68.4 to 73.7 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 105 | GLM-4.5 | 71.3 68.1 to 73.3 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 106 | Qwen3-Coder-Plus | 71.2 68 to 73.3 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 107 | GLM-4.6 | 70.8 67.6 to 73.2 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 108 | GPT-5.1 | 70.6 67.4 to 72.9 | NCRI 15.2; no filler; sealed | Nov 2025 | Official leaderboard |
| 109 | | 70.5 67.3 to 72.9 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 110 | | 70.5 67.5 to 72.7 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 111 | GPT-5.4 Mini | 70.5 67.3 to 72.4 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 112 | GPT-4o Nov 2024 | 70.3 67.2 to 72.6 | NCRI 15.2; no filler; sealed | Nov 2024 | Official leaderboard |
| 113 | | 70.2 66.7 to 72.5 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 114 | | 70.2 66.7 to 72.8 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard |
| 115 | MiMo-V2.5 | 69.9 66.9 to 72.3 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 116 | | 69.5 66 to 71.9 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 117 | | 69.4 66.4 to 71.6 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 118 | Grok 4.3 | 69.2 66 to 71.2 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 119 | | 68.9 65.8 to 71 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 120 | | 68.4 65.2 to 70.5 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 121 | | 68.4 65.6 to 71.3 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 122 | Claude Haiku 5.5 | 68.3 65.6 to 70.9 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 123 | | 67.9 64.3 to 70.2 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 124 | | 67.5 64.1 to 69.9 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 125 | | 67.4 63.9 to 69.6 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 126 | Grok 4.20 | 67.1 64 to 69.2 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 127 | GPT-5 | 67.0 63.6 to 69.1 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 128 | baidu/ernie-4.5-vl-424b-a47b | 66.9 63.7 to 69 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 129 | GLM-4.5V | 66.7 63.3 to 69 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 130 | | 66.6 63 to 68.9 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 131 | | 66.3 63 to 68.7 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 132 | nvidia/nemotron-3-super-120b-a12b | 66.2 62.9 to 68.4 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 133 | | 66.1 62.7 to 68.6 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 134 | Mercury 2.5 | 65.8 62.4 to 68.1 | NCRI 15.2; no filler; placed | Sep 2026 | Official leaderboard |
| 135 | inclusionai/ling-2.6-flash | 65.5 62.4 to 67.9 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 136 | | 65.5 61.8 to 67.9 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 137 | | 65.3 62 to 67.7 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 138 | | 65.2 61.7 to 67.6 | NCRI 15.2; no filler; sealed | Jan 2025 | Official leaderboard |
| 139 | GPT-4.1 mini | 65.0 61.4 to 67.4 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 140 | | 65.0 61.6 to 67.1 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 141 | reflection/beam-1.0-501b-a23b | 64.8 62.1 to 67.6 | NCRI 15.2; no filler; placed_after_release | Official leaderboard | |
| 142 | nousresearch/hermes-3-llama-3.1-405b | 64.6 61.2 to 66.9 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 143 | | 64.5 61.1 to 66.9 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 144 | google/gemini-2.5-flash-lite (snapshot unspecified) | 64.5 61 to 66.8 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 145 | | 64.0 60.6 to 66.3 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 146 | | 64.0 60.5 to 66.7 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard |
| 147 | | 63.8 60.4 to 66.1 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 148 | sao10k/l3.3-euryale-70b | 63.8 60.2 to 66.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 149 | mistralai/mistral-small-2603 | 63.5 60.1 to 65.7 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 150 | | 63.5 60 to 66 | NCRI 15.2; no filler; sealed | Sep 2024 | Official leaderboard |
| 151 | GLM-4.6V | 63.4 59.7 to 65.9 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 152 | | 63.3 59.6 to 65.7 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 153 | | 63.3 59.7 to 65.5 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 154 | Hermes 4 405B | 62.8 59.2 to 65.2 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 155 | meta-llama/llama-3.1-70b-instruct | 62.8 59.3 to 65.2 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 156 | local/qwen2.5-32b-instruct | 62.7 59.1 to 65.2 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 157 | | 62.6 59 to 64.8 | NCRI 15.2; no filler; sealed | Dec 2024 | Official leaderboard |
| 158 | | 62.5 59.1 to 64.8 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 159 | GLM-4.5-Air | 62.4 58.8 to 64.8 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 160 | | 62.1 58.5 to 64.6 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 161 | | 61.6 58.3 to 64.1 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 162 | | 61.4 57.8 to 63.7 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 163 | amazon/nova-pro-v1 | 61.3 57.5 to 63.7 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 164 | | 61.1 57.6 to 63.5 | NCRI 15.2; no filler; sealed | Sep 2024 | Official leaderboard |
| 165 | sao10k/l3.1-euryale-70b | 61.0 57.2 to 63.4 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 166 | | 61.0 57.5 to 63.4 | NCRI 15.2; no filler; sealed | Jun 2024 | Official leaderboard |
| 167 | | 60.1 56.3 to 62.5 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 168 | | 60.0 56.1 to 62.5 | NCRI 15.2; no filler; sealed | Nov 2024 | Official leaderboard |
| 169 | | 59.9 56.1 to 62.6 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard |
| 170 | | 59.7 55.6 to 62.4 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard |
| 171 | | 59.6 55.9 to 62.3 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 172 | | 59.5 55.7 to 61.9 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 173 | Ripple-65B Preview | 59.3 55 to 62.4 | NCRI 15.2; no filler; placed_after_release | Nov 2026 | Official leaderboard |
| 174 | | 59.2 55.2 to 61.8 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 175 | thedrummer/cydonia-24b-v4.1 | 59.1 55.3 to 61.6 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 176 | gpt-oss-120b | 59.0 54.9 to 61.4 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 177 | mistralai/ministral-14b-2512 | 58.7 55 to 61.2 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 178 | | 58.3 54.6 to 61 | NCRI 15.2; no filler; sealed | Dec 2024 | Official leaderboard |
| 179 | nousresearch/hermes-4-70b | 57.9 54 to 60.4 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 180 | | 57.7 53.9 to 60.2 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 181 | | 57.6 54 to 60.1 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 182 | | 57.5 53.3 to 59.9 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 183 | | 57.5 53.7 to 59.9 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 184 | Claude 3 Haiku | 57.3 53.2 to 59.8 | NCRI 15.2; no filler; sealed | Mar 2024 | Official leaderboard |
| 185 | local/mistral-small-24b-base-2501 | 56.2 52.2 to 58.6 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 186 | | 56.1 52.2 to 58.4 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 187 | | 56.0 52.2 to 58.5 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard |
| 188 | | 56.0 52 to 58.4 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 189 | glm-4.7-flash | 55.9 51.7 to 58.4 | NCRI 15.2; no filler; sealed | Jan 2026 | Official leaderboard |
| 190 | GPT-4o mini | 55.7 51.6 to 58.2 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 191 | mistralai/mistral-small-24b-instruct-2501 | 55.4 51.6 to 57.8 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 192 | openai/gpt-3.5-turbo (snapshot unspecified) | 55.2 50.8 to 57.8 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 193 | Mercury 2 | 55.1 51.1 to 57.7 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 194 | | 55.0 51.2 to 57.7 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 195 | gpt-oss-20b | 54.8 50.3 to 57.8 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 196 | | 54.3 50.3 to 57 | NCRI 15.2; no filler; placed | Dec 2025 | Official leaderboard |
| 197 | | 54.2 50 to 56.7 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 198 | local/olmo-3-32b-dpo | 54.0 50.1 to 56.8 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 199 | mistralai/mistral-small-3.2-24b-instruct | 53.8 49.4 to 56.5 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 200 | local/olmo-3-32b-rl | 53.8 49.9 to 56.4 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 201 | local/olmo-3-32b-mid | 53.3 49.5 to 55.7 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 202 | | 53.3 49.2 to 55.7 | NCRI 15.2; no filler; sealed | Jun 2024 | Official leaderboard |
| 203 | local/olmo-3-32b-sft | 52.8 48.6 to 55.4 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 204 | google/gemma-2-27b-it | 52.6 48.5 to 55.5 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 205 | local/nanbeige42-r2 | 52.6 48.1 to 56.3 | NCRI 15.2; no filler; placed_after_release | Official leaderboard | |
| 206 | local/qwen1.5-110b-chat | 52.6 48.3 to 55.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 207 | | 52.4 47.8 to 55.3 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 208 | local/ouro26-r4 | 51.7 46.1 to 56.6 | NCRI 15.2; no filler; placed_after_release | Official leaderboard | |
| 209 | | 51.4 46.8 to 54.3 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 210 | mistralai/ministral-8b-2512 | 51.0 46.6 to 54 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 211 | GPT-5 mini | 50.7 45.7 to 53.6 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 212 | | 50.6 45.8 to 53.4 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 213 | | 50.4 45.3 to 53.5 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 214 | GPT-4.1 nano | 50.2 45.4 to 52.9 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 215 | | 49.9 45.4 to 52.4 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 216 | | 49.7 45.4 to 52.7 | NCRI 15.2; no filler; sealed | Feb 2024 | Official leaderboard |
| 217 | GPT-3.5 Turbo Instruct | 49.6 45.2 to 51.9 | NCRI 15.2; no filler; sealed | Sep 2023 | Official leaderboard |
| 218 | GPT-5.4 Nano | 49.4 45.1 to 51.9 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 219 | | 49.3 44.9 to 52.2 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 220 | | 48.7 44.2 to 51.8 | NCRI 15.2; no filler; sealed | Jun 2025 | Official leaderboard |
| 221 | qwen/qwen-2.5-7b-instruct | 48.4 43.7 to 51.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 222 | local/gemma-2-9b-it | 47.5 42.9 to 50.6 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 223 | | 47.3 42 to 50.2 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 224 | local/ouro14-r4 | 47.3 41.4 to 52.4 | NCRI 15.2; no filler; placed_after_release | Official leaderboard | |
| 225 | mistralai/ministral-3b-2512 | 46.2 41.2 to 49.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 226 | | 46.0 40.9 to 49.3 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 227 | local/olmo-3-32b-pretrain | 45.0 39.9 to 48 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 228 | | 44.7 39.6 to 47.9 | NCRI 15.2; no filler; sealed | Jun 2024 | Official leaderboard |
| 229 | | 44.3 39.2 to 48 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 230 | | 44.0 39.1 to 46.5 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 231 | | 43.9 38.9 to 47 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 232 | | 43.2 38.4 to 46.1 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 233 | thedrummer/unslopnemo-12b | 42.8 37.7 to 45.4 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 234 | | 42.5 37.4 to 45.4 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 235 | local/llama-3.1-8b-instruct-fp8 | 41.8 35.8 to 45.4 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 236 | meta-llama/llama-3.1-8b-instruct | 41.6 35.5 to 45 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 237 | local/llama-3.1-8b-instruct | 40.9 34.8 to 44.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 238 | local/olmo3-32b-step260000 | 40.8 35.9 to 43.9 | NCRI 15.2; no filler; placed | Official leaderboard | |
| 239 | local/llama-2-70b-hf | 40.7 35.3 to 43.6 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 240 | sao10k/l3-lunaris-8b | 40.7 35.2 to 44.1 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 241 | | 40.6 35.1 to 43.8 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 242 | local/llama-3-8b-instruct | 39.2 33.9 to 42.4 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 243 | | 38.5 32.2 to 42.1 | NCRI 15.2; no filler; sealed | Jun 2025 | Official leaderboard |
| 244 | local/olmo3-32b-step360000 | 38.5 33 to 41.8 | NCRI 15.2; no filler; placed | Official leaderboard | |
| 245 | | 37.9 32 to 41.3 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 246 | | 37.5 31.7 to 39.9 | NCRI 15.2; no filler; sealed | Feb 2023 | Official leaderboard |
| 247 | aion-labs/aion-rp-llama-3.1-8b | 37.2 31.7 to 40.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 248 | ibm-granite/granite-4.0-h-micro | 37.2 31.1 to 40.8 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 249 | local/olmo3-32b-step130000 | 36.2 30.5 to 39.5 | NCRI 15.2; no filler; placed | Official leaderboard | |
| 250 | | 35.5 29.1 to 38.7 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 251 | local/olmo3-32b-step92000 | 34.5 28.4 to 37.4 | NCRI 15.2; no filler; placed | Official leaderboard | |
| 252 | local/olmo3-32b-step182000 | 34.4 28.3 to 37.7 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 253 | davinci-002 | 32.6 26 to 35.9 | NCRI 15.2; no filler; sealed | Mar 2022 | Official leaderboard |
| 254 | local/olmo3-7b-step1413814 | 32.0 25.5 to 35.1 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 255 | local/olmo3-32b-step65000 | 31.4 24.1 to 34.8 | NCRI 15.2; no filler; placed | Official leaderboard | |
| 256 | GPT-5 nano | 31.0 22.2 to 35.3 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 257 | local/olmo3-32b-step46000 | 29.5 22.7 to 33.1 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 258 | local/olmo3-32b-step33000 | 27.4 19.2 to 31.3 | NCRI 15.2; no filler; placed | Official leaderboard | |
| 259 | local/gemma-7b-it | 25.8 17.7 to 29.9 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 260 | undi95/remm-slerp-l2-13b | 25.7 16.3 to 30.1 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 261 | | 24.9 15.4 to 27.8 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 262 | local/olmo3-7b-step360000 | 24.6 15.9 to 29.1 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 263 | local/gemma-2-2b | 24.3 15.7 to 28.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 264 | | 24.1 14.5 to 28.7 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 265 | local/talkie-1930-13b-it | 23.9 18.8 to 28.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 266 | gryphe/mythomax-l2-13b | 23.8 13.2 to 29.4 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 267 | local/olmo3-7b-step714000 | 23.2 12.7 to 27.7 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 268 | local/olmo3-7b-step182000 | 22.7 13.1 to 27.4 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 269 | local/olmo3-32b-step16000 | 21.5 12.5 to 26.3 | NCRI 15.2; no filler; placed | Official leaderboard | |
| 270 | local/olmo3-32b-step23000 | 20.4 9.4 to 25.1 | NCRI 15.2; no filler; placed | Official leaderboard | |
| 271 | local/olmo3-7b-step92000 | 18.9 7.7 to 24.1 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 272 | local/llama-2-7b-hf | 18.7 7.1 to 24.2 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 273 | local/huginn-r12 | 16.8 2.9 to 25.9 | NCRI 15.2; no filler; placed_after_release | Official leaderboard | |
| 274 | | 16.1 2 to 22.1 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 275 | local/olmo3-32b-step12000 | 15.6 2.4 to 21 | NCRI 15.2; no filler; placed | Official leaderboard | |
| 276 | | 15.0 0.7 to 21.6 | NCRI 15.2; no filler; sealed | Feb 2023 | Official leaderboard |
| 277 | local/gemma-2b-it | 11.9 -5.3 to 18.5 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 278 | local/talkie-web-13b-base | 10.6 2.9 to 15.8 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 279 | openai/babbage-002 | 6.6 -12.5 to 15.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 280 | local/olmo3-7b-step46000 | 4.1 -14.4 to 13.3 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 281 | local/olmo3-7b-step23000 | 3.1 -14.3 to 13 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 282 | local/olmo3-7b-step12000 | -8.3 -22.7 to 3.8 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 283 | local/pythia-12b | -8.6 ±13.5 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 284 | local/olmo3-32b-step6000 | -9.9 -23.2 to 4.2 | NCRI 15.2; no filler; placed | Official leaderboard | |
| 285 | local/olmo3-7b-step6000 | -11.5 ±12.8 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 286 | local/talkie-1930-13b-base | -12.4 ±12.1 | NCRI 15.2; no filler; sealed | Official leaderboard | |
| 287 | | -23.1 -28.1 to -20.6 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| # | Model | Score ± 95% CI | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 173.9 | NCRI 15.2; dots@2048; placed | Sep 2026 | Official leaderboard |
| 2 | GPT-6.1 Sol | 156.6 | NCRI 15.2; dots@128; placed_after_release | Sep 2026 | Official leaderboard |
| 3 | | 154.1 | NCRI 15.2; qrep@512; placed_after_release | Sep 2026 | Official leaderboard |
| 4 | GPT-6.1 Terra | 149.6 | NCRI 15.2; qrep@512; placed_after_release | Oct 2026 | Official leaderboard |
| 5 | | 129.7 | NCRI 15.2; dots@128; placed | Sep 2026 | Official leaderboard |
| 6 | Claude Fable 5.1 | 129.6 | NCRI 15.2; dots@128; placed | Sep 2026 | Official leaderboard |
| 7 | | 127.6 | NCRI 15.2; dots@32; sealed | Aug 2026 | Official leaderboard |
| 8 | | 123.9 | NCRI 15.2; count@512; sealed | Feb 2026 | Official leaderboard |
| 9 | GPT-6 Sol | 122.2 | NCRI 15.2; dots@128; placed_after_release | Sep 2026 | Official leaderboard |
| 10 | Claude Fable 5 | 121.5 | NCRI 15.2; dots@32; sealed | Jun 2026 | Official leaderboard |
| 11 | GPT-5.6 Sol | 119.5 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard |
| 12 | Claude Opus 5.5 | 118.9 116.8 to 121.1 | NCRI 15.2; dots@512; placed_after_release | Sep 2026 | Official leaderboard |
| 13 | Claude Opus 5 | 115.8 | NCRI 15.2; dots@128; sealed | Jul 2026 | Official leaderboard |
| 14 | GPT-5.5 | 111.0 | NCRI 15.2; qrep@512; sealed | Apr 2026 | Official leaderboard |
| 15 | | 110.9 | NCRI 15.2; count@512; sealed | Jul 2026 | Official leaderboard |
| 16 | Claude Opus 4.6 | 110.8 | NCRI 15.2; qrep@512; sealed | Feb 2026 | Official leaderboard |
| 17 | | 110.5 | NCRI 15.2; count@512; sealed | Dec 2025 | Official leaderboard |
| 18 | Claude Opus 4.5 | 110.5 | NCRI 15.2; dots@512; sealed | Nov 2025 | Official leaderboard |
| 19 | Claude Opus 4 | 109.0 | NCRI 15.2; dots@512; sealed | May 2025 | Official leaderboard |
| 20 | | 108.8 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard |
| 21 | Claude Opus 4.1 | 107.5 | NCRI 15.2; dots@128; sealed | Aug 2025 | Official leaderboard |
| 22 | | 107.5 | NCRI 15.2; dots@512; sealed | May 2026 | Official leaderboard |
| 23 | Kimi K3 | 105.6 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard |
| 24 | Claude Opus 4.7 | 103.9 | NCRI 15.2; dots@128; sealed | Apr 2026 | Official leaderboard |
| 25 | Kimi K3.1 Pro | 102.8 100.8 to 105 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 26 | | 102.3 | NCRI 15.2; dots@128; sealed | Aug 2026 | Official leaderboard |
| 27 | Claude Opus 4.8 | 102.2 | NCRI 15.2; dots@128; sealed | May 2026 | Official leaderboard |
| 28 | | 100.8 | NCRI 15.2; dots@2048; placed | Sep 2026 | Official leaderboard |
| 29 | Claude Sonnet 4.5 | 100.8 | NCRI 15.2; qrep@512; sealed | Sep 2025 | Official leaderboard |
| 30 | | 97.7 95.8 to 99.8 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 31 | Claude Sonnet 4 | 96.5 | NCRI 15.2; count@512; sealed | May 2025 | Official leaderboard |
| 32 | | 95.7 ±2.1 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 33 | GPT-5.6 Terra | 95.1 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard |
| 34 | Claude Sonnet 5.5 | 95.1 92.6 to 97.7 | NCRI 15.2; no filler; placed_after_release | Sep 2026 | Official leaderboard |
| 35 | GPT-5.4 | 94.1 | NCRI 15.2; dots@128; sealed | Mar 2026 | Official leaderboard |
| 36 | | 94.0 ±2.1 | NCRI 15.2; no filler; placed_after_release | Nov 2026 | Official leaderboard |
| 37 | Claude Sonnet 5 | 91.1 | NCRI 15.2; dots@128; sealed | Jun 2026 | Official leaderboard |
| 38 | Claude Sonnet 4.6 | 90.0 | NCRI 15.2; dots@512; sealed | Feb 2026 | Official leaderboard |
| 39 | | 87.0 84.5 to 89 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 40 | Grok 4.6 | 86.8 84.1 to 88.9 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 41 | | 86.5 83.8 to 88.7 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 42 | | 86.0 83.3 to 88.1 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 43 | GPT-5.2 | 85.8 83.1 to 88 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 44 | | 85.8 83.2 to 88.1 | NCRI 15.2; no filler; sealed | May 2026 | Official leaderboard |
| 45 | | 85.4 82.6 to 87.4 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 46 | GPT-5.3 Codex | 85.2 82.5 to 87.3 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 47 | | 84.1 81.2 to 86.1 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 48 | | 84.1 81.3 to 86.1 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 49 | | 83.3 80.7 to 85.5 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 50 | | 83.1 80.2 to 85.2 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard |
| 51 | | 82.8 79.9 to 85 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 52 | | 82.5 79.9 to 84.6 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 53 | | 82.2 79.3 to 84.3 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 54 | | 81.7 78.6 to 83.9 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 55 | | 81.6 ±2.2 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 56 | | 81.3 79 to 83.5 | NCRI 15.2; no filler; placed_after_release | Sep 2026 | Official leaderboard |
| 57 | Grok 4.5 | 81.0 78.1 to 83.3 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 58 | | 80.6 77.9 to 82.7 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 59 | | 80.6 77.8 to 82.5 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 60 | Kimi K2.5 | 80.6 77.7 to 82.8 | NCRI 15.2; no filler; sealed | Jan 2026 | Official leaderboard |
| 61 | | 80.0 77.1 to 81.9 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 62 | Kimi K2.6 | 79.2 76.3 to 81.3 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 63 | | 79.1 76.1 to 81.2 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 64 | | 78.3 75.4 to 80.2 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 65 | | 78.1 75.2 to 80.1 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 66 | | 77.6 74.7 to 79.9 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 67 | GLM-5.2 | 77.3 74 to 79.6 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard |
| 68 | GPT-6 Luna | 77.3 74.7 to 79.1 | NCRI 15.2; no filler; placed_after_release | Sep 2026 | Official leaderboard |
| 69 | | 77.3 74.3 to 79.3 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 70 | MiMo-V2.5-Pro | 77.1 74.2 to 79.2 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 71 | | 76.7 73.8 to 79 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 72 | GPT-5.6 Luna | 76.6 73.5 to 78.7 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 73 | GLM-5.3-Flash | 76.4 73.2 to 78.6 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 74 | GLM-5 | 76.2 73.2 to 78.3 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 75 | | 75.5 72.6 to 77.5 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 76 | GLM-5.1 | 75.2 71.8 to 77.7 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 77 | GPT-4.1 | 75.1 72 to 77.4 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 78 | | 75.0 72 to 77.2 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 79 | | 74.6 71.4 to 76.7 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 80 | GLM-5.3 | 74.5 71.5 to 76.8 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 81 | | 74.4 71.3 to 76.8 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 82 | GPT-4o Aug 2024 | 74.2 71.2 to 76.4 | NCRI 15.2; no filler; sealed | Aug 2024 | Official leaderboard |
| 83 | Claude Haiku 4.5 | 74.2 71.1 to 76.4 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard |
| 84 | | 73.9 71 to 76.3 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 85 | | 73.2 70.2 to 75.3 | NCRI 15.2; no filler; sealed | Apr 2024 | Official leaderboard |
| 86 | | 73.1 69.7 to 75.5 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 87 | GPT-4 Jun 2023 | 72.8 69.7 to 75 | NCRI 15.2; no filler; sealed | Jun 2023 | Official leaderboard |
| 88 | | 72.6 69.2 to 74.7 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard |
| 89 | | 72.5 69.3 to 74.7 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 90 | Mercury 3 | 72.3 69.9 to 74.5 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 91 | | 72.3 69 to 74.5 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 92 | GPT-4o May 2024 | 72.2 69.1 to 74.3 | NCRI 15.2; no filler; sealed | May 2024 | Official leaderboard |
| 93 | GLM-4.7 | 71.3 68 to 73.5 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 94 | GLM-4.5 | 71.3 68.1 to 73.3 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 95 | Qwen3-Coder-Plus | 71.2 68 to 73.3 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 96 | GLM-4.6 | 70.8 67.6 to 73.2 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 97 | GPT-5.1 | 70.6 67.4 to 72.9 | NCRI 15.2; no filler; sealed | Nov 2025 | Official leaderboard |
| 98 | | 70.5 67.3 to 72.9 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 99 | | 70.5 67.5 to 72.7 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 100 | GPT-5.4 Mini | 70.5 67.3 to 72.4 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 101 | GPT-4o Nov 2024 | 70.3 67.2 to 72.6 | NCRI 15.2; no filler; sealed | Nov 2024 | Official leaderboard |
| 102 | | 70.2 66.7 to 72.5 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 103 | | 70.2 66.7 to 72.8 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard |
| 104 | MiMo-V2.5 | 69.9 66.9 to 72.3 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 105 | | 69.5 66 to 71.9 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 106 | | 69.4 66.4 to 71.6 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 107 | Grok 4.3 | 69.2 66 to 71.2 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 108 | | 68.9 65.8 to 71 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 109 | | 68.4 65.2 to 70.5 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 110 | | 68.4 65.6 to 71.3 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 111 | Claude Haiku 5.5 | 68.3 65.6 to 70.9 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 112 | | 67.9 64.3 to 70.2 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 113 | | 67.5 64.1 to 69.9 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 114 | | 67.4 63.9 to 69.6 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 115 | Grok 4.20 | 67.1 64 to 69.2 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 116 | GPT-5 | 67.0 63.6 to 69.1 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 117 | GLM-4.5V | 66.7 63.3 to 69 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 118 | | 66.6 63 to 68.9 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 119 | | 66.3 63 to 68.7 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 120 | | 66.1 62.7 to 68.6 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 121 | Mercury 2.5 | 65.8 62.4 to 68.1 | NCRI 15.2; no filler; placed | Sep 2026 | Official leaderboard |
| 122 | | 65.5 61.8 to 67.9 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 123 | | 65.3 62 to 67.7 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 124 | | 65.2 61.7 to 67.6 | NCRI 15.2; no filler; sealed | Jan 2025 | Official leaderboard |
| 125 | GPT-4.1 mini | 65.0 61.4 to 67.4 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 126 | | 65.0 61.6 to 67.1 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 127 | | 64.5 61.1 to 66.9 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 128 | | 64.0 60.6 to 66.3 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 129 | | 64.0 60.5 to 66.7 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard |
| 130 | | 63.8 60.4 to 66.1 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 131 | | 63.5 60 to 66 | NCRI 15.2; no filler; sealed | Sep 2024 | Official leaderboard |
| 132 | GLM-4.6V | 63.4 59.7 to 65.9 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 133 | | 63.3 59.6 to 65.7 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 134 | | 63.3 59.7 to 65.5 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 135 | Hermes 4 405B | 62.8 59.2 to 65.2 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 136 | | 62.6 59 to 64.8 | NCRI 15.2; no filler; sealed | Dec 2024 | Official leaderboard |
| 137 | | 62.5 59.1 to 64.8 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 138 | GLM-4.5-Air | 62.4 58.8 to 64.8 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 139 | | 62.1 58.5 to 64.6 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 140 | | 61.6 58.3 to 64.1 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 141 | | 61.4 57.8 to 63.7 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 142 | | 61.1 57.6 to 63.5 | NCRI 15.2; no filler; sealed | Sep 2024 | Official leaderboard |
| 143 | | 61.0 57.5 to 63.4 | NCRI 15.2; no filler; sealed | Jun 2024 | Official leaderboard |
| 144 | | 60.1 56.3 to 62.5 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 145 | | 60.0 56.1 to 62.5 | NCRI 15.2; no filler; sealed | Nov 2024 | Official leaderboard |
| 146 | | 59.9 56.1 to 62.6 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard |
| 147 | | 59.7 55.6 to 62.4 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard |
| 148 | | 59.6 55.9 to 62.3 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 149 | | 59.5 55.7 to 61.9 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 150 | Ripple-65B Preview | 59.3 55 to 62.4 | NCRI 15.2; no filler; placed_after_release | Nov 2026 | Official leaderboard |
| 151 | | 59.2 55.2 to 61.8 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 152 | gpt-oss-120b | 59.0 54.9 to 61.4 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 153 | | 58.3 54.6 to 61 | NCRI 15.2; no filler; sealed | Dec 2024 | Official leaderboard |
| 154 | | 57.7 53.9 to 60.2 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 155 | | 57.6 54 to 60.1 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 156 | | 57.5 53.3 to 59.9 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 157 | | 57.5 53.7 to 59.9 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 158 | Claude 3 Haiku | 57.3 53.2 to 59.8 | NCRI 15.2; no filler; sealed | Mar 2024 | Official leaderboard |
| 159 | | 56.1 52.2 to 58.4 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 160 | | 56.0 52.2 to 58.5 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard |
| 161 | | 56.0 52 to 58.4 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 162 | glm-4.7-flash | 55.9 51.7 to 58.4 | NCRI 15.2; no filler; sealed | Jan 2026 | Official leaderboard |
| 163 | GPT-4o mini | 55.7 51.6 to 58.2 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 164 | Mercury 2 | 55.1 51.1 to 57.7 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 165 | | 55.0 51.2 to 57.7 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 166 | gpt-oss-20b | 54.8 50.3 to 57.8 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 167 | | 54.3 50.3 to 57 | NCRI 15.2; no filler; placed | Dec 2025 | Official leaderboard |
| 168 | | 54.2 50 to 56.7 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 169 | | 53.3 49.2 to 55.7 | NCRI 15.2; no filler; sealed | Jun 2024 | Official leaderboard |
| 170 | | 52.4 47.8 to 55.3 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 171 | | 51.4 46.8 to 54.3 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 172 | GPT-5 mini | 50.7 45.7 to 53.6 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 173 | | 50.6 45.8 to 53.4 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 174 | | 50.4 45.3 to 53.5 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 175 | GPT-4.1 nano | 50.2 45.4 to 52.9 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 176 | | 49.9 45.4 to 52.4 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 177 | | 49.7 45.4 to 52.7 | NCRI 15.2; no filler; sealed | Feb 2024 | Official leaderboard |
| 178 | GPT-3.5 Turbo Instruct | 49.6 45.2 to 51.9 | NCRI 15.2; no filler; sealed | Sep 2023 | Official leaderboard |
| 179 | GPT-5.4 Nano | 49.4 45.1 to 51.9 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 180 | | 49.3 44.9 to 52.2 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 181 | | 48.7 44.2 to 51.8 | NCRI 15.2; no filler; sealed | Jun 2025 | Official leaderboard |
| 182 | | 47.3 42 to 50.2 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 183 | | 46.0 40.9 to 49.3 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 184 | | 44.7 39.6 to 47.9 | NCRI 15.2; no filler; sealed | Jun 2024 | Official leaderboard |
| 185 | | 44.3 39.2 to 48 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 186 | | 44.0 39.1 to 46.5 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 187 | | 43.9 38.9 to 47 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 188 | | 43.2 38.4 to 46.1 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 189 | | 42.5 37.4 to 45.4 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 190 | | 40.6 35.1 to 43.8 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 191 | | 38.5 32.2 to 42.1 | NCRI 15.2; no filler; sealed | Jun 2025 | Official leaderboard |
| 192 | | 37.9 32 to 41.3 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 193 | | 37.5 31.7 to 39.9 | NCRI 15.2; no filler; sealed | Feb 2023 | Official leaderboard |
| 194 | | 35.5 29.1 to 38.7 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 195 | davinci-002 | 32.6 26 to 35.9 | NCRI 15.2; no filler; sealed | Mar 2022 | Official leaderboard |
| 196 | GPT-5 nano | 31.0 22.2 to 35.3 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 197 | | 24.9 15.4 to 27.8 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 198 | | 24.1 14.5 to 28.7 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 199 | | 16.1 2 to 22.1 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard |
| 200 | | 15.0 0.7 to 21.6 | NCRI 15.2; no filler; sealed | Feb 2023 | Official leaderboard |
| 201 | | -23.1 -28.1 to -20.6 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 173.9 | NCRI 15.2; dots@2048; placed | Sep 2026 | Official leaderboard |
| 2 | GPT-6.2 Astra | 171.0 | Nov 2026 | Lab model card | |
| 3 | GPT-6.1 Sol | 156.6 | NCRI 15.2; dots@128; placed_after_release | Sep 2026 | Official leaderboard |
| 4 | 154.1 | NCRI 15.2; qrep@512; placed_after_release | Sep 2026 | Official leaderboard | |
| 5 | GPT-6.1 Terra | 149.6 | NCRI 15.2; qrep@512; placed_after_release | Oct 2026 | Official leaderboard |
| 6 | 129.7 | NCRI 15.2; dots@128; placed | Sep 2026 | Official leaderboard | |
| 7 | Claude Fable 5.1 | 129.6 | NCRI 15.2; dots@128; placed | Sep 2026 | Official leaderboard |
| 8 | 127.6 | NCRI 15.2; dots@32; sealed | Aug 2026 | Official leaderboard | |
| 9 | 123.9 | NCRI 15.2; count@512; sealed | Feb 2026 | Official leaderboard | |
| 10 | GPT-6 Sol | 122.2 | NCRI 15.2; dots@128; placed_after_release | Sep 2026 | Official leaderboard |
| 11 | Claude Fable 5 | 121.5 | NCRI 15.2; dots@32; sealed | Jun 2026 | Official leaderboard |
| 12 | GPT-5.6 Sol | 119.5 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard |
| 13 | Claude Opus 5.5 | 118.9 | NCRI 15.2; dots@512; placed_after_release | Sep 2026 | Official leaderboard |
| 14 | Claude Opus 5 | 115.8 | NCRI 15.2; dots@128; sealed | Jul 2026 | Official leaderboard |
| 15 | GPT-5.5 | 111.0 | NCRI 15.2; qrep@512; sealed | Apr 2026 | Official leaderboard |
| 16 | 110.9 | NCRI 15.2; count@512; sealed | Jul 2026 | Official leaderboard | |
| 17 | Claude Opus 4.6 | 110.8 | NCRI 15.2; qrep@512; sealed | Feb 2026 | Official leaderboard |
| 18 | 110.5 | NCRI 15.2; count@512; sealed | Dec 2025 | Official leaderboard | |
| 19 | Claude Opus 4.5 | 110.5 | NCRI 15.2; dots@512; sealed | Nov 2025 | Official leaderboard |
| 20 | Claude Opus 4 | 109.0 | NCRI 15.2; dots@512; sealed | May 2025 | Official leaderboard |
| 21 | 108.8 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard | |
| 22 | Claude Opus 4.1 | 107.5 | NCRI 15.2; dots@128; sealed | Aug 2025 | Official leaderboard |
| 23 | 107.5 | NCRI 15.2; dots@512; sealed | May 2026 | Official leaderboard | |
| 24 | Kimi K3 | 105.6 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard |
| 25 | Claude Opus 4.7 | 103.9 | NCRI 15.2; dots@128; sealed | Apr 2026 | Official leaderboard |
| 26 | Kimi K3.1 Pro | 102.8 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 27 | 102.3 | NCRI 15.2; dots@128; sealed | Aug 2026 | Official leaderboard | |
| 28 | Claude Opus 4.8 | 102.2 | NCRI 15.2; dots@128; sealed | May 2026 | Official leaderboard |
| 29 | 100.8 | NCRI 15.2; dots@2048; placed | Sep 2026 | Official leaderboard | |
| 30 | Claude Sonnet 4.5 | 100.8 | NCRI 15.2; qrep@512; sealed | Sep 2025 | Official leaderboard |
| 31 | 97.7 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard | |
| 32 | Claude Sonnet 4 | 96.5 | NCRI 15.2; count@512; sealed | May 2025 | Official leaderboard |
| 33 | 95.7 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard | |
| 34 | GPT-5.6 Terra | 95.1 | NCRI 15.2; qrep@512; sealed | Jul 2026 | Official leaderboard |
| 35 | Claude Sonnet 5.5 | 95.1 | NCRI 15.2; no filler; placed_after_release | Sep 2026 | Official leaderboard |
| 36 | GPT-5.4 | 94.1 | NCRI 15.2; dots@128; sealed | Mar 2026 | Official leaderboard |
| 37 | 94.0 | NCRI 15.2; no filler; placed_after_release | Nov 2026 | Official leaderboard | |
| 38 | Claude Sonnet 5 | 91.1 | NCRI 15.2; dots@128; sealed | Jun 2026 | Official leaderboard |
| 39 | Claude Sonnet 4.6 | 90.0 | NCRI 15.2; dots@512; sealed | Feb 2026 | Official leaderboard |
| 40 | 87.0 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 41 | Grok 4.6 | 86.8 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 42 | 86.5 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard | |
| 43 | 86.0 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard | |
| 44 | GPT-5.2 | 85.8 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 45 | 85.8 | NCRI 15.2; no filler; sealed | May 2026 | Official leaderboard | |
| 46 | 85.4 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard | |
| 47 | GPT-5.3 Codex | 85.2 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 48 | 84.1 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard | |
| 49 | 84.1 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 50 | 83.3 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard | |
| 51 | 83.1 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard | |
| 52 | 82.8 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard | |
| 53 | 82.5 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard | |
| 54 | 82.2 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard | |
| 55 | 81.7 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 56 | 81.6 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard | |
| 57 | 81.3 | NCRI 15.2; no filler; placed_after_release | Sep 2026 | Official leaderboard | |
| 58 | Grok 4.5 | 81.0 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 59 | 80.6 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard | |
| 60 | 80.6 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 61 | Kimi K2.5 | 80.6 | NCRI 15.2; no filler; sealed | Jan 2026 | Official leaderboard |
| 62 | 80.0 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 63 | Kimi K2.6 | 79.2 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 64 | 79.1 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 65 | 78.3 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard | |
| 66 | 78.1 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 67 | 77.6 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard | |
| 68 | GLM-5.2 | 77.3 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard |
| 69 | GPT-6 Luna | 77.3 | NCRI 15.2; no filler; placed_after_release | Sep 2026 | Official leaderboard |
| 70 | 77.3 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard | |
| 71 | MiMo-V2.5-Pro | 77.1 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 72 | 76.7 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard | |
| 73 | GPT-5.6 Luna | 76.6 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard |
| 74 | GLM-5.3-Flash | 76.4 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 75 | GLM-5 | 76.2 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 76 | 75.5 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard | |
| 77 | GLM-5.1 | 75.2 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 78 | GPT-4.1 | 75.1 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 79 | 75.0 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard | |
| 80 | 74.6 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard | |
| 81 | GLM-5.3 | 74.5 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard |
| 82 | 74.4 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard | |
| 83 | GPT-4o Aug 2024 | 74.2 | NCRI 15.2; no filler; sealed | Aug 2024 | Official leaderboard |
| 84 | Claude Haiku 4.5 | 74.2 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard |
| 85 | 73.9 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 86 | 73.2 | NCRI 15.2; no filler; sealed | Apr 2024 | Official leaderboard | |
| 87 | 73.1 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard | |
| 88 | GPT-4 Jun 2023 | 72.8 | NCRI 15.2; no filler; sealed | Jun 2023 | Official leaderboard |
| 89 | 72.6 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard | |
| 90 | 72.5 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard | |
| 91 | Mercury 3 | 72.3 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 92 | 72.3 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 93 | GPT-4o May 2024 | 72.2 | NCRI 15.2; no filler; sealed | May 2024 | Official leaderboard |
| 94 | GLM-4.7 | 71.3 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 95 | GLM-4.5 | 71.3 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 96 | Qwen3-Coder-Plus | 71.2 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 97 | GLM-4.6 | 70.8 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard |
| 98 | GPT-5.1 | 70.6 | NCRI 15.2; no filler; sealed | Nov 2025 | Official leaderboard |
| 99 | 70.5 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard | |
| 100 | 70.5 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard | |
| 101 | GPT-5.4 Mini | 70.5 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 102 | GPT-4o Nov 2024 | 70.3 | NCRI 15.2; no filler; sealed | Nov 2024 | Official leaderboard |
| 103 | 70.2 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard | |
| 104 | 70.2 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard | |
| 105 | MiMo-V2.5 | 69.9 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 106 | 69.5 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 107 | 69.4 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 108 | Grok 4.3 | 69.2 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard |
| 109 | 68.9 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard | |
| 110 | 68.4 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard | |
| 111 | 68.4 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard | |
| 112 | Claude Haiku 5.5 | 68.3 | NCRI 15.2; no filler; placed_after_release | Oct 2026 | Official leaderboard |
| 113 | 67.9 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard | |
| 114 | 67.5 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 115 | 67.4 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 116 | Grok 4.20 | 67.1 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 117 | GPT-5 | 67.0 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 118 | GLM-4.5V | 66.7 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 119 | 66.6 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 120 | 66.3 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard | |
| 121 | 66.1 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard | |
| 122 | Mercury 2.5 | 65.8 | NCRI 15.2; no filler; placed | Sep 2026 | Official leaderboard |
| 123 | 65.5 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 124 | 65.3 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard | |
| 125 | 65.2 | NCRI 15.2; no filler; sealed | Jan 2025 | Official leaderboard | |
| 126 | GPT-4.1 mini | 65.0 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 127 | 65.0 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 128 | 64.5 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 129 | 64.0 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard | |
| 130 | 64.0 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard | |
| 131 | 63.8 | NCRI 15.2; no filler; sealed | Sep 2025 | Official leaderboard | |
| 132 | 63.5 | NCRI 15.2; no filler; sealed | Sep 2024 | Official leaderboard | |
| 133 | GLM-4.6V | 63.4 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard |
| 134 | 63.3 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 135 | 63.3 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard | |
| 136 | Hermes 4 405B | 62.8 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 137 | 62.6 | NCRI 15.2; no filler; sealed | Dec 2024 | Official leaderboard | |
| 138 | 62.5 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard | |
| 139 | GLM-4.5-Air | 62.4 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard |
| 140 | 62.1 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard | |
| 141 | 61.6 | NCRI 15.2; no filler; sealed | Apr 2026 | Official leaderboard | |
| 142 | 61.4 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 143 | 61.1 | NCRI 15.2; no filler; sealed | Sep 2024 | Official leaderboard | |
| 144 | 61.0 | NCRI 15.2; no filler; sealed | Jun 2024 | Official leaderboard | |
| 145 | 60.1 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 146 | 60.0 | NCRI 15.2; no filler; sealed | Nov 2024 | Official leaderboard | |
| 147 | 59.9 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard | |
| 148 | 59.7 | NCRI 15.2; no filler; sealed | Jun 2026 | Official leaderboard | |
| 149 | 59.6 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard | |
| 150 | 59.5 | NCRI 15.2; no filler; sealed | Jul 2025 | Official leaderboard | |
| 151 | Ripple-65B Preview | 59.3 | NCRI 15.2; no filler; placed_after_release | Nov 2026 | Official leaderboard |
| 152 | 59.2 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard | |
| 153 | gpt-oss-120b | 59.0 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 154 | 58.3 | NCRI 15.2; no filler; sealed | Dec 2024 | Official leaderboard | |
| 155 | 57.7 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard | |
| 156 | 57.6 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard | |
| 157 | 57.5 | NCRI 15.2; no filler; sealed | Jul 2026 | Official leaderboard | |
| 158 | 57.5 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard | |
| 159 | Claude 3 Haiku | 57.3 | NCRI 15.2; no filler; sealed | Mar 2024 | Official leaderboard |
| 160 | 56.1 | NCRI 15.2; no filler; sealed | Aug 2026 | Official leaderboard | |
| 161 | 56.0 | NCRI 15.2; no filler; sealed | Oct 2025 | Official leaderboard | |
| 162 | 56.0 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard | |
| 163 | glm-4.7-flash | 55.9 | NCRI 15.2; no filler; sealed | Jan 2026 | Official leaderboard |
| 164 | GPT-4o mini | 55.7 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard |
| 165 | Mercury 2 | 55.1 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard |
| 166 | 55.0 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard | |
| 167 | gpt-oss-20b | 54.8 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 168 | 54.3 | NCRI 15.2; no filler; placed | Dec 2025 | Official leaderboard | |
| 169 | 54.2 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 170 | 53.3 | NCRI 15.2; no filler; sealed | Jun 2024 | Official leaderboard | |
| 171 | 52.4 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 172 | 51.4 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard | |
| 173 | GPT-5 mini | 50.7 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 174 | 50.6 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard | |
| 175 | 50.4 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard | |
| 176 | GPT-4.1 nano | 50.2 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard |
| 177 | 49.9 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard | |
| 178 | 49.7 | NCRI 15.2; no filler; sealed | Feb 2024 | Official leaderboard | |
| 179 | GPT-3.5 Turbo Instruct | 49.6 | NCRI 15.2; no filler; sealed | Sep 2023 | Official leaderboard |
| 180 | GPT-5.4 Nano | 49.4 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard |
| 181 | 49.3 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard | |
| 182 | 48.7 | NCRI 15.2; no filler; sealed | Jun 2025 | Official leaderboard | |
| 183 | 47.3 | NCRI 15.2; no filler; sealed | Dec 2025 | Official leaderboard | |
| 184 | 46.0 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 185 | 44.7 | NCRI 15.2; no filler; sealed | Jun 2024 | Official leaderboard | |
| 186 | 44.3 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 187 | 44.0 | NCRI 15.2; no filler; sealed | Mar 2026 | Official leaderboard | |
| 188 | 43.9 | NCRI 15.2; no filler; sealed | Apr 2025 | Official leaderboard | |
| 189 | 43.2 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard | |
| 190 | 42.5 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard | |
| 191 | 40.6 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard | |
| 192 | 38.5 | NCRI 15.2; no filler; sealed | Jun 2025 | Official leaderboard | |
| 193 | 37.9 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard | |
| 194 | 37.5 | NCRI 15.2; no filler; sealed | Feb 2023 | Official leaderboard | |
| 195 | 35.5 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 196 | davinci-002 | 32.6 | NCRI 15.2; no filler; sealed | Mar 2022 | Official leaderboard |
| 197 | GPT-5 nano | 31.0 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
| 198 | 24.9 | NCRI 15.2; no filler; sealed | Feb 2026 | Official leaderboard | |
| 199 | 24.1 | NCRI 15.2; no filler; sealed | Jul 2024 | Official leaderboard | |
| 200 | 16.1 | NCRI 15.2; no filler; sealed | Mar 2025 | Official leaderboard | |
| 201 | 15.0 | NCRI 15.2; no filler; sealed | Feb 2023 | Official leaderboard | |
| 202 | -23.1 | NCRI 15.2; no filler; sealed | Aug 2025 | Official leaderboard |
Labs report scores for 2 models in their cards. Switch the scores above to rank them together.
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: Official nocot-bench (NCRI) leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| GPT-6 Astra | NCRI 15.2; dots@2048; placed | 173.9 | |||
| GPT-6 Astra | NCRI 15.2; no filler; placed | 159.0 | |||
| GPT-6.1 Sol | NCRI 15.2; dots@128; placed_after_release | 156.6 | |||
| NCRI 15.2; qrep@512; placed_after_release | 154.1 | ||||
| NCRI 15.2; no filler; placed_after_release | 151.2 | ||||
| GPT-6.1 Terra | NCRI 15.2; qrep@512; placed_after_release | 149.6 | |||
| GPT-6.1 Sol | NCRI 15.2; no filler; placed_after_release | 149.0 | |||
| GPT-6.1 Terra | NCRI 15.2; no filler; placed_after_release | 130.5 | |||
| NCRI 15.2; dots@128; placed | 129.7 | ||||
| Claude Fable 5.1 | NCRI 15.2; dots@128; placed | 129.6 | |||
| Claude Fable 5.1 | NCRI 15.2; no filler; placed | 128.0 | |||
| NCRI 15.2; dots@32; sealed | 127.6 | ||||
| NCRI 15.2; no filler; sealed | 126.1 | ||||
| NCRI 15.2; no filler; placed | 125.8 | ||||
| NCRI 15.2; count@512; sealed | 123.9 | ||||
| GPT-6 Sol | NCRI 15.2; dots@128; placed_after_release | 122.2 | |||
| Claude Fable 5 | NCRI 15.2; dots@32; sealed | 121.5 | |||
| GPT-5.6 Sol | NCRI 15.2; qrep@512; sealed | 119.5 | |||
| Claude Fable 5 | NCRI 15.2; no filler; sealed | 119.0 | |||
| Claude Opus 5.5 | NCRI 15.2; dots@512; placed_after_release | 118.9 | |||
| Claude Opus 5 | NCRI 15.2; dots@128; sealed | 115.8 | |||
| NCRI 15.2; no filler; sealed | 115.3 | ||||
| Claude Opus 5 | NCRI 15.2; no filler; sealed | 115.1 | |||
| GPT-6 Sol | NCRI 15.2; no filler; placed_after_release | 114.6 | |||
| GPT-5.5 | NCRI 15.2; qrep@512; sealed | 111.0 | |||
| NCRI 15.2; count@512; sealed | 110.9 | ||||
| Claude Opus 4.6 | NCRI 15.2; qrep@512; sealed | 110.8 | |||
| NCRI 15.2; count@512; sealed | 110.5 | ||||
| Claude Opus 4.5 | NCRI 15.2; dots@512; sealed | 110.5 | |||
| GPT-5.6 Sol | NCRI 15.2; no filler; sealed | 109.4 | |||
| Claude Opus 4 | NCRI 15.2; dots@512; sealed | 109.0 | |||
| NCRI 15.2; qrep@512; sealed | 108.8 | ||||
| Claude Opus 4.1 | NCRI 15.2; dots@128; sealed | 107.5 | |||
| NCRI 15.2; dots@512; sealed | 107.5 | ||||
| Kimi K3 | NCRI 15.2; qrep@512; sealed | 105.6 | |||
| Claude Opus 4.6 | NCRI 15.2; no filler; sealed | 104.3 | |||
| Claude Opus 4.7 | NCRI 15.2; dots@128; sealed | 103.9 | |||
| Claude Opus 4.7 | NCRI 15.2; no filler; sealed | 103.7 | |||
| NCRI 15.2; no filler; sealed | 103.2 | ||||
| Kimi K3.1 Pro | NCRI 15.2; no filler; placed_after_release | 102.8 | |||
| Claude Opus 4.5 | NCRI 15.2; no filler; sealed | 102.7 | |||
| Claude Opus 4.1 | NCRI 15.2; no filler; sealed | 102.6 | |||
| NCRI 15.2; dots@128; sealed | 102.3 | ||||
| Claude Opus 4.8 | NCRI 15.2; dots@128; sealed | 102.2 | |||
| Claude Opus 4.8 | NCRI 15.2; no filler; sealed | 102.0 | |||
| NCRI 15.2; no filler; sealed | 101.5 | ||||
| Claude Opus 4 | NCRI 15.2; no filler; sealed | 101.4 | |||
| NCRI 15.2; no filler; sealed | 101.0 | ||||
| NCRI 15.2; dots@2048; placed | 100.8 | ||||
| Claude Sonnet 4.5 | NCRI 15.2; qrep@512; sealed | 100.8 | |||
| GPT-5.5 | NCRI 15.2; no filler; sealed | 100.2 | |||
| NCRI 15.2; no filler; placed | 98.3 | ||||
| NCRI 15.2; no filler; placed_after_release | 97.7 | ||||
| NCRI 15.2; no filler; sealed | 97.1 | ||||
| NCRI 15.2; no filler; sealed | 96.8 | ||||
| Kimi K3 | NCRI 15.2; no filler; sealed | 96.8 | |||
| Claude Sonnet 4 | NCRI 15.2; count@512; sealed | 96.5 | |||
| NCRI 15.2; no filler; placed_after_release | 95.7 | ||||
| GPT-5.6 Terra | NCRI 15.2; qrep@512; sealed | 95.1 | |||
| Claude Sonnet 5.5 | NCRI 15.2; no filler; placed_after_release | 95.1 | |||
| GPT-5.4 | NCRI 15.2; dots@128; sealed | 94.1 | |||
| NCRI 15.2; no filler; placed_after_release | 94.0 | ||||
| GPT-5.4 | NCRI 15.2; no filler; sealed | 94.0 | |||
| Claude Sonnet 4.5 | NCRI 15.2; no filler; sealed | 94.0 | |||
| Claude Sonnet 4 | NCRI 15.2; no filler; sealed | 91.3 | |||
| Claude Sonnet 5 | NCRI 15.2; dots@128; sealed | 91.1 | |||
| Claude Sonnet 5 | NCRI 15.2; no filler; sealed | 90.8 | |||
| Claude Sonnet 4.6 | NCRI 15.2; dots@512; sealed | 90.0 | |||
| GPT-5.6 Terra | NCRI 15.2; no filler; sealed | 89.0 | |||
| Claude Sonnet 4.6 | NCRI 15.2; no filler; sealed | 87.8 | |||
| NCRI 15.2; no filler; sealed | 87.0 | ||||
| Grok 4.6 | NCRI 15.2; no filler; sealed | 86.8 | |||
| NCRI 15.2; no filler; sealed | 86.5 | ||||
| NCRI 15.2; dots@128; sealed | 86.4 | ||||
| NCRI 15.2; no filler; sealed | 86.0 | ||||
| GPT-5.2 | NCRI 15.2; no filler; sealed | 85.8 | |||
| NCRI 15.2; no filler; sealed | 85.8 | ||||
| NCRI 15.2; no filler; sealed | 85.4 | ||||
| GPT-5.3 Codex | NCRI 15.2; no filler; sealed | 85.2 | |||
| NCRI 15.2; no filler; sealed | 84.1 | ||||
| NCRI 15.2; no filler; sealed | 84.1 | ||||
| NCRI 15.2; no filler; sealed | 83.3 | ||||
| NCRI 15.2; no filler; sealed | 83.1 | ||||
| NCRI 15.2; no filler; sealed | 82.8 | ||||
| NCRI 15.2; no filler; sealed | 82.5 | ||||
| NCRI 15.2; no filler; sealed | 82.2 | ||||
| NCRI 15.2; no filler; sealed | 81.9 | ||||
| NCRI 15.2; no filler; sealed | 81.7 | ||||
| NCRI 15.2; no filler; placed_after_release | 81.6 | ||||
| NCRI 15.2; no filler; placed_after_release | 81.3 | ||||
| NCRI 15.2; no filler; sealed | 81.0 | ||||
| Grok 4.5 | NCRI 15.2; no filler; sealed | 81.0 | |||
| NCRI 15.2; no filler; sealed | 80.6 | ||||
| NCRI 15.2; no filler; sealed | 80.6 | ||||
| Kimi K2.5 | NCRI 15.2; no filler; sealed | 80.6 | |||
| NCRI 15.2; no filler; sealed | 80.0 | ||||
| Kimi K2.6 | NCRI 15.2; no filler; sealed | 79.2 | |||
| NCRI 15.2; no filler; sealed | 79.1 | ||||
| NCRI 15.2; no filler; sealed | 78.3 | ||||
| NCRI 15.2; no filler; sealed | 78.1 | ||||
| NCRI 15.2; no filler; sealed | 77.6 | ||||
| GLM-5.2 | NCRI 15.2; no filler; sealed | 77.3 | |||
| GPT-6 Luna | NCRI 15.2; no filler; placed_after_release | 77.3 | |||
| NCRI 15.2; no filler; sealed | 77.3 | ||||
| MiMo-V2.5-Pro | NCRI 15.2; no filler; sealed | 77.1 | |||
| NCRI 15.2; no filler; sealed | 76.7 | ||||
| GPT-5.6 Luna | NCRI 15.2; no filler; sealed | 76.6 | |||
| GLM-5.3-Flash | NCRI 15.2; no filler; sealed | 76.4 | |||
| GLM-5 | NCRI 15.2; no filler; sealed | 76.2 | |||
| NCRI 15.2; no filler; sealed | 75.5 | ||||
| GLM-5.1 | NCRI 15.2; no filler; sealed | 75.2 | |||
| GPT-4.1 | NCRI 15.2; no filler; sealed | 75.1 | |||
| NCRI 15.2; no filler; sealed | 75.0 | ||||
| NCRI 15.2; no filler; sealed | 74.6 | ||||
| GLM-5.3 | NCRI 15.2; no filler; sealed | 74.5 | |||
| NCRI 15.2; no filler; sealed | 74.4 | ||||
| GPT-4o Aug 2024 | NCRI 15.2; no filler; sealed | 74.2 | |||
| Claude Haiku 4.5 | NCRI 15.2; no filler; sealed | 74.2 | |||
| NCRI 15.2; no filler; sealed | 73.9 | ||||
| NCRI 15.2; no filler; sealed | 73.2 | ||||
| NCRI 15.2; no filler; sealed | 73.1 | ||||
| GPT-4 Jun 2023 | NCRI 15.2; no filler; sealed | 72.8 | |||
| NCRI 15.2; no filler; sealed | 72.6 | ||||
| NCRI 15.2; no filler; sealed | 72.5 | ||||
| NCRI 15.2; no filler; sealed | 72.3 | ||||
| Mercury 3 | NCRI 15.2; no filler; placed_after_release | 72.3 | |||
| NCRI 15.2; no filler; sealed | 72.3 | ||||
| GPT-4o May 2024 | NCRI 15.2; no filler; sealed | 72.2 | |||
| GLM-4.7 | NCRI 15.2; no filler; sealed | 71.3 | |||
| GLM-4.5 | NCRI 15.2; no filler; sealed | 71.3 | |||
| Qwen3-Coder-Plus | NCRI 15.2; no filler; sealed | 71.2 | |||
| GLM-4.6 | NCRI 15.2; no filler; sealed | 70.8 | |||
| GPT-5.1 | NCRI 15.2; no filler; sealed | 70.6 | |||
| NCRI 15.2; no filler; sealed | 70.5 | ||||
| NCRI 15.2; no filler; sealed | 70.5 | ||||
| GPT-5.4 Mini | NCRI 15.2; no filler; sealed | 70.5 | |||
| GPT-4o Nov 2024 | NCRI 15.2; no filler; sealed | 70.3 | |||
| NCRI 15.2; no filler; sealed | 70.2 | ||||
| NCRI 15.2; no filler; sealed | 70.2 | ||||
| MiMo-V2.5 | NCRI 15.2; no filler; sealed | 69.9 | |||
| NCRI 15.2; no filler; sealed | 69.5 | ||||
| NCRI 15.2; no filler; sealed | 69.4 | ||||
| Grok 4.3 | NCRI 15.2; no filler; sealed | 69.2 | |||
| NCRI 15.2; no filler; sealed | 68.9 | ||||
| NCRI 15.2; no filler; sealed | 68.4 | ||||
| NCRI 15.2; no filler; placed_after_release | 68.4 | ||||
| Claude Haiku 5.5 | NCRI 15.2; no filler; placed_after_release | 68.3 | |||
| NCRI 15.2; no filler; sealed | 67.9 | ||||
| NCRI 15.2; no filler; sealed | 67.5 | ||||
| NCRI 15.2; no filler; sealed | 67.4 | ||||
| Grok 4.20 | NCRI 15.2; no filler; sealed | 67.1 | |||
| GPT-5 | NCRI 15.2; no filler; sealed | 67.0 | |||
| GLM-4.5V | NCRI 15.2; no filler; sealed | 66.7 | |||
| NCRI 15.2; no filler; sealed | 66.6 | ||||
| NCRI 15.2; no filler; sealed | 66.3 | ||||
| NCRI 15.2; no filler; sealed | 66.3 | ||||
| NCRI 15.2; no filler; sealed | 66.2 | ||||
| NCRI 15.2; no filler; sealed | 66.1 | ||||
| NCRI 15.2; no filler; sealed | 66.1 | ||||
| Mercury 2.5 | NCRI 15.2; no filler; placed | 65.8 | |||
| NCRI 15.2; no filler; sealed | 65.5 | ||||
| NCRI 15.2; no filler; sealed | 65.3 | ||||
| NCRI 15.2; no filler; sealed | 65.2 | ||||
| GPT-4.1 mini | NCRI 15.2; no filler; sealed | 65.0 | |||
| NCRI 15.2; no filler; sealed | 65.0 | ||||
| NCRI 15.2; no filler; sealed | 64.5 | ||||
| NCRI 15.2; no filler; sealed | 64.0 | ||||
| NCRI 15.2; no filler; sealed | 64.0 | ||||
| NCRI 15.2; no filler; sealed | 63.8 | ||||
| NCRI 15.2; no filler; sealed | 63.6 | ||||
| NCRI 15.2; no filler; sealed | 63.5 | ||||
| GLM-4.6V | NCRI 15.2; no filler; sealed | 63.4 | |||
| NCRI 15.2; no filler; sealed | 63.3 | ||||
| NCRI 15.2; no filler; sealed | 63.3 | ||||
| NCRI 15.2; no filler; sealed | 63.1 | ||||
| Hermes 4 405B | NCRI 15.2; no filler; sealed | 62.8 | |||
| NCRI 15.2; no filler; sealed | 62.6 | ||||
| NCRI 15.2; no filler; sealed | 62.5 | ||||
| GLM-4.5-Air | NCRI 15.2; no filler; sealed | 62.4 | |||
| NCRI 15.2; no filler; sealed | 62.1 | ||||
| NCRI 15.2; no filler; sealed | 62.1 | ||||
| NCRI 15.2; no filler; sealed | 61.6 | ||||
| NCRI 15.2; no filler; sealed | 61.4 | ||||
| NCRI 15.2; no filler; sealed | 61.1 | ||||
| NCRI 15.2; no filler; sealed | 61.0 | ||||
| NCRI 15.2; no filler; sealed | 60.1 | ||||
| NCRI 15.2; no filler; sealed | 60.0 | ||||
| NCRI 15.2; no filler; sealed | 59.9 | ||||
| NCRI 15.2; no filler; sealed | 59.7 | ||||
| NCRI 15.2; no filler; sealed | 59.6 | ||||
| NCRI 15.2; no filler; sealed | 59.5 | ||||
| Ripple-65B Preview | NCRI 15.2; no filler; placed_after_release | 59.3 | |||
| NCRI 15.2; no filler; sealed | 59.2 | ||||
| gpt-oss-120b | NCRI 15.2; no filler; sealed | 59.0 | |||
| NCRI 15.2; no filler; sealed | 58.3 | ||||
| gpt-oss-120b | NCRI 15.2; no filler; sealed | 58.1 | |||
| NCRI 15.2; no filler; sealed | 57.7 | ||||
| NCRI 15.2; no filler; sealed | 57.6 | ||||
| NCRI 15.2; no filler; sealed | 57.5 | ||||
| NCRI 15.2; no filler; sealed | 57.5 | ||||
| Claude 3 Haiku | NCRI 15.2; no filler; sealed | 57.3 | |||
| NCRI 15.2; no filler; sealed | 56.1 | ||||
| NCRI 15.2; no filler; sealed | 56.0 | ||||
| NCRI 15.2; no filler; sealed | 56.0 | ||||
| glm-4.7-flash | NCRI 15.2; no filler; sealed | 55.9 | |||
| NCRI 15.2; no filler; sealed | 55.8 | ||||
| GPT-4o mini | NCRI 15.2; no filler; sealed | 55.7 | |||
| Mercury 2 | NCRI 15.2; no filler; sealed | 55.1 | |||
| NCRI 15.2; no filler; sealed | 55.0 | ||||
| gpt-oss-20b | NCRI 15.2; no filler; sealed | 54.8 | |||
| NCRI 15.2; no filler; sealed | 54.8 | ||||
| NCRI 15.2; no filler; placed | 54.3 | ||||
| NCRI 15.2; no filler; sealed | 54.2 | ||||
| NCRI 15.2; no filler; sealed | 54.1 | ||||
| NCRI 15.2; no filler; sealed | 53.3 | ||||
| NCRI 15.2; no filler; sealed | 52.4 | ||||
| gpt-oss-20b | NCRI 15.2; no filler; sealed | 51.5 | |||
| NCRI 15.2; no filler; sealed | 51.4 | ||||
| GPT-5 mini | NCRI 15.2; no filler; sealed | 50.7 | |||
| NCRI 15.2; no filler; sealed | 50.6 | ||||
| NCRI 15.2; no filler; sealed | 50.4 | ||||
| GPT-4.1 nano | NCRI 15.2; no filler; sealed | 50.2 | |||
| NCRI 15.2; no filler; sealed | 49.9 | ||||
| NCRI 15.2; no filler; sealed | 49.7 | ||||
| GPT-3.5 Turbo Instruct | NCRI 15.2; no filler; sealed | 49.6 | |||
| GPT-5.4 Nano | NCRI 15.2; no filler; sealed | 49.4 | |||
| NCRI 15.2; no filler; sealed | 49.3 | ||||
| NCRI 15.2; no filler; sealed | 49.0 | ||||
| NCRI 15.2; no filler; sealed | 48.7 | ||||
| NCRI 15.2; no filler; sealed | 47.3 | ||||
| NCRI 15.2; no filler; sealed | 46.8 | ||||
| NCRI 15.2; no filler; sealed | 46.0 | ||||
| NCRI 15.2; no filler; sealed | 44.7 | ||||
| NCRI 15.2; no filler; sealed | 44.3 | ||||
| NCRI 15.2; no filler; sealed | 44.0 | ||||
| NCRI 15.2; no filler; sealed | 43.9 | ||||
| NCRI 15.2; no filler; sealed | 43.2 | ||||
| NCRI 15.2; no filler; sealed | 42.5 | ||||
| NCRI 15.2; no filler; sealed | 40.6 | ||||
| NCRI 15.2; no filler; sealed | 38.5 | ||||
| NCRI 15.2; no filler; sealed | 37.9 | ||||
| NCRI 15.2; no filler; sealed | 37.5 | ||||
| NCRI 15.2; no filler; sealed | 37.4 | ||||
| NCRI 15.2; no filler; sealed | 35.5 | ||||
| NCRI 15.2; no filler; sealed | 35.3 | ||||
| davinci-002 | NCRI 15.2; no filler; sealed | 32.6 | |||
| GPT-5 nano | NCRI 15.2; no filler; sealed | 31.0 | |||
| NCRI 15.2; no filler; sealed | 30.0 | ||||
| NCRI 15.2; no filler; sealed | 24.9 | ||||
| NCRI 15.2; no filler; sealed | 24.1 | ||||
| NCRI 15.2; no filler; sealed | 16.1 | ||||
| NCRI 15.2; no filler; sealed | 15.0 | ||||
| NCRI 15.2; no filler; sealed | -23.1 |
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: Official nocot-bench (NCRI) leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| GPT-6 Astra | NCRI 15.2; dots@2048; placed | 173.9 | |||
| GPT-6 Astra | NCRI 15.2; no filler; placed | 159.0 | |||
| GPT-6.1 Sol | NCRI 15.2; dots@128; placed_after_release | 156.6 | |||
| NCRI 15.2; qrep@512; placed_after_release | 154.1 | ||||
| NCRI 15.2; no filler; placed_after_release | 151.2 | ||||
| GPT-6.1 Terra | NCRI 15.2; qrep@512; placed_after_release | 149.6 | |||
| GPT-6.1 Sol | NCRI 15.2; no filler; placed_after_release | 149.0 | |||
| GPT-6.1 Terra | NCRI 15.2; no filler; placed_after_release | 130.5 | |||
| NCRI 15.2; dots@128; placed | 129.7 | ||||
| Claude Fable 5.1 | NCRI 15.2; dots@128; placed | 129.6 | |||
| Claude Fable 5.1 | NCRI 15.2; no filler; placed | 128.0 | |||
| NCRI 15.2; dots@32; sealed | 127.6 | ||||
| NCRI 15.2; no filler; sealed | 126.1 | ||||
| NCRI 15.2; no filler; placed | 125.8 | ||||
| NCRI 15.2; count@512; sealed | 123.9 | ||||
| GPT-6 Sol | NCRI 15.2; dots@128; placed_after_release | 122.2 | |||
| Claude Fable 5 | NCRI 15.2; dots@32; sealed | 121.5 | |||
| GPT-5.6 Sol | NCRI 15.2; qrep@512; sealed | 119.5 | |||
| Claude Fable 5 | NCRI 15.2; no filler; sealed | 119.0 | |||
| Claude Opus 5.5 | NCRI 15.2; dots@512; placed_after_release | 118.9 | |||
| Claude Opus 5 | NCRI 15.2; dots@128; sealed | 115.8 | |||
| NCRI 15.2; no filler; sealed | 115.3 | ||||
| Claude Opus 5 | NCRI 15.2; no filler; sealed | 115.1 | |||
| GPT-6 Sol | NCRI 15.2; no filler; placed_after_release | 114.6 | |||
| GPT-5.5 | NCRI 15.2; qrep@512; sealed | 111.0 | |||
| NCRI 15.2; count@512; sealed | 110.9 | ||||
| Claude Opus 4.6 | NCRI 15.2; qrep@512; sealed | 110.8 | |||
| NCRI 15.2; count@512; sealed | 110.5 | ||||
| Claude Opus 4.5 | NCRI 15.2; dots@512; sealed | 110.5 | |||
| GPT-5.6 Sol | NCRI 15.2; no filler; sealed | 109.4 | |||
| Claude Opus 4 | NCRI 15.2; dots@512; sealed | 109.0 | |||
| NCRI 15.2; qrep@512; sealed | 108.8 | ||||
| Claude Opus 4.1 | NCRI 15.2; dots@128; sealed | 107.5 | |||
| NCRI 15.2; dots@512; sealed | 107.5 | ||||
| Kimi K3 | NCRI 15.2; qrep@512; sealed | 105.6 | |||
| Claude Opus 4.6 | NCRI 15.2; no filler; sealed | 104.3 | |||
| Claude Opus 4.7 | NCRI 15.2; dots@128; sealed | 103.9 | |||
| Claude Opus 4.7 | NCRI 15.2; no filler; sealed | 103.7 | |||
| NCRI 15.2; no filler; sealed | 103.2 | ||||
| Kimi K3.1 Pro | NCRI 15.2; no filler; placed_after_release | 102.8 | |||
| Claude Opus 4.5 | NCRI 15.2; no filler; sealed | 102.7 | |||
| Claude Opus 4.1 | NCRI 15.2; no filler; sealed | 102.6 | |||
| NCRI 15.2; dots@128; sealed | 102.3 | ||||
| Claude Opus 4.8 | NCRI 15.2; dots@128; sealed | 102.2 | |||
| Claude Opus 4.8 | NCRI 15.2; no filler; sealed | 102.0 | |||
| NCRI 15.2; no filler; sealed | 101.5 | ||||
| Claude Opus 4 | NCRI 15.2; no filler; sealed | 101.4 | |||
| NCRI 15.2; no filler; sealed | 101.0 | ||||
| NCRI 15.2; dots@2048; placed | 100.8 | ||||
| Claude Sonnet 4.5 | NCRI 15.2; qrep@512; sealed | 100.8 | |||
| GPT-5.5 | NCRI 15.2; no filler; sealed | 100.2 | |||
| NCRI 15.2; no filler; placed | 98.3 | ||||
| NCRI 15.2; no filler; placed_after_release | 97.7 | ||||
| NCRI 15.2; no filler; sealed | 97.1 | ||||
| NCRI 15.2; no filler; sealed | 96.8 | ||||
| Kimi K3 | NCRI 15.2; no filler; sealed | 96.8 | |||
| Claude Sonnet 4 | NCRI 15.2; count@512; sealed | 96.5 | |||
| NCRI 15.2; no filler; placed_after_release | 95.7 | ||||
| GPT-5.6 Terra | NCRI 15.2; qrep@512; sealed | 95.1 | |||
| Claude Sonnet 5.5 | NCRI 15.2; no filler; placed_after_release | 95.1 | |||
| GPT-5.4 | NCRI 15.2; dots@128; sealed | 94.1 | |||
| NCRI 15.2; no filler; placed_after_release | 94.0 | ||||
| GPT-5.4 | NCRI 15.2; no filler; sealed | 94.0 | |||
| Claude Sonnet 4.5 | NCRI 15.2; no filler; sealed | 94.0 | |||
| Claude Sonnet 4 | NCRI 15.2; no filler; sealed | 91.3 | |||
| Claude Sonnet 5 | NCRI 15.2; dots@128; sealed | 91.1 | |||
| Claude Sonnet 5 | NCRI 15.2; no filler; sealed | 90.8 | |||
| Claude Sonnet 4.6 | NCRI 15.2; dots@512; sealed | 90.0 | |||
| GPT-5.6 Terra | NCRI 15.2; no filler; sealed | 89.0 | |||
| Claude Sonnet 4.6 | NCRI 15.2; no filler; sealed | 87.8 | |||
| NCRI 15.2; no filler; sealed | 87.0 | ||||
| Grok 4.6 | NCRI 15.2; no filler; sealed | 86.8 | |||
| NCRI 15.2; no filler; sealed | 86.5 | ||||
| NCRI 15.2; dots@128; sealed | 86.4 | ||||
| NCRI 15.2; no filler; sealed | 86.0 | ||||
| GPT-5.2 | NCRI 15.2; no filler; sealed | 85.8 | |||
| NCRI 15.2; no filler; sealed | 85.8 | ||||
| NCRI 15.2; no filler; sealed | 85.4 | ||||
| GPT-5.3 Codex | NCRI 15.2; no filler; sealed | 85.2 | |||
| NCRI 15.2; no filler; sealed | 84.1 | ||||
| NCRI 15.2; no filler; sealed | 84.1 | ||||
| NCRI 15.2; no filler; sealed | 83.3 | ||||
| NCRI 15.2; no filler; sealed | 83.1 | ||||
| NCRI 15.2; no filler; sealed | 82.8 | ||||
| NCRI 15.2; no filler; sealed | 82.5 | ||||
| NCRI 15.2; no filler; sealed | 82.2 | ||||
| NCRI 15.2; no filler; sealed | 81.9 | ||||
| NCRI 15.2; no filler; sealed | 81.7 | ||||
| NCRI 15.2; no filler; placed_after_release | 81.6 | ||||
| NCRI 15.2; no filler; placed_after_release | 81.3 | ||||
| NCRI 15.2; no filler; sealed | 81.0 | ||||
| Grok 4.5 | NCRI 15.2; no filler; sealed | 81.0 | |||
| NCRI 15.2; no filler; sealed | 80.6 | ||||
| NCRI 15.2; no filler; sealed | 80.6 | ||||
| Kimi K2.5 | NCRI 15.2; no filler; sealed | 80.6 | |||
| NCRI 15.2; no filler; sealed | 80.0 | ||||
| Kimi K2.6 | NCRI 15.2; no filler; sealed | 79.2 | |||
| NCRI 15.2; no filler; sealed | 79.1 | ||||
| NCRI 15.2; no filler; sealed | 78.3 | ||||
| NCRI 15.2; no filler; sealed | 78.1 | ||||
| NCRI 15.2; no filler; sealed | 77.6 | ||||
| GLM-5.2 | NCRI 15.2; no filler; sealed | 77.3 | |||
| GPT-6 Luna | NCRI 15.2; no filler; placed_after_release | 77.3 | |||
| NCRI 15.2; no filler; sealed | 77.3 | ||||
| MiMo-V2.5-Pro | NCRI 15.2; no filler; sealed | 77.1 | |||
| NCRI 15.2; no filler; sealed | 76.7 | ||||
| GPT-5.6 Luna | NCRI 15.2; no filler; sealed | 76.6 | |||
| GLM-5.3-Flash | NCRI 15.2; no filler; sealed | 76.4 | |||
| GLM-5 | NCRI 15.2; no filler; sealed | 76.2 | |||
| NCRI 15.2; no filler; sealed | 75.5 | ||||
| GLM-5.1 | NCRI 15.2; no filler; sealed | 75.2 | |||
| GPT-4.1 | NCRI 15.2; no filler; sealed | 75.1 | |||
| NCRI 15.2; no filler; sealed | 75.0 | ||||
| NCRI 15.2; no filler; sealed | 74.6 | ||||
| GLM-5.3 | NCRI 15.2; no filler; sealed | 74.5 | |||
| NCRI 15.2; no filler; sealed | 74.4 | ||||
| GPT-4o Aug 2024 | NCRI 15.2; no filler; sealed | 74.2 | |||
| Claude Haiku 4.5 | NCRI 15.2; no filler; sealed | 74.2 | |||
| NCRI 15.2; no filler; sealed | 73.9 | ||||
| NCRI 15.2; no filler; sealed | 73.2 | ||||
| NCRI 15.2; no filler; sealed | 73.1 | ||||
| GPT-4 Jun 2023 | NCRI 15.2; no filler; sealed | 72.8 | |||
| NCRI 15.2; no filler; sealed | 72.6 | ||||
| NCRI 15.2; no filler; sealed | 72.5 | ||||
| NCRI 15.2; no filler; sealed | 72.3 | ||||
| Mercury 3 | NCRI 15.2; no filler; placed_after_release | 72.3 | |||
| NCRI 15.2; no filler; sealed | 72.3 | ||||
| GPT-4o May 2024 | NCRI 15.2; no filler; sealed | 72.2 | |||
| GLM-4.7 | NCRI 15.2; no filler; sealed | 71.3 | |||
| GLM-4.5 | NCRI 15.2; no filler; sealed | 71.3 | |||
| Qwen3-Coder-Plus | NCRI 15.2; no filler; sealed | 71.2 | |||
| GLM-4.6 | NCRI 15.2; no filler; sealed | 70.8 | |||
| GPT-5.1 | NCRI 15.2; no filler; sealed | 70.6 | |||
| NCRI 15.2; no filler; sealed | 70.5 | ||||
| NCRI 15.2; no filler; sealed | 70.5 | ||||
| GPT-5.4 Mini | NCRI 15.2; no filler; sealed | 70.5 | |||
| GPT-4o Nov 2024 | NCRI 15.2; no filler; sealed | 70.3 | |||
| NCRI 15.2; no filler; sealed | 70.2 | ||||
| NCRI 15.2; no filler; sealed | 70.2 | ||||
| MiMo-V2.5 | NCRI 15.2; no filler; sealed | 69.9 | |||
| NCRI 15.2; no filler; sealed | 69.5 | ||||
| NCRI 15.2; no filler; sealed | 69.4 | ||||
| Grok 4.3 | NCRI 15.2; no filler; sealed | 69.2 | |||
| NCRI 15.2; no filler; sealed | 68.9 | ||||
| NCRI 15.2; no filler; sealed | 68.4 | ||||
| NCRI 15.2; no filler; placed_after_release | 68.4 | ||||
| Claude Haiku 5.5 | NCRI 15.2; no filler; placed_after_release | 68.3 | |||
| NCRI 15.2; no filler; sealed | 67.9 | ||||
| NCRI 15.2; no filler; sealed | 67.5 | ||||
| NCRI 15.2; no filler; sealed | 67.4 | ||||
| Grok 4.20 | NCRI 15.2; no filler; sealed | 67.1 | |||
| GPT-5 | NCRI 15.2; no filler; sealed | 67.0 | |||
| GLM-4.5V | NCRI 15.2; no filler; sealed | 66.7 | |||
| NCRI 15.2; no filler; sealed | 66.6 | ||||
| NCRI 15.2; no filler; sealed | 66.3 | ||||
| NCRI 15.2; no filler; sealed | 66.3 | ||||
| NCRI 15.2; no filler; sealed | 66.2 | ||||
| NCRI 15.2; no filler; sealed | 66.1 | ||||
| NCRI 15.2; no filler; sealed | 66.1 | ||||
| Mercury 2.5 | NCRI 15.2; no filler; placed | 65.8 | |||
| NCRI 15.2; no filler; sealed | 65.5 | ||||
| NCRI 15.2; no filler; sealed | 65.3 | ||||
| NCRI 15.2; no filler; sealed | 65.2 | ||||
| GPT-4.1 mini | NCRI 15.2; no filler; sealed | 65.0 | |||
| NCRI 15.2; no filler; sealed | 65.0 | ||||
| NCRI 15.2; no filler; sealed | 64.5 | ||||
| NCRI 15.2; no filler; sealed | 64.0 | ||||
| NCRI 15.2; no filler; sealed | 64.0 | ||||
| NCRI 15.2; no filler; sealed | 63.8 | ||||
| NCRI 15.2; no filler; sealed | 63.6 | ||||
| NCRI 15.2; no filler; sealed | 63.5 | ||||
| GLM-4.6V | NCRI 15.2; no filler; sealed | 63.4 | |||
| NCRI 15.2; no filler; sealed | 63.3 | ||||
| NCRI 15.2; no filler; sealed | 63.3 | ||||
| NCRI 15.2; no filler; sealed | 63.1 | ||||
| Hermes 4 405B | NCRI 15.2; no filler; sealed | 62.8 | |||
| NCRI 15.2; no filler; sealed | 62.6 | ||||
| NCRI 15.2; no filler; sealed | 62.5 | ||||
| GLM-4.5-Air | NCRI 15.2; no filler; sealed | 62.4 | |||
| NCRI 15.2; no filler; sealed | 62.1 | ||||
| NCRI 15.2; no filler; sealed | 62.1 | ||||
| NCRI 15.2; no filler; sealed | 61.6 | ||||
| NCRI 15.2; no filler; sealed | 61.4 | ||||
| NCRI 15.2; no filler; sealed | 61.1 | ||||
| NCRI 15.2; no filler; sealed | 61.0 | ||||
| NCRI 15.2; no filler; sealed | 60.1 | ||||
| NCRI 15.2; no filler; sealed | 60.0 | ||||
| NCRI 15.2; no filler; sealed | 59.9 | ||||
| NCRI 15.2; no filler; sealed | 59.7 | ||||
| NCRI 15.2; no filler; sealed | 59.6 | ||||
| NCRI 15.2; no filler; sealed | 59.5 | ||||
| Ripple-65B Preview | NCRI 15.2; no filler; placed_after_release | 59.3 | |||
| NCRI 15.2; no filler; sealed | 59.2 | ||||
| gpt-oss-120b | NCRI 15.2; no filler; sealed | 59.0 | |||
| NCRI 15.2; no filler; sealed | 58.3 | ||||
| gpt-oss-120b | NCRI 15.2; no filler; sealed | 58.1 | |||
| NCRI 15.2; no filler; sealed | 57.7 | ||||
| NCRI 15.2; no filler; sealed | 57.6 | ||||
| NCRI 15.2; no filler; sealed | 57.5 | ||||
| NCRI 15.2; no filler; sealed | 57.5 | ||||
| Claude 3 Haiku | NCRI 15.2; no filler; sealed | 57.3 | |||
| NCRI 15.2; no filler; sealed | 56.1 | ||||
| NCRI 15.2; no filler; sealed | 56.0 | ||||
| NCRI 15.2; no filler; sealed | 56.0 | ||||
| glm-4.7-flash | NCRI 15.2; no filler; sealed | 55.9 | |||
| NCRI 15.2; no filler; sealed | 55.8 | ||||
| GPT-4o mini | NCRI 15.2; no filler; sealed | 55.7 | |||
| Mercury 2 | NCRI 15.2; no filler; sealed | 55.1 | |||
| NCRI 15.2; no filler; sealed | 55.0 | ||||
| gpt-oss-20b | NCRI 15.2; no filler; sealed | 54.8 | |||
| NCRI 15.2; no filler; sealed | 54.8 | ||||
| NCRI 15.2; no filler; placed | 54.3 | ||||
| NCRI 15.2; no filler; sealed | 54.2 | ||||
| NCRI 15.2; no filler; sealed | 54.1 | ||||
| NCRI 15.2; no filler; sealed | 53.3 | ||||
| NCRI 15.2; no filler; sealed | 52.4 | ||||
| gpt-oss-20b | NCRI 15.2; no filler; sealed | 51.5 | |||
| NCRI 15.2; no filler; sealed | 51.4 | ||||
| GPT-5 mini | NCRI 15.2; no filler; sealed | 50.7 | |||
| NCRI 15.2; no filler; sealed | 50.6 | ||||
| NCRI 15.2; no filler; sealed | 50.4 | ||||
| GPT-4.1 nano | NCRI 15.2; no filler; sealed | 50.2 | |||
| NCRI 15.2; no filler; sealed | 49.9 | ||||
| NCRI 15.2; no filler; sealed | 49.7 | ||||
| GPT-3.5 Turbo Instruct | NCRI 15.2; no filler; sealed | 49.6 | |||
| GPT-5.4 Nano | NCRI 15.2; no filler; sealed | 49.4 | |||
| NCRI 15.2; no filler; sealed | 49.3 | ||||
| NCRI 15.2; no filler; sealed | 49.0 | ||||
| NCRI 15.2; no filler; sealed | 48.7 | ||||
| NCRI 15.2; no filler; sealed | 47.3 | ||||
| NCRI 15.2; no filler; sealed | 46.8 | ||||
| NCRI 15.2; no filler; sealed | 46.0 | ||||
| NCRI 15.2; no filler; sealed | 44.7 | ||||
| NCRI 15.2; no filler; sealed | 44.3 | ||||
| NCRI 15.2; no filler; sealed | 44.0 | ||||
| NCRI 15.2; no filler; sealed | 43.9 | ||||
| NCRI 15.2; no filler; sealed | 43.2 | ||||
| NCRI 15.2; no filler; sealed | 42.5 | ||||
| NCRI 15.2; no filler; sealed | 40.6 | ||||
| NCRI 15.2; no filler; sealed | 38.5 | ||||
| NCRI 15.2; no filler; sealed | 37.9 | ||||
| NCRI 15.2; no filler; sealed | 37.5 | ||||
| NCRI 15.2; no filler; sealed | 37.4 | ||||
| NCRI 15.2; no filler; sealed | 35.5 | ||||
| NCRI 15.2; no filler; sealed | 35.3 | ||||
| davinci-002 | NCRI 15.2; no filler; sealed | 32.6 | |||
| GPT-5 nano | NCRI 15.2; no filler; sealed | 31.0 | |||
| NCRI 15.2; no filler; sealed | 30.0 | ||||
| NCRI 15.2; no filler; sealed | 24.9 | ||||
| NCRI 15.2; no filler; sealed | 24.1 | ||||
| NCRI 15.2; no filler; sealed | 16.1 | ||||
| NCRI 15.2; no filler; sealed | 15.0 | ||||
| NCRI 15.2; no filler; sealed | -23.1 |
3 models have held number one on the official leaderboard since its release in September 2026 3 models have held number one with claims included since its release in September 2026
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official nocot-bench (NCRI) leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official nocot-bench (NCRI) leaderboard and labs’ model cards
Labs report it only in system cards
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark. Some system cards do, below.
2 system cards report it, from 2 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Nov 17, 2026 | GPT-6.2 Astra system card | 171 | UK AISI No-CoT Reasoning Index (NCRI, Nanda) without filler tokens (4.2.2.5) | Monitorability |
| Oct 13, 2026 | | 150.3 | No Chain of Thought Reasoning Index (NCRI, Nanda 2026), 150.3 without filler tokens (152.1 with filler tokens); about 5.5 serial arithmetic steps at 50% (3.4.5.3, Figures 3.23 and 3.24) | Monitorability |
One lab claim next to the leaderboard
Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. Every benchmark.
| Model | Lab says | Board says | Gap |
|---|---|---|---|
| | 150.3 No Chain of Thought Reasoning Index (NCRI, Nanda 2026), 150.3 without filler tokens (152.1 with filler tokens); about 5.5 serial arithmetic steps at 50% (3.4.5.3, Figures 3.23 and 3.24) | 154.1 Leaderboard, NCRI 15.2; qrep@512; placed_after_release | −3.8 |
More in Safety and alignment
- FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
- StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is.
Inkling 98.6%
- Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts.
Gemini 4 Carbon 0%
- Covert misbehavior (Thinking Machines) In use Agentic tasks that check whether a model misbehaves and conceals it. Scored by the rate of covert misbehavior.
Inkling 2 Mini 8.30%
- Destructive actions (Thinking Machines) In use Agentic tasks that check whether a model takes destructive actions it was not authorized to take. Scored by the rate of destructive actions.
Inkling 2 Small 4.70%
- Misbehavior (Thinking Machines) In use Agentic tasks that check whether a model fabricates results, ignores instructions or hides what it did when a task is hard. Scored by the rate of misbehavior.
Inkling 2 Mini 9.80%
Data audit
All 11 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Pass Independent runs Scores from the official leaderboard (287 entries) and system cards (2 scores).
- Pass Models scored 202 models we track, plus 86 other entries on its leaderboard.
- Pass Newest score The newest is from November 2026, for GPT-6.2 Astra.
- Pass Frontier models 8 of the top 10 have a score, led by GPT-6.2 Astra (#1).
- Note Headroom Scored as an index, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on github.com: 253 rows for models we track and 86 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 2 pass
- Pass Lab claims The one pair agrees within 10 points.
- Pass Known issues No published audit found problems.