Benchmark Almanac

CircuitRubric

Claude Opus 4.8 96% leads since June 2026
Not in model cards

Models write analog circuit netlists from design requests. A program compares transistor connections and relative device sizes with accepted reference circuits.

The maker compares 35 model labels on 125 circuits. Prompt detail and reasoning settings vary. Full credit requires matching the circuit structure and sizing ratios.

Released
June 2026
Kind
Electronics
Level
University
Scored
Checked automatically
Answers
Code
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Code
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
Strongest model tested
GPT-5.5 #21 on the capability index
Last model added
June 2026
Scores refreshed
November 20th, 2026
35
Entries on its own leaderboard, 28 models we track

Claude Opus 4.8 leads with 96% among models released by June 2026 Claude Opus 4.8 leads with 96% among models released by June 2026

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

The newest model with a score here came out in June 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 Claude Opus 4.8 spec, conventions, run fe-opus-spec-conventions 96%
  2. 2 GPT-5.5 spec, strict_ports, run or-gpt-5.5-closed-spec-strict_ports 95.2%
  3. 3 Qwen3.7-Plus spec, strict_ports, run or-qwen3.7-plus-2026-spec-strict_ports 95.2%
  4. 4 Kimi K2.6 spec, strict_ports, run or-kimi-k2.6-2026-spec-strict_ports 94.4%
  5. 5 Claude Sonnet 4.6 spec, conventions, run fe-sonnet-spec-conventions 92.8%
  6. 6 DeepSeek-V4-Pro spec, strict_ports, run or-deepseek-v4-pro-2026-spec-strict_ports 92%
  7. 7 Qwen3.7-Max spec, strict_ports, run or-qwen3.7-max-2026-spec-strict_ports 92%
  8. 8 GLM-5.1 spec, strict_ports, run or-glm-5.1-2026-spec-strict_ports 91.2%
  9. 9 Gemini 3.1 Pro short, conventions, run 20260623-020043-gemini-3.1-pro 89.6%
  10. 10 MiniMax-M3 spec, strict_ports, run or-minimax-m3-2026-spec-strict_ports 89.6%
  11. 11 Claude Haiku 4.5 spec, strict_ports, run fe-haiku-spec-strict_ports 88%
  12. 12 Llama 4 Maverick spec, strict_ports, run or-llama-4-maverick-2026-spec-strict_ports 85.6%
  13. 13 GLM-4.5 spec, strict_ports, run or-glm-4.5-2025-spec-strict_ports 85.6%
  14. 14 DeepSeek-V4-Flash spec, strict_ports, run or-deepseek-v4-flash-2026-spec-strict_ports 85.6%
  15. 15 Llama 3.3 70B Instruct spec, strict_ports, run or-llama-3.3-70b-2025-spec-strict_ports 84.8%
  16. 16 Kimi K2 Sep 2025, spec, strict_ports, run or-kimi-k2-2025-spec-strict_ports 82.4%
  17. 17 gpt-oss-20b spec, strict_ports, run lb-gptoss-20b-spec-strict_ports 78.4%
  18. 18 deepseek/deepseek-chat-v3-0324 spec, strict_ports, run or-deepseek-v3-2025-spec-strict_ports 76.8%
  19. 19 Claude Opus 4.7 short, topology_ports, high, run fe-opus47-high-short-topology_ports 71.2%
  20. 20 qwen3.5:latest spec, strict_ports, run 20260613-045341-qwen3.5 68.8%
  21. 21 phi4:14b spec, strict_ports, run lb-phi4-spec-strict_ports 68%
  22. 22 Qwen2.5-Coder (7B) spec, topology_ports, run 20260613-042138-qwen2.5-coder 66.4%
  23. 23 qwen3:30b-a3b-instruct-2507-q4_K_M spec, strict_ports, run 30b-spec-strict_ports 64%
  24. 24 Qwen3-235B-A22B-Instruct Jul 2025, spec, strict_ports, run or-qwen3-235b-spec-strict_ports 59.2%
  25. 25 Gemini 3.5 Flash short, topology_ports, run 20260622-224557-gemini-3.5-flash 58.4%
  26. 26 Qwen3-14B spec, topology, run lb-qwen3-14b-spec-topology 54.4%
  27. 27 deepseek-coder-v2:16b spec, strict_ports, run lb-dscoder-v2-spec-strict_ports 45.6%
  28. 28 GLM-5.2 short, topology_ports, run or-glm-5.2-short-topology_ports 44.8%
  29. 29 Qwen2.5-Coder-14B spec, strict_ports, run lb-coder14b-spec-strict_ports 42.4%
  30. 30 Gemma 3 12B spec, strict_ports, run lb-gemma3-spec-strict_ports 34.4%
  31. 31 Llama 3.1-8B spec, strict_ports, run 20260613-045341-llama3.1 27.2%
  32. 32 Gemma 2 9B spec, topology_ports, run lb-gemma2-spec-topology_ports 24%
  33. 33 granite3.1-dense:8b spec, strict_ports, run 20260613-045341-granite3.1 16%
  34. 34 Gemma 4 31B IT short, topology_ports, run 20260622-224557-gemma-4-31b 12.8%
  35. 35 deepseek-r1:14b short, strict_ports, run lb-thinkfalse-deepseek-r1-14b-short-strict_ports 0.80%

Source: Official CircuitRubric leaderboard

  1. 1 Claude Opus 4.8 spec, conventions, run fe-opus-spec-conventions 96%
  2. 2 GPT-5.5 spec, strict_ports, run or-gpt-5.5-closed-spec-strict_ports 95.2%
  3. 3 Qwen3.7-Plus spec, strict_ports, run or-qwen3.7-plus-2026-spec-strict_ports 95.2%
  4. 4 Kimi K2.6 spec, strict_ports, run or-kimi-k2.6-2026-spec-strict_ports 94.4%
  5. 5 Claude Sonnet 4.6 spec, conventions, run fe-sonnet-spec-conventions 92.8%
  6. 6 DeepSeek-V4-Pro spec, strict_ports, run or-deepseek-v4-pro-2026-spec-strict_ports 92%
  7. 7 Qwen3.7-Max spec, strict_ports, run or-qwen3.7-max-2026-spec-strict_ports 92%
  8. 8 GLM-5.1 spec, strict_ports, run or-glm-5.1-2026-spec-strict_ports 91.2%
  9. 9 Gemini 3.1 Pro short, conventions, run 20260623-020043-gemini-3.1-pro 89.6%
  10. 10 MiniMax-M3 spec, strict_ports, run or-minimax-m3-2026-spec-strict_ports 89.6%
  11. 11 Claude Haiku 4.5 spec, strict_ports, run fe-haiku-spec-strict_ports 88%
  12. 12 Llama 4 Maverick spec, strict_ports, run or-llama-4-maverick-2026-spec-strict_ports 85.6%
  13. 13 GLM-4.5 spec, strict_ports, run or-glm-4.5-2025-spec-strict_ports 85.6%
  14. 14 DeepSeek-V4-Flash spec, strict_ports, run or-deepseek-v4-flash-2026-spec-strict_ports 85.6%
  15. 15 Llama 3.3 70B Instruct spec, strict_ports, run or-llama-3.3-70b-2025-spec-strict_ports 84.8%
  16. 16 Kimi K2 Sep 2025, spec, strict_ports, run or-kimi-k2-2025-spec-strict_ports 82.4%
  17. 17 gpt-oss-20b spec, strict_ports, run lb-gptoss-20b-spec-strict_ports 78.4%
  18. 18 Claude Opus 4.7 short, topology_ports, high, run fe-opus47-high-short-topology_ports 71.2%
  19. 19 Qwen2.5-Coder (7B) spec, topology_ports, run 20260613-042138-qwen2.5-coder 66.4%
  20. 20 Qwen3-235B-A22B-Instruct Jul 2025, spec, strict_ports, run or-qwen3-235b-spec-strict_ports 59.2%
  21. 21 Gemini 3.5 Flash short, topology_ports, run 20260622-224557-gemini-3.5-flash 58.4%
  22. 22 Qwen3-14B spec, topology, run lb-qwen3-14b-spec-topology 54.4%
  23. 23 GLM-5.2 short, topology_ports, run or-glm-5.2-short-topology_ports 44.8%
  24. 24 Qwen2.5-Coder-14B spec, strict_ports, run lb-coder14b-spec-strict_ports 42.4%
  25. 25 Gemma 3 12B spec, strict_ports, run lb-gemma3-spec-strict_ports 34.4%
  26. 26 Llama 3.1-8B spec, strict_ports, run 20260613-045341-llama3.1 27.2%
  27. 27 Gemma 2 9B spec, topology_ports, run lb-gemma2-spec-topology_ports 24%
  28. 28 Gemma 4 31B IT short, topology_ports, run 20260622-224557-gemma-4-31b 12.8%

Source: Official CircuitRubric leaderboard

#EntryScoreSettingDateSource
1 Claude Opus 4.8 96% spec, conventions, run fe-opus-spec-conventions, anthropic May 2026 Official leaderboard
2 GPT-5.5 95.2% spec, strict_ports, run or-gpt-5.5-closed-spec-strict_ports, openai Apr 2026 Official leaderboard
3 Qwen3.7-Plus 95.2% spec, strict_ports, run or-qwen3.7-plus-2026-spec-strict_ports, openai Jun 2026 Official leaderboard
4 Kimi K2.6 94.4% spec, strict_ports, run or-kimi-k2.6-2026-spec-strict_ports, openai Apr 2026 Official leaderboard
5 Claude Sonnet 4.6 92.8% spec, conventions, run fe-sonnet-spec-conventions, anthropic Feb 2026 Official leaderboard
6 DeepSeek-V4-Pro 92% spec, strict_ports, run or-deepseek-v4-pro-2026-spec-strict_ports, openai Apr 2026 Official leaderboard
7 Qwen3.7-Max 92% spec, strict_ports, run or-qwen3.7-max-2026-spec-strict_ports, openai May 2026 Official leaderboard
8 GLM-5.1 91.2% spec, strict_ports, run or-glm-5.1-2026-spec-strict_ports, openai Apr 2026 Official leaderboard
9 Gemini 3.1 Pro 89.6% short, conventions, run 20260623-020043-gemini-3.1-pro, openai Feb 2026 Official leaderboard
10 MiniMax-M3 89.6% spec, strict_ports, run or-minimax-m3-2026-spec-strict_ports, openai Jun 2026 Official leaderboard
11 Claude Haiku 4.5 88% spec, strict_ports, run fe-haiku-spec-strict_ports, anthropic Oct 2025 Official leaderboard
12 Llama 4 Maverick 85.6% spec, strict_ports, run or-llama-4-maverick-2026-spec-strict_ports, openai Apr 2025 Official leaderboard
13 GLM-4.5 85.6% spec, strict_ports, run or-glm-4.5-2025-spec-strict_ports, openai Aug 2025 Official leaderboard
14 DeepSeek-V4-Flash 85.6% spec, strict_ports, run or-deepseek-v4-flash-2026-spec-strict_ports, openai Apr 2026 Official leaderboard
15 Llama 3.3 70B Instruct 84.8% spec, strict_ports, run or-llama-3.3-70b-2025-spec-strict_ports, openai Dec 2024 Official leaderboard
16 Kimi K2 Sep 2025 82.4% spec, strict_ports, run or-kimi-k2-2025-spec-strict_ports, openai Sep 2025 Official leaderboard
17 gpt-oss-20b 78.4% spec, strict_ports, run lb-gptoss-20b-spec-strict_ports, ollama Aug 2025 Official leaderboard
18 deepseek/deepseek-chat-v3-0324 76.8% spec, strict_ports, run or-deepseek-v3-2025-spec-strict_ports, openai Official leaderboard
19 Claude Opus 4.7 71.2% short, topology_ports, high, run fe-opus47-high-short-topology_ports, anthropic Apr 2026 Official leaderboard
20 qwen3.5:latest 68.8% spec, strict_ports, run 20260613-045341-qwen3.5, ollama Official leaderboard
21 phi4:14b 68% spec, strict_ports, run lb-phi4-spec-strict_ports, ollama Official leaderboard
22 Qwen2.5-Coder (7B) 66.4% spec, topology_ports, run 20260613-042138-qwen2.5-coder, ollama Sep 2024 Official leaderboard
23 qwen3:30b-a3b-instruct-2507-q4_K_M 64% spec, strict_ports, run 30b-spec-strict_ports, ollama Official leaderboard
24 Qwen3-235B-A22B-Instruct Jul 2025 59.2% spec, strict_ports, run or-qwen3-235b-spec-strict_ports, openai Jul 2025 Official leaderboard
25 Gemini 3.5 Flash 58.4% short, topology_ports, run 20260622-224557-gemini-3.5-flash, openai May 2026 Official leaderboard
26 Qwen3-14B 54.4% spec, topology, run lb-qwen3-14b-spec-topology, ollama Apr 2025 Official leaderboard
27 deepseek-coder-v2:16b 45.6% spec, strict_ports, run lb-dscoder-v2-spec-strict_ports, ollama Official leaderboard
28 GLM-5.2 44.8% short, topology_ports, run or-glm-5.2-short-topology_ports, openai Jun 2026 Official leaderboard
29 Qwen2.5-Coder-14B 42.4% spec, strict_ports, run lb-coder14b-spec-strict_ports, ollama Sep 2024 Official leaderboard
30 Gemma 3 12B 34.4% spec, strict_ports, run lb-gemma3-spec-strict_ports, ollama Mar 2025 Official leaderboard
31 Llama 3.1-8B 27.2% spec, strict_ports, run 20260613-045341-llama3.1, ollama Jul 2024 Official leaderboard
32 Gemma 2 9B 24% spec, topology_ports, run lb-gemma2-spec-topology_ports, ollama Jun 2024 Official leaderboard
33 granite3.1-dense:8b 16% spec, strict_ports, run 20260613-045341-granite3.1, ollama Official leaderboard
34 Gemma 4 31B IT 12.8% short, topology_ports, run 20260622-224557-gemma-4-31b, openai Apr 2026 Official leaderboard
35 deepseek-r1:14b 0.80% short, strict_ports, run lb-thinkfalse-deepseek-r1-14b-short-strict_ports, ollama Official leaderboard
#ModelScoreSettingDateSource
1 Claude Opus 4.8 96% spec, conventions, run fe-opus-spec-conventions, anthropic May 2026 Official leaderboard
2 GPT-5.5 95.2% spec, strict_ports, run or-gpt-5.5-closed-spec-strict_ports, openai Apr 2026 Official leaderboard
3 Qwen3.7-Plus 95.2% spec, strict_ports, run or-qwen3.7-plus-2026-spec-strict_ports, openai Jun 2026 Official leaderboard
4 Kimi K2.6 94.4% spec, strict_ports, run or-kimi-k2.6-2026-spec-strict_ports, openai Apr 2026 Official leaderboard
5 Claude Sonnet 4.6 92.8% spec, conventions, run fe-sonnet-spec-conventions, anthropic Feb 2026 Official leaderboard
6 DeepSeek-V4-Pro 92% spec, strict_ports, run or-deepseek-v4-pro-2026-spec-strict_ports, openai Apr 2026 Official leaderboard
7 Qwen3.7-Max 92% spec, strict_ports, run or-qwen3.7-max-2026-spec-strict_ports, openai May 2026 Official leaderboard
8 GLM-5.1 91.2% spec, strict_ports, run or-glm-5.1-2026-spec-strict_ports, openai Apr 2026 Official leaderboard
9 Gemini 3.1 Pro 89.6% short, conventions, run 20260623-020043-gemini-3.1-pro, openai Feb 2026 Official leaderboard
10 MiniMax-M3 89.6% spec, strict_ports, run or-minimax-m3-2026-spec-strict_ports, openai Jun 2026 Official leaderboard
11 Claude Haiku 4.5 88% spec, strict_ports, run fe-haiku-spec-strict_ports, anthropic Oct 2025 Official leaderboard
12 Llama 4 Maverick 85.6% spec, strict_ports, run or-llama-4-maverick-2026-spec-strict_ports, openai Apr 2025 Official leaderboard
13 GLM-4.5 85.6% spec, strict_ports, run or-glm-4.5-2025-spec-strict_ports, openai Aug 2025 Official leaderboard
14 DeepSeek-V4-Flash 85.6% spec, strict_ports, run or-deepseek-v4-flash-2026-spec-strict_ports, openai Apr 2026 Official leaderboard
15 Llama 3.3 70B Instruct 84.8% spec, strict_ports, run or-llama-3.3-70b-2025-spec-strict_ports, openai Dec 2024 Official leaderboard
16 Kimi K2 Sep 2025 82.4% spec, strict_ports, run or-kimi-k2-2025-spec-strict_ports, openai Sep 2025 Official leaderboard
17 gpt-oss-20b 78.4% spec, strict_ports, run lb-gptoss-20b-spec-strict_ports, ollama Aug 2025 Official leaderboard
18 Claude Opus 4.7 71.2% short, topology_ports, high, run fe-opus47-high-short-topology_ports, anthropic Apr 2026 Official leaderboard
19 Qwen2.5-Coder (7B) 66.4% spec, topology_ports, run 20260613-042138-qwen2.5-coder, ollama Sep 2024 Official leaderboard
20 Qwen3-235B-A22B-Instruct Jul 2025 59.2% spec, strict_ports, run or-qwen3-235b-spec-strict_ports, openai Jul 2025 Official leaderboard
21 Gemini 3.5 Flash 58.4% short, topology_ports, run 20260622-224557-gemini-3.5-flash, openai May 2026 Official leaderboard
22 Qwen3-14B 54.4% spec, topology, run lb-qwen3-14b-spec-topology, ollama Apr 2025 Official leaderboard
23 GLM-5.2 44.8% short, topology_ports, run or-glm-5.2-short-topology_ports, openai Jun 2026 Official leaderboard
24 Qwen2.5-Coder-14B 42.4% spec, strict_ports, run lb-coder14b-spec-strict_ports, ollama Sep 2024 Official leaderboard
25 Gemma 3 12B 34.4% spec, strict_ports, run lb-gemma3-spec-strict_ports, ollama Mar 2025 Official leaderboard
26 Llama 3.1-8B 27.2% spec, strict_ports, run 20260613-045341-llama3.1, ollama Jul 2024 Official leaderboard
27 Gemma 2 9B 24% spec, topology_ports, run lb-gemma2-spec-topology_ports, ollama Jun 2024 Official leaderboard
28 Gemma 4 31B IT 12.8% short, topology_ports, run 20260622-224557-gemma-4-31b, openai Apr 2026 Official leaderboard

More effort adds a median 40 points

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Opus 4.8 anthropic
short, topo… short, conv… spec, conve… short, topo… short, topo… short, stri… short, conv… short, topo…
96%
GPT-5.5 openai
short, form… short, stri… verbose, st… spec, stric… short, topo…
95.2%
Qwen3.7-Plus openai
short, stri… verbose, st… spec, stric…
95.2%
Kimi K2.6 openai
short, form… short, topo… short, stri… verbose, st… spec, stric…
94.4%
Claude Sonnet 4.6 anthropic
short, topo… short, conv… verbose, st… spec, stric…
92.8%
DeepSeek-V4-Pro openai
short, form… short, topo… short, conv… spec, stric…
92%
Qwen3.7-Max openai
short, topo… short, conv… verbose, st… spec, stric…
92%
GLM-5.1 openai
short, form… short, stri… verbose, st… spec, stric…
91.2%
Gemini 3.1 Pro openai
short, stri…
89.6%
MiniMax-M3 openai
short, conv… verbose, st… spec, stric…
89.6%
Claude Haiku 4.5 anthropic
short, topo… short, conv… verbose, co… spec, conve…
88%
Llama 4 Maverick openai
short, stri… spec, stric…
85.6%

Source: Official CircuitRubric leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 4.8 spec, conventions, run fe-opus-spec-conventions anthropic 96% 212
GPT-5.5 spec, strict_ports, run or-gpt-5.5-closed-spec-strict_ports openai 95.2%
Qwen3.7-Plus spec, strict_ports, run or-qwen3.7-plus-2026-spec-strict_ports openai 95.2% 55
Kimi K2.6 spec, strict_ports, run or-kimi-k2.6-2026-spec-strict_ports openai 94.4% 163
Claude Sonnet 4.6 spec, conventions, run fe-sonnet-spec-conventions anthropic 92.8% 325
Claude Opus 4.8 short, conventions, xhigh, run fe-opus48-xhigh-short-conventions anthropic 92.8% 1,777
DeepSeek-V4-Pro spec, strict_ports, run or-deepseek-v4-pro-2026-spec-strict_ports openai 92% 181
Qwen3.7-Max spec, strict_ports, run or-qwen3.7-max-2026-spec-strict_ports openai 92%
Claude Sonnet 4.6 spec, strict_ports, run fe-sonnet-spec-strict_ports anthropic 91.2% 302
GLM-5.1 spec, strict_ports, run or-glm-5.1-2026-spec-strict_ports openai 91.2% 27
Gemini 3.1 Pro short, conventions, run 20260623-020043-gemini-3.1-pro openai 89.6% 1,546
MiniMax-M3 spec, strict_ports, run or-minimax-m3-2026-spec-strict_ports openai 89.6% 82
Gemini 3.1 Pro short, topology_ports, run 20260622-224557-gemini-3.1-pro openai 88.8% 2,024
Claude Haiku 4.5 spec, strict_ports, run fe-haiku-spec-strict_ports anthropic 88% 200
Llama 4 Maverick spec, strict_ports, run or-llama-4-maverick-2026-spec-strict_ports openai 85.6%
GLM-4.5 spec, strict_ports, run or-glm-4.5-2025-spec-strict_ports openai 85.6% 680
GPT-5.5 short, conventions, run or-gpt-5.5-closed-short-conventions openai 85.6% 882
DeepSeek-V4-Flash spec, strict_ports, run or-deepseek-v4-flash-2026-spec-strict_ports openai 85.6%
Llama 3.3 70B Instruct spec, strict_ports, run or-llama-3.3-70b-2025-spec-strict_ports openai 84.8%
Claude Haiku 4.5 spec, conventions, run fe-haiku-spec-conventions anthropic 84.8% 239
Gemini 3.1 Pro short, strict_ports, run 20260623-020041-gemini-3.1-pro openai 84.8% 1,641
Gemini 3.1 Pro short, topology_ports, low, run gemini-3.1-pro-low-short-topology_ports openai 84.8% 828
GPT-5.5 verbose, strict_ports, run or-gpt-5.5-closed-verbose-strict_ports openai 84.8% 130
Claude Opus 4.8 short, topology_ports, max, run fe-opus48-max-short-topology_ports anthropic 84% 4,289
Kimi K2 Sep 2025 spec, strict_ports, run or-kimi-k2-2025-spec-strict_ports openai 82.4% 1
Claude Opus 4.8 short, topology_ports, xhigh, run fe-opus48-xhigh-short-topology_ports anthropic 82.4% 114
Qwen3.7-Max verbose, strict_ports, run or-qwen3.7-max-2026-verbose-strict_ports openai 79.2% 3,755
gpt-oss-20b spec, strict_ports, run lb-gptoss-20b-spec-strict_ports ollama 78.4% 240
GPT-5.5 short, topology_ports, xhigh, run or-gpt-5.5-xhigh-short-topology_ports openai 77.6% 2,968
Claude Opus 4.8 short, strict_ports, xhigh, run fe-opus48-xhigh-short-strict_ports anthropic 77.6% 1,788
gpt-oss-20b spec, topology_ports, run lb-gptoss-20b-spec-topology_ports ollama 75.2% 268
Kimi K2.6 verbose, strict_ports, run or-kimi-k2.6-2026-verbose-strict_ports openai 75.2% 3,255
GPT-5.5 short, topology_ports, run or-gpt-5.5-short-topology_ports openai 75.2% 1,044
Claude Opus 4.8 short, topology_ports, high, run fe-opus48-high-short-topology_ports anthropic 75.2% 896
Claude Opus 4.8 verbose, conventions, run fe-opus-verbose-conventions anthropic 75.2% 308
Claude Opus 4.8 verbose, strict_ports, run fe-opus-verbose-strict_ports anthropic 75.2% 301
GLM-5.1 verbose, strict_ports, run or-glm-5.1-2026-verbose-strict_ports openai 72% 545
GPT-5.5 short, strict_ports, run or-gpt-5.5-closed-short-strict_ports openai 72% 981
Claude Opus 4.7 short, topology_ports, high, run fe-opus47-high-short-topology_ports anthropic 71.2% 1,065
Claude Opus 4.8 short, conventions, run fe-opus-short-conventions anthropic 68.8% 303
Claude Opus 4.7 short, conventions, run fe-opus47-short-conventions anthropic 68% 277
Kimi K2.6 short, conventions, run or-kimi-k2.6-2026-short-conventions openai 68% 1,966
Qwen2.5-Coder (7B) spec, topology_ports, run 20260613-042138-qwen2.5-coder ollama 66.4% 114
Kimi K2.6 short, strict_ports, run or-kimi-k2.6-2026-short-strict_ports openai 66.4% 774
Claude Sonnet 4.6 verbose, conventions, run fe-sonnet-verbose-conventions anthropic 65.6% 587
Claude Opus 4.7 short, strict_ports, run fe-opus47-short-strict_ports anthropic 65.6% 294
Qwen3.7-Max short, strict_ports, run or-qwen3.7-max-2026-short-strict_ports openai 65.6% 3,546
Qwen2.5-Coder (7B) spec, topology_ports, run lb-coder-spec-topology_ports ollama 64.8% 115
Claude Sonnet 4.6 verbose, strict_ports, run fe-sonnet-verbose-strict_ports anthropic 64.8% 589
GLM-5.1 short, conventions, run or-glm-5.1-2026-short-conventions openai 64% 512
Qwen2.5-Coder (7B) spec, strict_ports, run 20260613-045341-qwen2.5-coder ollama 63.2% 111
GLM-5.1 short, topology_ports, run or-glm-5.1-short-topology_ports openai 63.2% 3,958
GLM-5.1 short, strict_ports, run or-glm-5.1-2026-short-strict_ports openai 62.4% 1,421
Qwen3.7-Max short, conventions, run or-qwen3.7-max-2026-short-conventions openai 62.4% 3,840
Claude Opus 4.8 short, topology_ports, xhigh, run fe-opus48-xhigh-nothink-short-topology_ports anthropic 62.4% 25
Qwen2.5-Coder (7B) spec, strict_ports, run lb-coder-spec-strict_ports ollama 61.6% 112
Claude Opus 4.7 short, topology_ports, run fe-opus47-short-topology_ports anthropic 61.6% 344
Qwen3.7-Plus verbose, strict_ports, run or-qwen3.7-plus-2026-verbose-strict_ports openai 61.6% 5,455
Qwen3-235B-A22B-Instruct Jul 2025 spec, strict_ports, run or-qwen3-235b-spec-strict_ports openai 59.2% 151
Claude Opus 4.8 short, strict_ports, run fe-opus-short-strict_ports anthropic 59.2% 270
DeepSeek-V4-Pro verbose, strict_ports, run or-deepseek-v4-pro-2026-verbose-strict_ports openai 58.4% 671
Gemini 3.5 Flash short, topology_ports, run 20260622-224557-gemini-3.5-flash openai 58.4% 2,494
Kimi K2.6 short, topology_ports, run or-kimi-k2.6-short-topology_ports openai 57.6% 7,603
MiniMax-M3 verbose, strict_ports, run or-minimax-m3-2026-verbose-strict_ports openai 55.2% 2,183
Qwen3-14B spec, topology, run lb-qwen3-14b-spec-topology ollama 54.4% 178
Claude Opus 4.8 short, topology_ports, run fe-opus48-short-topology_ports anthropic 53.6% 338
Qwen3-14B spec, strict_ports, run lb-qwen3-14b-spec-strict_ports ollama 52% 135
DeepSeek-V4-Pro short, conventions, run or-deepseek-v4-pro-2026-short-conventions openai 52% 452
DeepSeek-V4-Pro short, strict_ports, run or-deepseek-v4-pro-2026-short-strict_ports openai 52% 731
Qwen3.7-Max short, topology_ports, run or-qwen3.7-max-short-topology_ports openai 52% 4,934
Qwen3.7-Plus short, conventions, run or-qwen3.7-plus-2026-short-conventions openai 52% 4,854
Qwen3-14B spec, conventions, run lb-qwen3-14b-spec-conventions ollama 48.8% 166
Qwen3.7-Plus short, strict_ports, run or-qwen3.7-plus-2026-short-strict_ports openai 48.8% 4,755
GLM-4.5 verbose, strict_ports, run or-glm-4.5-2025-verbose-strict_ports openai 48% 4,761
Claude Sonnet 4.6 short, strict_ports, run fe-sonnet-short-strict_ports anthropic 48% 599
MiniMax-M3 short, conventions, run or-minimax-m3-2026-short-conventions openai 45.6% 2,476
MiniMax-M3 short, strict_ports, run or-minimax-m3-2026-short-strict_ports openai 45.6% 1,648
GLM-5.2 short, topology_ports, run or-glm-5.2-short-topology_ports openai 44.8% 4,045
Claude Sonnet 4.6 short, conventions, run fe-sonnet-short-conventions anthropic 44% 594
Qwen2.5-Coder-14B spec, strict_ports, run lb-coder14b-spec-strict_ports ollama 42.4% 166
gpt-oss-20b spec, topology, run lb-gptoss-20b-spec-topology ollama 41.6% 270
DeepSeek-V4-Flash verbose, strict_ports, run or-deepseek-v4-flash-2026-verbose-strict_ports openai 41.6%
Qwen2.5-Coder-14B spec, conventions, run lb-coder14b-spec-conventions ollama 40% 168
DeepSeek-V4-Pro short, topology_ports, run or-deepseek-v4-pro-short-topology_ports openai 40% 2,045
Gemma 3 12B spec, strict_ports, run lb-gemma3-spec-strict_ports ollama 34.4% 222
GLM-4.5 short, strict_ports, run or-glm-4.5-2025-short-strict_ports openai 33.6% 5,574
DeepSeek-V4-Flash short, strict_ports, run or-deepseek-v4-flash-2026-short-strict_ports openai 33.6% 863
DeepSeek-V4-Flash short, conventions, run or-deepseek-v4-flash-2026-short-conventions openai 32.8% 1,320
Qwen2.5-Coder-14B spec, topology, run lb-coder14b-spec-topology ollama 32% 196
gpt-oss-20b spec, conventions, run lb-gptoss-20b-spec-conventions ollama 32% 258
Qwen2.5-Coder-14B spec, topology_ports, run lb-coder14b-spec-topology_ports ollama 30.4% 99
Qwen3-14B spec, topology_ports, run lb-qwen3-14b-spec-topology_ports ollama 30.4% 122
Claude Haiku 4.5 verbose, strict_ports, run fe-haiku-verbose-strict_ports anthropic 30.4% 540
Llama 3.1-8B spec, strict_ports, run 20260613-045341-llama3.1 ollama 27.2% 159
GLM-4.5 short, conventions, run or-glm-4.5-2025-short-conventions openai 26.4% 6,048
Claude Haiku 4.5 verbose, conventions, run fe-haiku-verbose-conventions anthropic 26.4% 552
Kimi K2 Sep 2025 verbose, strict_ports, run or-kimi-k2-2025-verbose-strict_ports openai 25.6%
Gemma 2 9B spec, topology_ports, run lb-gemma2-spec-topology_ports ollama 24% 110
Gemma 3 12B spec, topology_ports, run lb-gemma3-spec-topology_ports ollama 24% 194
Gemma 2 9B spec, topology_ports, run 20260613-042138-gemma2 ollama 22.4% 111
Llama 3.1-8B spec, strict_ports, run lb-llama3.1-spec-strict_ports ollama 22.4% 156
Kimi K2 Sep 2025 short, strict_ports, run or-kimi-k2-2025-short-strict_ports openai 20.8%
Gemma 2 9B spec, strict_ports, run lb-gemma2-spec-strict_ports ollama 19.2% 123
Kimi K2 Sep 2025 short, conventions, run or-kimi-k2-2025-short-conventions openai 19.2%
Claude Haiku 4.5 short, strict_ports, run fe-haiku-short-strict_ports anthropic 19.2% 553
Claude Haiku 4.5 short, conventions, run fe-haiku-short-conventions anthropic 18.4% 524
Gemma 2 9B spec, strict_ports, run 20260613-045341-gemma2 ollama 17.6% 122
Llama 4 Maverick verbose, strict_ports, run or-llama-4-maverick-2026-verbose-strict_ports openai 13.6%
Gemma 4 31B IT short, topology_ports, run 20260622-224557-gemma-4-31b openai 12.8% 218
Llama 4 Maverick short, conventions, run or-llama-4-maverick-2026-short-conventions openai 12%
Claude Sonnet 4.6 short, topology, run fe-sonnet-short-topology anthropic 12% 848
Llama 4 Maverick short, strict_ports, run or-llama-4-maverick-2026-short-strict_ports openai 11.2%
gpt-oss-20b verbose, strict_ports, run lb-gptoss-20b-verbose-strict_ports ollama 9.60% 5
Qwen3-235B-A22B-Instruct Jul 2025 verbose, strict_ports, run or-qwen3-235b-verbose-strict_ports openai 8.80% 217
Llama 3.1-8B spec, topology_ports, run 20260613-042138-llama3.1 ollama 7.20% 177
Qwen3-235B-A22B-Instruct Jul 2025 short, conventions, run or-qwen3-235b-short-conventions openai 7.20% 251
Gemma 4 31B IT short, conventions, run gemma-4-31b-conventions-short openai 7.20% 219
Qwen2.5-Coder (7B) spec, conventions, run lb-coder-spec-conventions ollama 6.40% 102
Llama 3.3 70B Instruct verbose, strict_ports, run or-llama-3.3-70b-2025-verbose-strict_ports openai 6.40%
gpt-oss-20b short, topology_ports, run lb-gptoss-20b-short-topology_ports ollama 6.40% 569
Qwen3-235B-A22B-Instruct Jul 2025 short, strict_ports, run or-qwen3-235b-short-strict_ports openai 5.60% 240
Llama 3.1-8B spec, topology_ports, run lb-llama3.1-spec-topology_ports ollama 4.80% 179
Qwen3-14B verbose, conventions, run lb-qwen3-14b-verbose-conventions ollama 4.80% 211
gpt-oss-20b verbose, conventions, run lb-gptoss-20b-verbose-conventions ollama 4.80% 505
Qwen2.5-Coder (7B) verbose, strict_ports, run lb-coder-verbose-strict_ports ollama 4.00% 346
Llama 3.3 70B Instruct short, conventions, run or-llama-3.3-70b-2025-short-conventions openai 4.00%
gpt-oss-20b short, conventions, run lb-gptoss-20b-short-conventions ollama 4.00% 539
Qwen2.5-Coder (7B) spec, topology, run lb-coder-spec-topology ollama 3.20% 104
Qwen3-14B short, conventions, run lb-qwen3-14b-short-conventions ollama 3.20% 205
gpt-oss-20b short, strict_ports, run lb-gptoss-20b-short-strict_ports ollama 3.20% 543
Claude Haiku 4.5 short, topology, run fe-haiku-short-topology anthropic 3.20% 404
Qwen2.5-Coder (7B) short, strict_ports, run lb-coder-short-strict_ports ollama 2.40% 212
Gemma 2 9B spec, conventions, run lb-gemma2-spec-conventions ollama 1.60% 141
Qwen2.5-Coder-14B short, strict_ports, run lb-coder14b-short-strict_ports ollama 1.60% 215
Qwen2.5-Coder (7B) short, topology_ports, run lb-coder-short-topology_ports ollama 1.60% 150
Llama 3.3 70B Instruct short, strict_ports, run or-llama-3.3-70b-2025-short-strict_ports openai 1.60%
Qwen3-14B short, strict_ports, run lb-qwen3-14b-short-strict_ports ollama 1.60% 233
Qwen3-14B short, strict_ports, run lb-thinkfalse-qwen3-14b-short-strict_ports ollama 1.60% 234
GPT-5.5 short, formatted, run or-gpt-5.5-short-formatted openai 1.60% 2,341
Gemma 2 9B short, strict_ports, run lb-gemma2-short-strict_ports ollama 0.80% 107
Gemma 2 9B verbose, strict_ports, run lb-gemma2-verbose-strict_ports ollama 0.80% 94
Qwen2.5-Coder-14B short, topology_ports, run lb-coder14b-short-topology_ports ollama 0.80% 208
Qwen2.5-Coder (7B) verbose, conventions, run lb-coder-verbose-conventions ollama 0.80% 802
Gemma 3 12B short, strict_ports, run lb-gemma3-short-strict_ports ollama 0.80% 281
Gemma 3 12B spec, topology, run lb-gemma3-spec-topology ollama 0.80% 240
Gemma 3 12B verbose, strict_ports, run lb-gemma3-verbose-strict_ports ollama 0.80% 289
Qwen3-14B short, topology, run lb-qwen3-14b-short-topology ollama 0.80% 237
Qwen3-14B verbose, strict_ports, run lb-qwen3-14b-verbose-strict_ports ollama 0.80% 248
Gemma 2 9B short, conventions, run lb-gemma2-short-conventions ollama 0% 227
Gemma 2 9B short, topology, run lb-gemma2-short-topology ollama 0% 146
Gemma 2 9B short, topology_ports, run lb-gemma2-short-topology_ports ollama 0% 146
Gemma 2 9B spec, topology, run lb-gemma2-spec-topology ollama 0% 138
Gemma 2 9B verbose, conventions, run lb-gemma2-verbose-conventions ollama 0% 126
Llama 3.1-8B short, conventions, run lb-llama3.1-short-conventions ollama 0% 246
Llama 3.1-8B short, strict_ports, run lb-llama3.1-short-strict_ports ollama 0% 241
Llama 3.1-8B short, topology, run lb-llama3.1-short-topology ollama 0% 222
Llama 3.1-8B short, topology_ports, run lb-llama3.1-short-topology_ports ollama 0% 307
Llama 3.1-8B spec, conventions, run lb-llama3.1-spec-conventions ollama 0% 179
Llama 3.1-8B spec, topology, run lb-llama3.1-spec-topology ollama 0% 180
Llama 3.1-8B verbose, conventions, run lb-llama3.1-verbose-conventions ollama 0% 218
Llama 3.1-8B verbose, strict_ports, run lb-llama3.1-verbose-strict_ports ollama 0% 280
Qwen2.5-Coder-14B short, conventions, run lb-coder14b-short-conventions ollama 0% 272
Qwen2.5-Coder-14B short, topology, run lb-coder14b-short-topology ollama 0% 218
Qwen2.5-Coder-14B verbose, conventions, run lb-coder14b-verbose-conventions ollama 0% 240
Qwen2.5-Coder-14B verbose, strict_ports, run lb-coder14b-verbose-strict_ports ollama 0% 233
Qwen2.5-Coder (7B) short, conventions, run lb-coder-short-conventions ollama 0% 803
Qwen2.5-Coder (7B) short, topology, run lb-coder-short-topology ollama 0% 342
Gemma 3 12B short, conventions, run lb-gemma3-short-conventions ollama 0% 276
Gemma 3 12B short, topology, run lb-gemma3-short-topology ollama 0% 372
Gemma 3 12B short, topology_ports, run lb-gemma3-short-topology_ports ollama 0% 318
Gemma 3 12B spec, conventions, run lb-gemma3-spec-conventions ollama 0% 212
Gemma 3 12B verbose, conventions, run lb-gemma3-verbose-conventions ollama 0% 261
Qwen3-14B short, topology_ports, run lb-qwen3-14b-short-topology_ports ollama 0% 240
gpt-oss-20b short, topology, run lb-gptoss-20b-short-topology ollama 0% 632
GLM-5.1 short, formatted, run or-glm-5.1-short-formatted openai 0% 5,439
Kimi K2.6 short, formatted, run or-kimi-k2.6-short-formatted openai 0% 8,574
DeepSeek-V4-Pro short, formatted, run or-deepseek-v4-pro-short-formatted openai 0% 2,303
GLM-5.2 short, formatted, run or-glm-5.2-short-formatted openai 0% 3,857

More effort adds a median 40 points

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Opus 4.8 anthropic
short, topo… short, conv… spec, conve… short, topo… short, topo… short, stri… short, conv… short, topo…
96%
GPT-5.5 openai
short, form… short, stri… verbose, st… spec, stric… short, topo…
95.2%
Qwen3.7-Plus openai
short, stri… verbose, st… spec, stric…
95.2%
Kimi K2.6 openai
short, form… short, topo… short, stri… verbose, st… spec, stric…
94.4%
Claude Sonnet 4.6 anthropic
short, topo… short, conv… verbose, st… spec, stric…
92.8%
DeepSeek-V4-Pro openai
short, form… short, topo… short, conv… spec, stric…
92%
Qwen3.7-Max openai
short, topo… short, conv… verbose, st… spec, stric…
92%
GLM-5.1 openai
short, form… short, stri… verbose, st… spec, stric…
91.2%
Gemini 3.1 Pro openai
short, stri…
89.6%
MiniMax-M3 openai
short, conv… verbose, st… spec, stric…
89.6%
Claude Haiku 4.5 anthropic
short, topo… short, conv… verbose, co… spec, conve…
88%
Llama 4 Maverick openai
short, stri… spec, stric…
85.6%

Source: Official CircuitRubric leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 4.8 spec, conventions, run fe-opus-spec-conventions anthropic 96% 212
GPT-5.5 spec, strict_ports, run or-gpt-5.5-closed-spec-strict_ports openai 95.2%
Qwen3.7-Plus spec, strict_ports, run or-qwen3.7-plus-2026-spec-strict_ports openai 95.2% 55
Kimi K2.6 spec, strict_ports, run or-kimi-k2.6-2026-spec-strict_ports openai 94.4% 163
Claude Sonnet 4.6 spec, conventions, run fe-sonnet-spec-conventions anthropic 92.8% 325
Claude Opus 4.8 short, conventions, xhigh, run fe-opus48-xhigh-short-conventions anthropic 92.8% 1,777
DeepSeek-V4-Pro spec, strict_ports, run or-deepseek-v4-pro-2026-spec-strict_ports openai 92% 181
Qwen3.7-Max spec, strict_ports, run or-qwen3.7-max-2026-spec-strict_ports openai 92%
Claude Sonnet 4.6 spec, strict_ports, run fe-sonnet-spec-strict_ports anthropic 91.2% 302
GLM-5.1 spec, strict_ports, run or-glm-5.1-2026-spec-strict_ports openai 91.2% 27
Gemini 3.1 Pro short, conventions, run 20260623-020043-gemini-3.1-pro openai 89.6% 1,546
MiniMax-M3 spec, strict_ports, run or-minimax-m3-2026-spec-strict_ports openai 89.6% 82
Gemini 3.1 Pro short, topology_ports, run 20260622-224557-gemini-3.1-pro openai 88.8% 2,024
Claude Haiku 4.5 spec, strict_ports, run fe-haiku-spec-strict_ports anthropic 88% 200
Llama 4 Maverick spec, strict_ports, run or-llama-4-maverick-2026-spec-strict_ports openai 85.6%
GLM-4.5 spec, strict_ports, run or-glm-4.5-2025-spec-strict_ports openai 85.6% 680
GPT-5.5 short, conventions, run or-gpt-5.5-closed-short-conventions openai 85.6% 882
DeepSeek-V4-Flash spec, strict_ports, run or-deepseek-v4-flash-2026-spec-strict_ports openai 85.6%
Llama 3.3 70B Instruct spec, strict_ports, run or-llama-3.3-70b-2025-spec-strict_ports openai 84.8%
Claude Haiku 4.5 spec, conventions, run fe-haiku-spec-conventions anthropic 84.8% 239
Gemini 3.1 Pro short, strict_ports, run 20260623-020041-gemini-3.1-pro openai 84.8% 1,641
Gemini 3.1 Pro short, topology_ports, low, run gemini-3.1-pro-low-short-topology_ports openai 84.8% 828
GPT-5.5 verbose, strict_ports, run or-gpt-5.5-closed-verbose-strict_ports openai 84.8% 130
Claude Opus 4.8 short, topology_ports, max, run fe-opus48-max-short-topology_ports anthropic 84% 4,289
Kimi K2 Sep 2025 spec, strict_ports, run or-kimi-k2-2025-spec-strict_ports openai 82.4% 1
Claude Opus 4.8 short, topology_ports, xhigh, run fe-opus48-xhigh-short-topology_ports anthropic 82.4% 114
Qwen3.7-Max verbose, strict_ports, run or-qwen3.7-max-2026-verbose-strict_ports openai 79.2% 3,755
gpt-oss-20b spec, strict_ports, run lb-gptoss-20b-spec-strict_ports ollama 78.4% 240
GPT-5.5 short, topology_ports, xhigh, run or-gpt-5.5-xhigh-short-topology_ports openai 77.6% 2,968
Claude Opus 4.8 short, strict_ports, xhigh, run fe-opus48-xhigh-short-strict_ports anthropic 77.6% 1,788
gpt-oss-20b spec, topology_ports, run lb-gptoss-20b-spec-topology_ports ollama 75.2% 268
Kimi K2.6 verbose, strict_ports, run or-kimi-k2.6-2026-verbose-strict_ports openai 75.2% 3,255
GPT-5.5 short, topology_ports, run or-gpt-5.5-short-topology_ports openai 75.2% 1,044
Claude Opus 4.8 short, topology_ports, high, run fe-opus48-high-short-topology_ports anthropic 75.2% 896
Claude Opus 4.8 verbose, conventions, run fe-opus-verbose-conventions anthropic 75.2% 308
Claude Opus 4.8 verbose, strict_ports, run fe-opus-verbose-strict_ports anthropic 75.2% 301
GLM-5.1 verbose, strict_ports, run or-glm-5.1-2026-verbose-strict_ports openai 72% 545
GPT-5.5 short, strict_ports, run or-gpt-5.5-closed-short-strict_ports openai 72% 981
Claude Opus 4.7 short, topology_ports, high, run fe-opus47-high-short-topology_ports anthropic 71.2% 1,065
Claude Opus 4.8 short, conventions, run fe-opus-short-conventions anthropic 68.8% 303
Claude Opus 4.7 short, conventions, run fe-opus47-short-conventions anthropic 68% 277
Kimi K2.6 short, conventions, run or-kimi-k2.6-2026-short-conventions openai 68% 1,966
Qwen2.5-Coder (7B) spec, topology_ports, run 20260613-042138-qwen2.5-coder ollama 66.4% 114
Kimi K2.6 short, strict_ports, run or-kimi-k2.6-2026-short-strict_ports openai 66.4% 774
Claude Sonnet 4.6 verbose, conventions, run fe-sonnet-verbose-conventions anthropic 65.6% 587
Claude Opus 4.7 short, strict_ports, run fe-opus47-short-strict_ports anthropic 65.6% 294
Qwen3.7-Max short, strict_ports, run or-qwen3.7-max-2026-short-strict_ports openai 65.6% 3,546
Qwen2.5-Coder (7B) spec, topology_ports, run lb-coder-spec-topology_ports ollama 64.8% 115
Claude Sonnet 4.6 verbose, strict_ports, run fe-sonnet-verbose-strict_ports anthropic 64.8% 589
GLM-5.1 short, conventions, run or-glm-5.1-2026-short-conventions openai 64% 512
Qwen2.5-Coder (7B) spec, strict_ports, run 20260613-045341-qwen2.5-coder ollama 63.2% 111
GLM-5.1 short, topology_ports, run or-glm-5.1-short-topology_ports openai 63.2% 3,958
GLM-5.1 short, strict_ports, run or-glm-5.1-2026-short-strict_ports openai 62.4% 1,421
Qwen3.7-Max short, conventions, run or-qwen3.7-max-2026-short-conventions openai 62.4% 3,840
Claude Opus 4.8 short, topology_ports, xhigh, run fe-opus48-xhigh-nothink-short-topology_ports anthropic 62.4% 25
Qwen2.5-Coder (7B) spec, strict_ports, run lb-coder-spec-strict_ports ollama 61.6% 112
Claude Opus 4.7 short, topology_ports, run fe-opus47-short-topology_ports anthropic 61.6% 344
Qwen3.7-Plus verbose, strict_ports, run or-qwen3.7-plus-2026-verbose-strict_ports openai 61.6% 5,455
Qwen3-235B-A22B-Instruct Jul 2025 spec, strict_ports, run or-qwen3-235b-spec-strict_ports openai 59.2% 151
Claude Opus 4.8 short, strict_ports, run fe-opus-short-strict_ports anthropic 59.2% 270
DeepSeek-V4-Pro verbose, strict_ports, run or-deepseek-v4-pro-2026-verbose-strict_ports openai 58.4% 671
Gemini 3.5 Flash short, topology_ports, run 20260622-224557-gemini-3.5-flash openai 58.4% 2,494
Kimi K2.6 short, topology_ports, run or-kimi-k2.6-short-topology_ports openai 57.6% 7,603
MiniMax-M3 verbose, strict_ports, run or-minimax-m3-2026-verbose-strict_ports openai 55.2% 2,183
Qwen3-14B spec, topology, run lb-qwen3-14b-spec-topology ollama 54.4% 178
Claude Opus 4.8 short, topology_ports, run fe-opus48-short-topology_ports anthropic 53.6% 338
Qwen3-14B spec, strict_ports, run lb-qwen3-14b-spec-strict_ports ollama 52% 135
DeepSeek-V4-Pro short, conventions, run or-deepseek-v4-pro-2026-short-conventions openai 52% 452
DeepSeek-V4-Pro short, strict_ports, run or-deepseek-v4-pro-2026-short-strict_ports openai 52% 731
Qwen3.7-Max short, topology_ports, run or-qwen3.7-max-short-topology_ports openai 52% 4,934
Qwen3.7-Plus short, conventions, run or-qwen3.7-plus-2026-short-conventions openai 52% 4,854
Qwen3-14B spec, conventions, run lb-qwen3-14b-spec-conventions ollama 48.8% 166
Qwen3.7-Plus short, strict_ports, run or-qwen3.7-plus-2026-short-strict_ports openai 48.8% 4,755
GLM-4.5 verbose, strict_ports, run or-glm-4.5-2025-verbose-strict_ports openai 48% 4,761
Claude Sonnet 4.6 short, strict_ports, run fe-sonnet-short-strict_ports anthropic 48% 599
MiniMax-M3 short, conventions, run or-minimax-m3-2026-short-conventions openai 45.6% 2,476
MiniMax-M3 short, strict_ports, run or-minimax-m3-2026-short-strict_ports openai 45.6% 1,648
GLM-5.2 short, topology_ports, run or-glm-5.2-short-topology_ports openai 44.8% 4,045
Claude Sonnet 4.6 short, conventions, run fe-sonnet-short-conventions anthropic 44% 594
Qwen2.5-Coder-14B spec, strict_ports, run lb-coder14b-spec-strict_ports ollama 42.4% 166
gpt-oss-20b spec, topology, run lb-gptoss-20b-spec-topology ollama 41.6% 270
DeepSeek-V4-Flash verbose, strict_ports, run or-deepseek-v4-flash-2026-verbose-strict_ports openai 41.6%
Qwen2.5-Coder-14B spec, conventions, run lb-coder14b-spec-conventions ollama 40% 168
DeepSeek-V4-Pro short, topology_ports, run or-deepseek-v4-pro-short-topology_ports openai 40% 2,045
Gemma 3 12B spec, strict_ports, run lb-gemma3-spec-strict_ports ollama 34.4% 222
GLM-4.5 short, strict_ports, run or-glm-4.5-2025-short-strict_ports openai 33.6% 5,574
DeepSeek-V4-Flash short, strict_ports, run or-deepseek-v4-flash-2026-short-strict_ports openai 33.6% 863
DeepSeek-V4-Flash short, conventions, run or-deepseek-v4-flash-2026-short-conventions openai 32.8% 1,320
Qwen2.5-Coder-14B spec, topology, run lb-coder14b-spec-topology ollama 32% 196
gpt-oss-20b spec, conventions, run lb-gptoss-20b-spec-conventions ollama 32% 258
Qwen2.5-Coder-14B spec, topology_ports, run lb-coder14b-spec-topology_ports ollama 30.4% 99
Qwen3-14B spec, topology_ports, run lb-qwen3-14b-spec-topology_ports ollama 30.4% 122
Claude Haiku 4.5 verbose, strict_ports, run fe-haiku-verbose-strict_ports anthropic 30.4% 540
Llama 3.1-8B spec, strict_ports, run 20260613-045341-llama3.1 ollama 27.2% 159
GLM-4.5 short, conventions, run or-glm-4.5-2025-short-conventions openai 26.4% 6,048
Claude Haiku 4.5 verbose, conventions, run fe-haiku-verbose-conventions anthropic 26.4% 552
Kimi K2 Sep 2025 verbose, strict_ports, run or-kimi-k2-2025-verbose-strict_ports openai 25.6%
Gemma 2 9B spec, topology_ports, run lb-gemma2-spec-topology_ports ollama 24% 110
Gemma 3 12B spec, topology_ports, run lb-gemma3-spec-topology_ports ollama 24% 194
Gemma 2 9B spec, topology_ports, run 20260613-042138-gemma2 ollama 22.4% 111
Llama 3.1-8B spec, strict_ports, run lb-llama3.1-spec-strict_ports ollama 22.4% 156
Kimi K2 Sep 2025 short, strict_ports, run or-kimi-k2-2025-short-strict_ports openai 20.8%
Gemma 2 9B spec, strict_ports, run lb-gemma2-spec-strict_ports ollama 19.2% 123
Kimi K2 Sep 2025 short, conventions, run or-kimi-k2-2025-short-conventions openai 19.2%
Claude Haiku 4.5 short, strict_ports, run fe-haiku-short-strict_ports anthropic 19.2% 553
Claude Haiku 4.5 short, conventions, run fe-haiku-short-conventions anthropic 18.4% 524
Gemma 2 9B spec, strict_ports, run 20260613-045341-gemma2 ollama 17.6% 122
Llama 4 Maverick verbose, strict_ports, run or-llama-4-maverick-2026-verbose-strict_ports openai 13.6%
Gemma 4 31B IT short, topology_ports, run 20260622-224557-gemma-4-31b openai 12.8% 218
Llama 4 Maverick short, conventions, run or-llama-4-maverick-2026-short-conventions openai 12%
Claude Sonnet 4.6 short, topology, run fe-sonnet-short-topology anthropic 12% 848
Llama 4 Maverick short, strict_ports, run or-llama-4-maverick-2026-short-strict_ports openai 11.2%
gpt-oss-20b verbose, strict_ports, run lb-gptoss-20b-verbose-strict_ports ollama 9.60% 5
Qwen3-235B-A22B-Instruct Jul 2025 verbose, strict_ports, run or-qwen3-235b-verbose-strict_ports openai 8.80% 217
Llama 3.1-8B spec, topology_ports, run 20260613-042138-llama3.1 ollama 7.20% 177
Qwen3-235B-A22B-Instruct Jul 2025 short, conventions, run or-qwen3-235b-short-conventions openai 7.20% 251
Gemma 4 31B IT short, conventions, run gemma-4-31b-conventions-short openai 7.20% 219
Qwen2.5-Coder (7B) spec, conventions, run lb-coder-spec-conventions ollama 6.40% 102
Llama 3.3 70B Instruct verbose, strict_ports, run or-llama-3.3-70b-2025-verbose-strict_ports openai 6.40%
gpt-oss-20b short, topology_ports, run lb-gptoss-20b-short-topology_ports ollama 6.40% 569
Qwen3-235B-A22B-Instruct Jul 2025 short, strict_ports, run or-qwen3-235b-short-strict_ports openai 5.60% 240
Llama 3.1-8B spec, topology_ports, run lb-llama3.1-spec-topology_ports ollama 4.80% 179
Qwen3-14B verbose, conventions, run lb-qwen3-14b-verbose-conventions ollama 4.80% 211
gpt-oss-20b verbose, conventions, run lb-gptoss-20b-verbose-conventions ollama 4.80% 505
Qwen2.5-Coder (7B) verbose, strict_ports, run lb-coder-verbose-strict_ports ollama 4.00% 346
Llama 3.3 70B Instruct short, conventions, run or-llama-3.3-70b-2025-short-conventions openai 4.00%
gpt-oss-20b short, conventions, run lb-gptoss-20b-short-conventions ollama 4.00% 539
Qwen2.5-Coder (7B) spec, topology, run lb-coder-spec-topology ollama 3.20% 104
Qwen3-14B short, conventions, run lb-qwen3-14b-short-conventions ollama 3.20% 205
gpt-oss-20b short, strict_ports, run lb-gptoss-20b-short-strict_ports ollama 3.20% 543
Claude Haiku 4.5 short, topology, run fe-haiku-short-topology anthropic 3.20% 404
Qwen2.5-Coder (7B) short, strict_ports, run lb-coder-short-strict_ports ollama 2.40% 212
Gemma 2 9B spec, conventions, run lb-gemma2-spec-conventions ollama 1.60% 141
Qwen2.5-Coder-14B short, strict_ports, run lb-coder14b-short-strict_ports ollama 1.60% 215
Qwen2.5-Coder (7B) short, topology_ports, run lb-coder-short-topology_ports ollama 1.60% 150
Llama 3.3 70B Instruct short, strict_ports, run or-llama-3.3-70b-2025-short-strict_ports openai 1.60%
Qwen3-14B short, strict_ports, run lb-qwen3-14b-short-strict_ports ollama 1.60% 233
Qwen3-14B short, strict_ports, run lb-thinkfalse-qwen3-14b-short-strict_ports ollama 1.60% 234
GPT-5.5 short, formatted, run or-gpt-5.5-short-formatted openai 1.60% 2,341
Gemma 2 9B short, strict_ports, run lb-gemma2-short-strict_ports ollama 0.80% 107
Gemma 2 9B verbose, strict_ports, run lb-gemma2-verbose-strict_ports ollama 0.80% 94
Qwen2.5-Coder-14B short, topology_ports, run lb-coder14b-short-topology_ports ollama 0.80% 208
Qwen2.5-Coder (7B) verbose, conventions, run lb-coder-verbose-conventions ollama 0.80% 802
Gemma 3 12B short, strict_ports, run lb-gemma3-short-strict_ports ollama 0.80% 281
Gemma 3 12B spec, topology, run lb-gemma3-spec-topology ollama 0.80% 240
Gemma 3 12B verbose, strict_ports, run lb-gemma3-verbose-strict_ports ollama 0.80% 289
Qwen3-14B short, topology, run lb-qwen3-14b-short-topology ollama 0.80% 237
Qwen3-14B verbose, strict_ports, run lb-qwen3-14b-verbose-strict_ports ollama 0.80% 248
Gemma 2 9B short, conventions, run lb-gemma2-short-conventions ollama 0% 227
Gemma 2 9B short, topology, run lb-gemma2-short-topology ollama 0% 146
Gemma 2 9B short, topology_ports, run lb-gemma2-short-topology_ports ollama 0% 146
Gemma 2 9B spec, topology, run lb-gemma2-spec-topology ollama 0% 138
Gemma 2 9B verbose, conventions, run lb-gemma2-verbose-conventions ollama 0% 126
Llama 3.1-8B short, conventions, run lb-llama3.1-short-conventions ollama 0% 246
Llama 3.1-8B short, strict_ports, run lb-llama3.1-short-strict_ports ollama 0% 241
Llama 3.1-8B short, topology, run lb-llama3.1-short-topology ollama 0% 222
Llama 3.1-8B short, topology_ports, run lb-llama3.1-short-topology_ports ollama 0% 307
Llama 3.1-8B spec, conventions, run lb-llama3.1-spec-conventions ollama 0% 179
Llama 3.1-8B spec, topology, run lb-llama3.1-spec-topology ollama 0% 180
Llama 3.1-8B verbose, conventions, run lb-llama3.1-verbose-conventions ollama 0% 218
Llama 3.1-8B verbose, strict_ports, run lb-llama3.1-verbose-strict_ports ollama 0% 280
Qwen2.5-Coder-14B short, conventions, run lb-coder14b-short-conventions ollama 0% 272
Qwen2.5-Coder-14B short, topology, run lb-coder14b-short-topology ollama 0% 218
Qwen2.5-Coder-14B verbose, conventions, run lb-coder14b-verbose-conventions ollama 0% 240
Qwen2.5-Coder-14B verbose, strict_ports, run lb-coder14b-verbose-strict_ports ollama 0% 233
Qwen2.5-Coder (7B) short, conventions, run lb-coder-short-conventions ollama 0% 803
Qwen2.5-Coder (7B) short, topology, run lb-coder-short-topology ollama 0% 342
Gemma 3 12B short, conventions, run lb-gemma3-short-conventions ollama 0% 276
Gemma 3 12B short, topology, run lb-gemma3-short-topology ollama 0% 372
Gemma 3 12B short, topology_ports, run lb-gemma3-short-topology_ports ollama 0% 318
Gemma 3 12B spec, conventions, run lb-gemma3-spec-conventions ollama 0% 212
Gemma 3 12B verbose, conventions, run lb-gemma3-verbose-conventions ollama 0% 261
Qwen3-14B short, topology_ports, run lb-qwen3-14b-short-topology_ports ollama 0% 240
gpt-oss-20b short, topology, run lb-gptoss-20b-short-topology ollama 0% 632
GLM-5.1 short, formatted, run or-glm-5.1-short-formatted openai 0% 5,439
Kimi K2.6 short, formatted, run or-kimi-k2.6-short-formatted openai 0% 8,574
DeepSeek-V4-Pro short, formatted, run or-deepseek-v4-pro-short-formatted openai 0% 2,303
GLM-5.2 short, formatted, run or-glm-5.2-short-formatted openai 0% 3,857

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard Passed 85% in Jun 2026, by Claude Opus 4.8

Source: Official CircuitRubric leaderboard

Llama 4 Maverick on the official leaderboard passed 85% in April 2025

Models released before it came out already scored that high. All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Claude Opus 4.8 Claude Haiku 4.5 Llama 4 Maverick Qwen2.5-Coder (7B) Llama 3.1-8B 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Claude Opus 4.8 Claude Haiku 4.5

Source: Official CircuitRubric leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Engineering and CAD

All 112 in Engineering and CAD
  • EEBench In use 13 electrical-engineering circuit and PCB design tasks graded by deterministic build checks, simulation and BOM constraints. No LLM judge. Gemini 4 Carbon 83.9%
  • 3DCodeBench In use Agentic procedural 3D modeling via code: author engine-ready 3D assets through software APIs. Graded for executability and shape fidelity. LongCat-2.5 55.4%
  • CADGenBench Only in system cards Agents construct CAD models from design prompts. The generation split is graded for executability and geometric correctness. Claude Fable 5 45.1%
  • PartBench Only in system cards Parametric mechanical part design and CAD-style generation, graded for geometric and manufacturing fidelity. Grok 4.6 58.5%
  • Razavi-Bench Not in model cards Models explain what happens in analog circuits and why. A language model awards up to four points per answer against a reference and a rubric. Gemini 4 Carbon 98.3%
  • AnalogCoder Not in model cards Models write Python programs that build analog circuits. A simulator checks whether each circuit performs its requested function. The authors report first-attempt success and how many tasks each model solves. Llama 3.1-405B 83.3%

Data audit

All 9 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (35 entries).
  • Pass Models scored 28 models we track, plus 7 other entries on its leaderboard.
  • Pass Newest score The newest is from June 2026, for GLM-5.2.
  • Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21).
  • Note Headroom Saturated. The best is 96%, past 85% since April 2025.
Leaderboard import 3 pass
  • Pass Importer We read the board on github.com: 178 rows for models we track and 7 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check