Benchmark Almanac

RWE-bench

Claude Fable 5.2 62% leads since November 2026
Not in model cards

Agents do mechanical, electrical and aerospace design tasks, such as a crankshaft, dense PCB routing or turbine-vane cooling, with CAD and CAE tools. A deterministic verifier runs physics solvers and passes a design only if it meets every requirement.

Engineers on the team wrote each task with its verifier and a reference solution. Each model runs in its native harness at max effort. The tasks are never licensed for training. Launch coverage reports 20 tasks and 11 models, with GPT-6 Astra at 37%.

Made by
Xplore AI
Released
October 2026
Kind
CAD models
Level
Expert
Scored
Checked automatically
Answers
Agent task
Tasks
Kept private
Strongest model tested
Claude Fable 5.2 #1 on the capability index
Last model added
November 2026
Scores refreshed
November 12th, 2026
15
Models scored on its own leaderboard

Claude Fable 5.2 leads with 62%

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

15 models. Scroll sideways for the rest.

  1. 62.0 Claude Fable 5.2
  2. 37.0 GPT-6 Astra
  3. 29.0 Claude Opus 5.5
  4. 25.0 GPT-6.1 Sol
  5. 20.0 Gemini 4 Carbon
  6. 19.0 GPT-6.1 Terra
  7. 12.0 Gemini 4 Argon
  8. 7.00 Claude Fable 5.1
  9. 1.00 Kimi K3.1 Pro
  10. 0.0 Qwen 3.8 Max
  11. 0.0 GLM-5.3
  12. 0.0 DeepSeek V4.1 Flash
  13. 0.0 Grok 4.7
  14. 0.0 Muse Wonder
  15. 0.0 Nemotron 3.5 Super VL
  1. 1 Claude Fable 5.2 max 62%
  2. 2 GPT-6 Astra max 37%
  3. 3 Claude Opus 5.5 max 29%
  4. 4 GPT-6.1 Sol max 25%
  5. 5 Gemini 4 Carbon high 20%
  6. 6 GPT-6.1 Terra max 19%
  7. 7 Gemini 4 Argon high 12%
  8. 8 Claude Fable 5.1 max 7.00%
  9. 9 Kimi K3.1 Pro max 1.00%
  10. 10 Qwen 3.8 Max xhigh 0%
  11. 11 GLM-5.3 max 0%
  12. 12 DeepSeek V4.1 Flash max 0%
  13. 13 Grok 4.7 xhigh 0%
  14. 14 Muse Wonder max 0%
  15. 15 Nemotron 3.5 Super VL max 0%

Source: Official RWE-bench leaderboard

#ModelScoreSettingDateSource
1 Claude Fable 5.2 62% max, Claude Code Nov 2026 Official leaderboard
2 GPT-6 Astra 37% max, Codex Sep 2026 Official leaderboard
3 Claude Opus 5.5 29% max, Claude Code Sep 2026 Official leaderboard
4 GPT-6.1 Sol 25% max, Codex Sep 2026 Official leaderboard
5 Gemini 4 Carbon 20% high, Antigravity Oct 2026 Official leaderboard
6 GPT-6.1 Terra 19% max, Codex Oct 2026 Official leaderboard
7 Gemini 4 Argon 12% high, Gemini CLI Sep 2026 Official leaderboard
8 Claude Fable 5.1 7.00% max, Claude Code Sep 2026 Official leaderboard
9 Kimi K3.1 Pro 1.00% max, Kimi Code Oct 2026 Official leaderboard
10 Qwen 3.8 Max 0% xhigh, Qwen Coder Aug 2026 Official leaderboard
11 GLM-5.3 0% max, ZCode Aug 2026 Official leaderboard
12 DeepSeek V4.1 Flash 0% max, DSH Sep 2026 Official leaderboard
13 Grok 4.7 0% xhigh, Grok Build Sep 2026 Official leaderboard
14 Muse Wonder 0% max, Muse Code Oct 2026 Official leaderboard
15 Nemotron 3.5 Super VL 0% max, OpenHands Oct 2026 Official leaderboard

Average pass rate against cost or time per run

Figure 2 with time on the x axis: each point is one model in its harness, and the Pareto frontier joins the models no other passes more for less time. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

0% 20% 40% 60% 80% 010K20K30K Time per run (s), lower is better Average pass rate Qwen 3.8 Max (xhigh): Average pass rate 0%, time per run 16K s GLM-5.3 (max): Average pass rate 0%, time per run 17K s Claude Fable 5.1 (max): Average pass rate 7%, time per run 13K s Grok 4.7 (xhigh): Average pass rate 0%, time per run 13K s Claude Opus 5.5 (max): Average pass rate 29%, time per run 15K s GPT-6.1 Sol (max): Average pass rate 25%, time per run 12K s Muse Wonder (max): Average pass rate 0%, time per run 9,756 s Nemotron 3.5 Super VL (max): Average pass rate 0%, time per run 27K s Kimi K3.1 Pro (max): Average pass rate 1%, time per run 15K s Gemini 4 Carbon (high): Average pass rate 20%, time per run 26K s GPT-6 Astra (max): Average pass rate 37%, time per run 10K s DeepSeek V4.1 Flash (max): Average pass rate 0%, time per run 7,308 s Gemini 4 Argon (high): Average pass rate 12%, time per run 8,928 s GPT-6.1 Terra (max): Average pass rate 19%, time per run 9,828 s Claude Fable 5.2 (max): Average pass rate 62%, time per run 20K s Claude Fable 5.2 (max)GPT-6 Astra (max)GPT-6.1 Terra (max)Gemini 4 Argon (high)DeepSeek V4.1 Flash (max) 0% 20% 40% 60% 80% 010K20K30K Time per run (s), lower is better Average pass rate Qwen 3.8 Max (xhigh): Average pass rate 0%, time per run 16K s GLM-5.3 (max): Average pass rate 0%, time per run 17K s Claude Fable 5.1 (max): Average pass rate 7%, time per run 13K s Grok 4.7 (xhigh): Average pass rate 0%, time per run 13K s Claude Opus 5.5 (max): Average pass rate 29%, time per run 15K s GPT-6.1 Sol (max): Average pass rate 25%, time per run 12K s Muse Wonder (max): Average pass rate 0%, time per run 9,756 s Nemotron 3.5 Super VL (max): Average pass rate 0%, time per run 27K s Kimi K3.1 Pro (max): Average pass rate 1%, time per run 15K s Gemini 4 Carbon (high): Average pass rate 20%, time per run 26K s GPT-6 Astra (max): Average pass rate 37%, time per run 10K s DeepSeek V4.1 Flash (max): Average pass rate 0%, time per run 7,308 s Gemini 4 Argon (high): Average pass rate 12%, time per run 8,928 s GPT-6.1 Terra (max): Average pass rate 19%, time per run 9,828 s Claude Fable 5.2 (max): Average pass rate 62%, time per run 20K s Claude Fable 5.2 (max)GPT-6 Astra (max)GPT-6.1 Terra (max)Gemini 4 Argon (high)DeepSeek V4.1 Flash (max)
ModelSettingAverage pass rateTime per run
Claude Fable 5.2 max, Claude Code 62% 19,620 s
GPT-6 Astra max, Codex 37% 10,116 s
Claude Opus 5.5 max, Claude Code 29% 15,228 s
GPT-6.1 Sol max, Codex 25% 11,664 s
Gemini 4 Carbon high, Antigravity 20% 25,668 s
GPT-6.1 Terra max, Codex 19% 9,828 s
Gemini 4 Argon high, Gemini CLI 12% 8,928 s
Claude Fable 5.1 max, Claude Code 7% 13,248 s
Kimi K3.1 Pro max, Kimi Code 1% 14,652 s
DeepSeek V4.1 Flash max, DSH 0% 7,308 s
Muse Wonder max, Muse Code 0% 9,756 s
Grok 4.7 xhigh, Grok Build 0% 13,356 s
Qwen 3.8 Max xhigh, Qwen Coder 0% 15,696 s
GLM-5.3 max, ZCode 0% 16,596 s
Nemotron 3.5 Super VL max, OpenHands 0% 27,252 s

The best score, 62%, costs $115 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI DeepSeek Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% 80% $1.00 $10 $100 $1,000 Cost per task, log scale Claude Fable 5.2, max GPT-6 Astra, max Claude Opus 5.5, max GPT-6.1 Sol, max DeepSeek V4.1 Flash, max 0% 20% 40% 60% 80% $1.00 $10 $100 $1,000 Cost per task, log scale Claude Fable 5.2, max GPT-6 Astra, max

Source: Official RWE-bench leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5.2 max Claude Code 62% $115 999,000
GPT-6 Astra max Codex 37% $59 358,000
Claude Opus 5.5 max Claude Code 29% $58 739,000
GPT-6.1 Sol max Codex 25% $10 454,000
Gemini 4 Carbon high Antigravity 20% $31 499,000
GPT-6.1 Terra max Codex 19% $11 554,000
Gemini 4 Argon high Gemini CLI 12% $24 359,000
Claude Fable 5.1 max Claude Code 7.00% $98 652,000
Kimi K3.1 Pro max Kimi Code 1.00% $71 548,000
Qwen 3.8 Max xhigh Qwen Coder 0% $17 349,000
GLM-5.3 max ZCode 0% $23 1,029,000
DeepSeek V4.1 Flash max DSH 0% $1.03 424,000
Grok 4.7 xhigh Grok Build 0% $66 517,000
Muse Wonder max Muse Code 0% $28 644,000
Nemotron 3.5 Super VL max OpenHands 0% $2.93 983,000

2 models have held number one on the official leaderboard since its release in October 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Anthropic
Own leaderboard 2 leaders
Claude Fable 5.2 62%

Source: Official RWE-bench leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) Claude Fable 5.2 GPT-6 Astra Claude Fable 5.1 Qwen 3.8 Max 0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) Claude Fable 5.2 GPT-6 Astra

Source: Official RWE-bench leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Engineering and CAD

All 23 in Engineering and CAD
  • EEBench In use 13 electrical-engineering circuit and PCB design tasks graded by deterministic build checks, simulation and BOM constraints. No LLM judge. Gemini 4 Carbon 83.9%
  • Parametric CAD Bench In use Agents author editable parametric FreeCAD part designs, graded against held-back references for geometry and spec consistency. Claude Opus 5.5 61%
  • 3DCodeBench In use Agentic procedural 3D modeling via code: author engine-ready 3D assets through software APIs. Graded for executability and shape fidelity. LongCat-2.5 55.4%
  • CADGenBench Only in system cards Agents construct CAD models from design prompts. The generation split is graded for executability and geometric correctness. Claude Fable 5 45.1%
  • PartBench Only in system cards Parametric mechanical part design and CAD-style generation, graded for geometric and manufacturing fidelity. Grok 4.6 58.5%
  • LMBuild Not in model cards Language model agents design 3D structures as parts, joints, materials and assembly steps. Scoring checks that the structures can be built and work as intended. GPT-6 Astra 66.5%