Benchmark Almanac

CadEval

o3 74% leads since April 2025
Not in model cards

Models write OpenSCAD code for single mechanical parts from text descriptions. Rendered parts are checked against reference geometry, scored by pass rate.

Made by
Will Patrick
Released
April 2025
Kind
CAD models
Level
University
Scored
Checked automatically
Answers
Code
Tasks
All public
Strongest model tested
o3 #83 on the capability index
Last model added
April 2025
Scores refreshed
November 12th, 2026
15
Entries on its own leaderboard, 13 models we track

o3 leads with 74% among models released by April 2025

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in April 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

15 entries. Scroll sideways for the rest.

  1. 74.0 o3
  2. 64.0 Gemini 2.5 Pro
  3. 62.0 o4-mini
  4. 56.0 o1
  5. 54.0 o3-mini
  6. 54.0 Claude 3.7 Sonnet
  7. 48.0 Claude 3.5 Sonnet (October 2024)
  8. 42.0 GPT-4.1
  9. 34.0 gemini-1.5-pro [snapshot unspecified]
  10. 32.0 Claude 3.5 Haiku
  11. 30.0 Gemini 2.0 Flash
  12. 26.0 GPT-4o
  13. 20.0 zoo-ml-text-to-cad
  14. 16.0 GPT-4.1 mini
  15. 12.0 Claude 3 Haiku
  1. 1 o3 run 20250422_095709; default; source-reported means; check counts retained 74%
  2. 2 Gemini 2.5 Pro Mar 2025, run 20250422_095709; default; source-reported means; check counts retained 64%
  3. 3 o4-mini run 20250422_095709; default; source-reported means; check counts retained 62%
  4. 4 o1 run 20250422_095709; default; source-reported means; check counts retained 56%
  5. 5 o3-mini run 20250422_095709; default; source-reported means; check counts retained 54%
  6. 6 Claude 3.7 Sonnet run 20250422_095709; default; source-reported means; check counts retained 54%
  7. 7 Claude 3.5 Sonnet (October 2024) run 20250422_095709; default; source-reported means; check counts retained 48%
  8. 8 GPT-4.1 run 20250422_095709; default; source-reported means; check counts retained 42%
  9. 9 gemini-1.5-pro [snapshot unspecified] run 20250422_095709; default; source-reported means; check counts retained 34%
  10. 10 Claude 3.5 Haiku run 20250422_095709; default; source-reported means; check counts retained 32%
  11. 11 Gemini 2.0 Flash Feb 2025, run 20250422_095709; default; source-reported means; check counts retained 30%
  12. 12 GPT-4o Aug 2024, run 20250422_095709; default; source-reported means; check counts retained 26%
  13. 13 zoo-ml-text-to-cad run 20250422_095709; default; source-reported means; check counts retained 20%
  14. 14 GPT-4.1 mini run 20250422_095709; default; source-reported means; check counts retained 16%
  15. 15 Claude 3 Haiku run 20250422_095709; default; source-reported means; check counts retained 12%

Source: Official CadEval leaderboard

#EntryScoreSettingDateSource
1 o3 74% run 20250422_095709; default; source-reported means; check counts retained Apr 2025 Official leaderboard
2 Gemini 2.5 Pro Mar 2025 64% run 20250422_095709; default; source-reported means; check counts retained Mar 2025 Official leaderboard
3 o4-mini 62% run 20250422_095709; default; source-reported means; check counts retained Apr 2025 Official leaderboard
4 o1 56% run 20250422_095709; default; source-reported means; check counts retained Dec 2024 Official leaderboard
5 o3-mini 54% run 20250422_095709; default; source-reported means; check counts retained Jan 2025 Official leaderboard
6 Claude 3.7 Sonnet 54% run 20250422_095709; default; source-reported means; check counts retained Feb 2025 Official leaderboard
7 Claude 3.5 Sonnet (October 2024) 48% run 20250422_095709; default; source-reported means; check counts retained Oct 2024 Official leaderboard
8 GPT-4.1 42% run 20250422_095709; default; source-reported means; check counts retained Apr 2025 Official leaderboard
9 gemini-1.5-pro [snapshot unspecified] 34% run 20250422_095709; default; source-reported means; check counts retained Apr 2025 Official leaderboard
10 Claude 3.5 Haiku 32% run 20250422_095709; default; source-reported means; check counts retained Oct 2024 Official leaderboard
11 Gemini 2.0 Flash Feb 2025 30% run 20250422_095709; default; source-reported means; check counts retained Feb 2025 Official leaderboard
12 GPT-4o Aug 2024 26% run 20250422_095709; default; source-reported means; check counts retained Aug 2024 Official leaderboard
13 zoo-ml-text-to-cad 20% run 20250422_095709; default; source-reported means; check counts retained Apr 2025 Official leaderboard
14 GPT-4.1 mini 16% run 20250422_095709; default; source-reported means; check counts retained Apr 2025 Official leaderboard
15 Claude 3 Haiku 12% run 20250422_095709; default; source-reported means; check counts retained Mar 2024 Official leaderboard

The best score, 74%, costs $0.11 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Anthropic Google DeepMind OpenAI Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.0001 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale o3, run 20250422_095709; default; source-reported means; check counts retained Gemini 2.5 Pro, run 20250422_095709; default; source-reported means; check counts retained Claude 3.5 Sonnet (October 2024), run 20250422_095709; default; source-reported means; check counts retained GPT-4.1, run 20250422_095709; default; source-reported means; check counts retained Claude 3.5 Haiku, run 20250422_095709; default; source-reported means; check counts retained 0% 20% 40% 60% 80% 100% $0.0001 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale o3, run 20250422_095709; default; source-reported means; check counts retained Gemini 2.5 Pro, run 20250422_095709; default; source-reported means; check counts retained

Source: Official CadEval leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
o3 run 20250422_095709; default; source-reported means; check counts retained 74% $0.11 2,569
Gemini 2.5 Pro Mar 2025 run 20250422_095709; default; source-reported means; check counts retained 64% $0.04 4,037
o4-mini run 20250422_095709; default; source-reported means; check counts retained 62% $0.01 2,796
o1 run 20250422_095709; default; source-reported means; check counts retained 56% $0.25 4,046
o3-mini run 20250422_095709; default; source-reported means; check counts retained 54% $0.02 3,780
Claude 3.7 Sonnet run 20250422_095709; default; source-reported means; check counts retained 54% $0.03 1,727
Claude 3.5 Sonnet (October 2024) run 20250422_095709; default; source-reported means; check counts retained 48% $0.005 261
GPT-4.1 run 20250422_095709; default; source-reported means; check counts retained 42% $0.004 418
Claude 3.5 Haiku run 20250422_095709; default; source-reported means; check counts retained 32% $0.001 313
Gemini 2.0 Flash Feb 2025 run 20250422_095709; default; source-reported means; check counts retained 30% $0.0002 377
GPT-4o Aug 2024 run 20250422_095709; default; source-reported means; check counts retained 26% $0.004 300
GPT-4.1 mini run 20250422_095709; default; source-reported means; check counts retained 16% $0.0009 492
Claude 3 Haiku run 20250422_095709; default; source-reported means; check counts retained 12% $0.0007 502

2 models have held number one on the official leaderboard since its release in April 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind OpenAI
Own leaderboard Not tested since Apr 2025
o3 74%

Source: Official CadEval leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) o3 Gemini 2.5 Pro o1 Claude 3.5 Sonnet (October 2024) GPT-4o Claude 3 Haiku 0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) o3 Gemini 2.5 Pro

Source: Official CadEval leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Engineering and CAD

All 23 in Engineering and CAD
  • EEBench In use 13 electrical-engineering circuit and PCB design tasks graded by deterministic build checks, simulation and BOM constraints. No LLM judge. Gemini 4 Carbon 83.9%
  • Parametric CAD Bench In use Agents author editable parametric FreeCAD part designs, graded against held-back references for geometry and spec consistency. Claude Opus 5.5 61%
  • 3DCodeBench In use Agentic procedural 3D modeling via code: author engine-ready 3D assets through software APIs. Graded for executability and shape fidelity. LongCat-2.5 55.4%
  • CADGenBench Only in system cards Agents construct CAD models from design prompts. The generation split is graded for executability and geometric correctness. Claude Fable 5 45.1%
  • PartBench Only in system cards Parametric mechanical part design and CAD-style generation, graded for geometric and manufacturing fidelity. Grok 4.6 58.5%
  • LMBuild Not in model cards Language model agents design 3D structures as parts, joints, materials and assembly steps. Scoring checks that the structures can be built and work as intended. GPT-6 Astra 66.5%