PutnamBench
Leanstral 1.5 87.5% leads the models we track 
Seed-Prover 1.5 87.9% leads since December 2025 Models write formal Lean proofs of Putnam competition problems. A proof counts only when the verifier accepts it.
The Lean set now has 672 problems, and five systems solved all of them by September 2026, so the board ranks full solvers by cost. Labs often report Putnam-200, a fixed subset first used by Seed-Prover.
- Made by
- UT Austin (Tsoukalas et al.)
- Released
- July 2024
- Kind
- Formal proofs
- Level
- Competition
- Scored
- Checked automatically
- Answers
- Written proof
- Tasks
- All public
- Strongest model tested
-
DeepSeek-V4-Flash #86 on the capability index - Last in a launch post
- July 2026, dropped after it saturated
- Last model added
- July 2026
- Scores refreshed
- November 12th, 2026
In the authors’ words
We present PutnamBench, a new multi-language benchmark for evaluating the ability of neural theorem-provers to solve competition mathematics problems. PutnamBench consists of 1692 hand-constructed formalizations of 640 theorems sourced from the William Lowell Putnam Mathematical Competition, the premier undergraduate-level mathematics competition in North America. All the problems have formalizations in Lean 4 and Isabelle; a substantial subset also has Coq formalizations. PutnamBench requires significant problem-solving ability and proficiency in a broad range of topics taught in undergraduate mathematics courses. We use PutnamBench to evaluate several established neural and symbolic theorem-provers. These approaches can only solve a handful of the PutnamBench problems, establishing the benchmark as a difficult open challenge for research on neural theorem-proving. PutnamBench is available at https://github.com/trishullab/PutnamBench.
Aleph Prover (Logical Intelligence) tops the leaderboard with 100%
Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Leanstral 1.5, at 87.5%.
Seed-Prover 1.5 leads with 87.9%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.
The newest model with a score here came out in July 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- Claude Fable 5.2
-
Gemini 4 Carbon - GPT-6 Astra
- Claude Opus 5.5
- and 16 more
- 100.0 Aleph Prover (Logical Intelligence)
- 100.0 Humanfia
- 100.0 NEAR AI (w/ DeepSeek V4)
- 100.0 Forall (Astrio)
- 97.9 Mistral Prover (DeepSeek V4 Pro + Leanstral)
- 95.5 Goedel-Architect
- 87.5 Leanstral 1.5
- 86.5 Seed-Prover 1.5
- 68.8 Hilbert
- 54.3 AxProverBase (Axiomatic AI)
- 49.0 Seed-Prover
- 16.4 C2C-Goedel
- 13.5 Ax-Prover (Axiomatic AI)
- 12.8 Goedel-Prover-V2
- 9.67 Ensemble Prover (Graviterra)
- 6.99 DeepSeek-Prover-V2-671B
- 4.17 GPT-5 (ReAct, 10 turns)
- 3.72 C2C-Kimina
- 3.42 DSP+
- 2.08 Bourbaki
- 1.49 Kimina-Prover-7B-Distill
- 1.19 Self-play Theorem Prover
- 1.04 ABEL
- 1.04 Goedel-Prover-SFT
- 0.89 InternLM2.5-StepProver
- 0.60 InternLM-Math 7B
- 0.45 Gemini 2.5 Pro
- 0.30 o4-mini
- 0.15 COPRA (GPT-4o)
- 0.15 GPT-4o (snapshot unspecified)
- 0.15 Gemini 2.0 Flash Thinking
- 0.15 Deepseek R1 (snapshot unspecified)
- 0.0 ReProver w/ retrieval
- 0.0 ReProver w/o retrieval
- 0.0 GPT-4o mini
- 0.0 o3-mini
- 0.0 Claude 3.7 Sonnet
- 0.0 DeepSeek-V3
- 0.0 Grok-3 mini
- 1 Aleph Prover (Logical Intelligence) Avg $74, Max $1468 per problem 100%
- 2 Humanfia Avg $44.5 total cost/problem 100%
- 3 NEAR AI (w/ DeepSeek V4) Mean $0.17, median $0.04, max $11.18 per problem 100%
- 4 Forall (Astrio) Mean $3.52 actor-side cost/problem; median $2.49 100%
- 5 Mistral Prover (DeepSeek V4 Pro + Leanstral) Avg 5.38M tokens per solved problem 97.9%
- 6 Goedel-Architect Avg $5.8 total cost/problem 95.5%
- 7 Leanstral 1.5 4m tokens per problem 87.5%
- 8 Seed-Prover 1.5 10 H20 days per problem 86.5%
- 9 Hilbert avg pass@1840 68.8%
- 10 AxProverBase (Axiomatic AI) Avg $12.60, Opus 4.5 w/ 50 iterations 54.3%
- 11 Seed-Prover MEDIUM 49%
- 12 C2C-Goedel pass@256 16.4%
- 13 Ax-Prover (Axiomatic AI) pass@1, avg. 100 tool calls 13.5%
- 14 Goedel-Prover-V2 pass@184 12.8%
- 15 Ensemble Prover (Graviterra) Not reported 9.67%
- 16 DeepSeek-Prover-V2-671B pass@1024 6.99%
- 17 GPT-5 (ReAct, 10 turns) pass@1, 10 tool calls 4.17%
- 18 C2C-Kimina pass@256 3.72%
- 19 DSP+ pass@128 3.42%
- 20 Bourbaki pass@512 2.08%
- 21 Kimina-Prover-7B-Distill pass@192 1.49%
- 22 Self-play Theorem Prover pass@3200 1.19%
- 23 ABEL pass@596 1.04%
- 24 Goedel-Prover-SFT pass@512 1.04%
- 25 InternLM2.5-StepProver pass@2x32x600 0.89%
- 26 InternLM-Math 7B pass@4096 0.60%
- 27 Gemini 2.5 Pro Mar 2025, pass@1 0.45%
- 28 o4-mini pass@1 0.30%
- 29 COPRA (GPT-4o) pass@1 0.15%
- 30 GPT-4o (snapshot unspecified) pass@10 0.15%
- 31 Gemini 2.0 Flash Thinking Jan 2025, pass@1 0.15%
- 32 Deepseek R1 (snapshot unspecified) pass@1 0.15%
- 33 ReProver w/ retrieval pass@1 0%
- 34 ReProver w/o retrieval pass@1 0%
- 35 GPT-4o mini pass@1 0%
- 36 o3-mini pass@1 0%
- 37 Claude 3.7 Sonnet pass@1 0%
- 38 DeepSeek-V3 Mar 2025, pass@1 0%
- 39 Grok-3 mini pass@1 0%
Source: Official PutnamBench leaderboard
- 1 Seed-Prover 1.5 87.9%
- 2 Leanstral 1.5 4m tokens per problem 87.5%
- 3 DeepSeek-V4-Flash 81%
- 4 DiffusionGemma 26B A4B 67.4%
- 5 Seed-Prover 50.4%
- 6 Seed2.0 Pro 35.5%
- 7 DeepSeek-Prover-V2-671B 7.40%
- 8 Gemini 2.5 Pro Mar 2025, pass@1 0.45%
- 9 o4-mini pass@1 0.30%
- 10 Gemini 2.0 Flash Thinking Jan 2025, pass@1 0.15%
- 11 GPT-4o mini pass@1 0%
- 12 o3-mini pass@1 0%
- 13 Claude 3.7 Sonnet pass@1 0%
- 14 DeepSeek-V3 Mar 2025, pass@1 0%
- 15 Grok-3 mini pass@1 0%
Sources: Labs’ model cards and official PutnamBench leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Aleph Prover (Logical Intelligence) | 100% | Avg $74, Max $1468 per problem | Aug 2026 | Official leaderboard |
| 2 | Humanfia | 100% | Avg $44.5 total cost/problem | Aug 2026 | Official leaderboard |
| 3 | NEAR AI (w/ DeepSeek V4) | 100% | Mean $0.17, median $0.04, max $11.18 per problem | Sep 2026 | Official leaderboard |
| 4 | Forall (Astrio) | 100% | Mean $3.52 actor-side cost/problem; median $2.49 | Sep 2026 | Official leaderboard |
| 5 | Mistral Prover (DeepSeek V4 Pro + Leanstral) | 97.9% | Avg 5.38M tokens per solved problem | Sep 2026 | Official leaderboard |
| 6 | Goedel-Architect | 95.5% | Avg $5.8 total cost/problem | Jul 2026 | Official leaderboard |
| 7 | Leanstral 1.5 | 87.5% | 4m tokens per problem | Jul 2026 | Official leaderboard |
| 8 | Seed-Prover 1.5 | 86.5% | 10 H20 days per problem | Dec 2025 | Official leaderboard |
| 9 | Hilbert | 68.8% | avg pass@1840 | Oct 2025 | Official leaderboard |
| 10 | AxProverBase (Axiomatic AI) | 54.3% | Avg $12.60, Opus 4.5 w/ 50 iterations | Apr 2026 | Official leaderboard |
| 11 | Seed-Prover | 49% | MEDIUM | Jul 2025 | Official leaderboard |
| 12 | C2C-Goedel | 16.4% | pass@256 | Aug 2026 | Official leaderboard |
| 13 | Ax-Prover (Axiomatic AI) | 13.5% | pass@1, avg. 100 tool calls | Dec 2025 | Official leaderboard |
| 14 | Goedel-Prover-V2 | 12.8% | pass@184 | Jul 2025 | Official leaderboard |
| 15 | Ensemble Prover (Graviterra) | 9.67% | Not reported | Sep 2026 | Official leaderboard |
| 16 | DeepSeek-Prover-V2-671B | 6.99% | pass@1024 | Apr 2025 | Official leaderboard |
| 17 | GPT-5 (ReAct, 10 turns) | 4.17% | pass@1, 10 tool calls | Nov 2025 | Official leaderboard |
| 18 | C2C-Kimina | 3.72% | pass@256 | Aug 2026 | Official leaderboard |
| 19 | DSP+ | 3.42% | pass@128 | Jun 2025 | Official leaderboard |
| 20 | Bourbaki | 2.08% | pass@512 | Jul 2025 | Official leaderboard |
| 21 | Kimina-Prover-7B-Distill | 1.49% | pass@192 | Apr 2025 | Official leaderboard |
| 22 | Self-play Theorem Prover | 1.19% | pass@3200 | Feb 2025 | Official leaderboard |
| 23 | ABEL | 1.04% | pass@596 | Oct 2024 | Official leaderboard |
| 24 | Goedel-Prover-SFT | 1.04% | pass@512 | Jan 2025 | Official leaderboard |
| 25 | InternLM2.5-StepProver | 0.89% | pass@2x32x600 | Nov 2024 | Official leaderboard |
| 26 | InternLM-Math 7B | 0.60% | pass@4096 | Jul 2024 | Official leaderboard |
| 27 | Gemini 2.5 Pro Mar 2025 | 0.45% | pass@1 | Mar 2025 | Official leaderboard |
| 28 | o4-mini | 0.30% | pass@1 | Apr 2025 | Official leaderboard |
| 29 | COPRA (GPT-4o) | 0.15% | pass@1 | Jul 2024 | Official leaderboard |
| 30 | GPT-4o (snapshot unspecified) | 0.15% | pass@10 | Jul 2024 | Official leaderboard |
| 31 | Gemini 2.0 Flash Thinking Jan 2025 | 0.15% | pass@1 | Jan 2025 | Official leaderboard |
| 32 | Deepseek R1 (snapshot unspecified) | 0.15% | pass@1 | Mar 2025 | Official leaderboard |
| 33 | ReProver w/ retrieval | 0% | pass@1 | Jul 2024 | Official leaderboard |
| 34 | ReProver w/o retrieval | 0% | pass@1 | Jul 2024 | Official leaderboard |
| 35 | GPT-4o mini | 0% | pass@1 | Jul 2024 | Official leaderboard |
| 36 | o3-mini | 0% | pass@1 | Jan 2025 | Official leaderboard |
| 37 | Claude 3.7 Sonnet | 0% | pass@1 | Feb 2025 | Official leaderboard |
| 38 | DeepSeek-V3 Mar 2025 | 0% | pass@1 | Mar 2025 | Official leaderboard |
| 39 | Grok-3 mini | 0% | pass@1 | Apr 2025 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Seed-Prover 1.5 | 87.9% | Dec 2025 | Lab model card | |
| 2 | Leanstral 1.5 | 87.5% | 4m tokens per problem | Jul 2026 | Official leaderboard |
| 3 | DeepSeek-V4-Flash | 81% | Apr 2026 | Lab model card | |
| 4 | DiffusionGemma 26B A4B | 67.4% | Jul 2026 | Lab model card | |
| 5 | Seed-Prover | 50.4% | Jul 2025 | Lab model card | |
| 6 | Seed2.0 Pro | 35.5% | Feb 2026 | Lab model card | |
| 7 | DeepSeek-Prover-V2-671B | 7.40% | Apr 2025 | Lab model card | |
| 8 | Gemini 2.5 Pro Mar 2025 | 0.45% | pass@1 | Mar 2025 | Official leaderboard |
| 9 | o4-mini | 0.30% | pass@1 | Apr 2025 | Official leaderboard |
| 10 | Gemini 2.0 Flash Thinking Jan 2025 | 0.15% | pass@1 | Jan 2025 | Official leaderboard |
| 11 | GPT-4o mini | 0% | pass@1 | Jul 2024 | Official leaderboard |
| 12 | o3-mini | 0% | pass@1 | Jan 2025 | Official leaderboard |
| 13 | Claude 3.7 Sonnet | 0% | pass@1 | Feb 2025 | Official leaderboard |
| 14 | DeepSeek-V3 Mar 2025 | 0% | pass@1 | Mar 2025 | Official leaderboard |
| 15 | Grok-3 mini | 0% | pass@1 | Apr 2025 | Official leaderboard |
Labs report scores for 7 models in their cards. Switch the scores above to rank them together.
7 models have held number one on the official leaderboard since its release in July 2024 6 models have held number one with claims included since its release in July 2024
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Sources: Official PutnamBench leaderboard and labs’ model cards
ByteDance Seed’s own number for Seed-Prover 1.5 passed 85% in December 2025
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official PutnamBench leaderboard and labs’ model cards
Labs headline it less often now, last in July 2026
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Jul 2, 2026 | Leanstral 1.5 | 87.4 | PutnamBench 587/672 solved (pass@8, 4M tokens per attempt) |
| Feb 14, 2026 | Seed2.0 Pro | 35.5 | Putnam-200 Pass@8, agentic Lean setup (Table 5) |
| Dec 19, 2025 | Seed-Prover 1.5 | 87.9 | PutnamBench 580/660 (87.9%) |
| Jul 31, 2025 | Seed-Prover | 50.4 | PutnamBench 331/657 (Medium) |
| Apr 30, 2025 | DeepSeek-Prover-V2-671B | 7.4 | PutnamBench 49 out of 658 problems solved |
5 system cards report it, from 4 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Jul 31, 2026 | DiffusionGemma 26B A4B technical report | 67.4 | Putnam, cited as PutnamBench (Tsoukalas et al., 2024), TD, 57.1 no-think, scoring setup not stated (Table 3) | Capabilities |
| Jul 2, 2026 | Leanstral 1.5 technical report | 87.4 | PutnamBench 587/672 solved, pass@8, 4M-token limit per trajectory, $1.68 per problem (Table 1) | Capabilities |
| Apr 26, 2026 | DeepSeek-V4-Flash technical report | 81 | Putnam-200 Pass@8, fixed random subset of PutnamBench, agentic Lean setting with LeanExplore, DeepSeek-V4-Flash-Max (Figure 8 left) | Capabilities |
| Feb 14, 2026 | Seed2.0 Pro model card | 35.5 | Putnam-200, a 200-problem PutnamBench subset, Pass@8, agentic Lean setup (Table 5) | Capabilities |
| Apr 30, 2025 | DeepSeek-Prover-V2-671B technical report | 7.14 | PutnamBench, CoT, sample budget 1024, 47 of 658 solved (Table 4). Version 1 reported 49 | Capabilities |
The lab's number comes out ahead in 3 of 3 pairs that differ
Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 1 of them match, which usually means one side copied the other. Every benchmark.
| Model | Lab says | Board says | Gap |
|---|---|---|---|
| Seed-Prover 1.5 | 87.9% PutnamBench 580/660 (87.9%) | 86.5% Leaderboard, 10 H20 days per problem | +1.4 pts |
| Seed-Prover | 50.4% PutnamBench 331/657 (Medium) | 49% Leaderboard, MEDIUM | +1.4 pts |
| DeepSeek-Prover-V2-671B | 7.40% PutnamBench 49 out of 658 problems solved | 6.99% Leaderboard, pass@1024 | +0.4 pts |
| Leanstral 1.5 | 87.4% PutnamBench 587/672 solved (pass@8, 4M tokens per attempt) | 87.5% Leaderboard, 4m tokens per problem | Same |
More in Math
- AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
- IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
- HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
- PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked.
Qwen3.7-Max 86.5% - MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
- MathArena Apex Shortlist In use Models solve 47 hard final-answer problems that MathArena selected from 2025 competitions, where the best models scored about 50% at selection. Scored by accuracy over repeated attempts. GPT-5.5 98.4%