Benchmark Almanac

Math

We track 130 benchmarks in math. They run from everyday level to research frontier level.

26 in use 48 past 85% 70 with public code

OpenAI held the top math score 55% of the contested time

That's about 25 of the 46 months since January 2023, averaged over these benchmarks. We only count time while other labs are still testing, and stop once a model passes 85%. Faded stretches no longer count.

OpenAI Google DeepMind Anthropic Meta Other labs Qwen Tencent Deep Cogito DeepSeek Z.ai Meituan xAI Moonshot AI Microsoft Ant Group
School Students, from grade school to high school
GSM8K Passed 85% in Mar 2023, by GPT-4
Llama 3.1-405B 96.8%
Competition The strongest students, in olympiad-style contests
MATH Passed 85% in Dec 2024, by Gemini 2.0 Flash
Cogito v2 Preview 671B MoE 98.2%
MATH Level 5 Passed 85% in Sep 2024, by o1-mini
GPT-5 98.1%
MATH-500 Passed 85% in Sep 2024, by o1-preview
LongCat-Flash-Thinking 99.2%
AIME 2026 Passed 85% in Feb 2026, by GPT-5.2
GPT-5.5 100%
OTIS Mock AIME 2024-2025 Passed 85% in Jul 2025, by Qwen3-235B-A22B-Thinking
GPT-5.5 100%
PolyMath Passed 85% in May 2026, by Qwen3.7-Max
Qwen3.7-Max 86.5%
HMMT 2026 Passed 85% in Feb 2026, by GPT-5.2
GPT-5.5 98.5%
IMO-AnswerBench Passed 85% in Jan 2026, by LongCat-Flash-Thinking-2601
GLM-5.2 91%
Expert Specialists with years of training, often over hours of work
FrontierMath Tiers 1-3 v2 Passed 85% in Jun 2026, by GPT-5.5 Pro
Claude Fable 5.2 95.1%
Research frontier Researchers, on open problems or tasks that take days
FrontierMath Tier 4 v2 Passed 85% in Jun 2026, by Claude Fable 5
GPT-6.1 Sol 100%

Sources: Official leaderboards, Epoch AI runs (CC BY 4.0) and labs’ model cards

Each benchmark counts from the one source its own page leads with: its official leaderboard when that is current, else Epoch AI’s own runs, else labs’ own numbers. We date independent runs by the model’s release and lab reports by when they came out.

OpenAI Anthropic xAI Google DeepMind Qwen Z.ai DeepSeek
School Students, from grade school to high school
GSM8K Not tested since Sep 2023
davinci-002 56.8%
Competition The strongest students, in olympiad-style contests
MATH Level 5 Passed 85% in Sep 2024, by o1-mini
GPT-5 98.1%
MATH-500 Passed 85% in Dec 2024, by o1
Gemini 3 Pro 96.4%
OTIS Mock AIME 2024-2025 Passed 85% in Jul 2025, by Qwen3-235B-A22B-Thinking
GPT-5.5 100%
AIME 2026 Passed 85% in Feb 2026, by GPT-5.2
GPT-5.5 100%
HMMT 2026 Passed 85% in Feb 2026, by GPT-5.2
GPT-5.5 98.5%
PolyMath Not tested since Apr 2025
Qwen3-235B-A22B 54.6%
MathArena (Apex, IMO, USAMO) Not tested since Jul 2026
Claude Opus 4.8 81.3%
HMMT Nov 2025 Passed 85% in Nov 2025, by GLM-4.6
GPT-5.2 99.2%
Expert Specialists with years of training, often over hours of work
FrontierMath Tiers 1-3 v2 Passed 85% in Jun 2026, by GPT-5.5 Pro
Claude Fable 5.2 95.1%
Research frontier Researchers, on open problems or tasks that take days
FrontierMath Tier 4 v2 Passed 85% in Jun 2026, by Claude Fable 5
GPT-6.1 Sol 100%

Sources: Official leaderboards, Epoch AI runs (CC BY 4.0) and boards copied by Epoch AI (CC BY 4.0)

Every independent run: each benchmark’s own leaderboard, Epoch AI’s runs and boards Epoch AI copied, dated by the model’s release. We leave out benchmarks that only labs report.

OpenAI Google DeepMind Anthropic Meta Other labs Qwen Tencent Deep Cogito DeepSeek Z.ai Meituan xAI Moonshot AI Ant Group
School Students, from grade school to high school
GSM8K Passed 85% in Mar 2023, by GPT-4
Llama 3.1-405B 96.8%
Competition The strongest students, in olympiad-style contests
MATH Passed 85% in Dec 2024, by Gemini 2.0 Flash
Cogito v2 Preview 671B MoE 98.2%
MATH Level 5 Passed 85% in Sep 2024, by o1-mini
GPT-5 98.1%
MATH-500 Passed 85% in Sep 2024, by o1-preview
LongCat-Flash-Thinking 99.2%
AIME 2026 Passed 85% in Feb 2026, by GPT-5.2
GPT-5.5 100%
OTIS Mock AIME 2024-2025 Passed 85% in Jul 2025, by Qwen3-235B-A22B-Thinking
GPT-5.5 100%
HMMT 2026 Passed 85% in Feb 2026, by GPT-5.2
GPT-5.5 98.5%
PolyMath Passed 85% in May 2026, by Qwen3.7-Max
Qwen3.7-Max 86.5%
IMO-AnswerBench Passed 85% in Jan 2026, by LongCat-Flash-Thinking-2601
GLM-5.2 91%
Expert Specialists with years of training, often over hours of work
FrontierMath Tiers 1-3 v2 Passed 85% in Jun 2026, by GPT-5.5 Pro
Claude Fable 5.2 95.1%
Research frontier Researchers, on open problems or tasks that take days
FrontierMath Tier 4 v2 Passed 85% in Jun 2026, by Claude Fable 5
GPT-6.1 Sol 100%

Sources: Official leaderboards, Epoch AI runs (CC BY 4.0), boards copied by Epoch AI (CC BY 4.0), labs’ model cards and claims collected by Epoch AI (CC BY 4.0)

Every score, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

From everyday to research frontier level

Grouped by who could solve the tasks, from things most adults can do to open research problems. The bar under each name is its best score.

Everyday

Most adults, without training

School

Students, from grade school to high school

University

Undergraduates and entry-level professionals

Competition

The strongest students, in olympiad-style contests

Expert

Specialists with years of training, often over hours of work

Research frontier

Researchers, on open problems or tasks that take days

By kind

Word problems

BenchmarkLevelScoredBest scoreStatus
GSM8K 8.5K grade-school math word problems. Final numeric answer scored by exact match. School Checked automatically 96.8% Fading
U-MATH Models solve 1,100 unpublished open-ended university math problems from teaching materials, balanced across six subjects. About 20% include images, and an LLM judge checks each solution. University Graded by a model 86.8% Not in cards
SnorkelSequences Models compute values over integer sequences built from chains of filters, transforms and operators. The 250 procedurally generated questions test compositional multi-step math. School Checked automatically 77.6% Not in cards
SnorkelGraph Models solve 200 procedurally generated graph problems, such as minimum vertex cover, given as node and edge lists in text. Answers are checked against verifiable ground truth. University Checked automatically 84.5% Not in cards
MathBench Models answer 3,709 math questions in five stages from basic arithmetic to college, in Chinese and English. Each stage also tests theory knowledge, and Circular Evaluation reshuffles answer options to check consistency. University Checked automatically 70.9% Not in cards
GSM8K Platinum Initial 268 Models answer the original 268-question cleaned GSM8K subset. The live matrix reports each model configuration separately. School Checked automatically 99.6% Not in cards
GSM1k Models solve new grade school math word problems written to mirror GSM8K in style and difficulty. The gap between GSM8K and GSM1k accuracy shows overfitting or contamination. School Checked automatically 96.6% Not in cards
CMATH 1.7K Chinese elementary-school math word problems. Final-answer accuracy. School Checked automatically 96.7% Fading
GSM8K-Platinum Models solve the GSM8K test problems after MIT researchers fixed wrong labels and removed badly written questions. Errors on it are more likely real model mistakes. School Checked automatically 99.8% System cards only
UGMathBench Models solve 5,062 undergraduate math problems across 16 subjects. Each problem has three randomized versions, and effective accuracy counts problems solved in all versions. University Checked automatically 81.9% System cards only
DeepMind Mathematics Models answer generated school-level math questions across algebra, arithmetic, calculus, probability and other modules. Answers are checked for an exact match. School Checked automatically 81.9% Fading
GRE Quantitative Models take the GRE quantitative reasoning section. Scored on the 130 to 170 scale. University Checked automatically 163.0 System cards only
GSM-Symbolic Models solve grade-school word problems generated from symbolic templates of GSM8K questions, with names and numbers changed. P1 and P2 add one or two extra clauses. School Checked automatically 87.4% Fading
HRM8K Models solve 8,011 math problems given as English and Korean parallel pairs. It measures how much math reasoning drops when problems are in Korean. School Checked automatically 92.2% Fading
Hungarian National High-School Exam Models answer the 33 questions of the 2023 Hungarian national high school mathematics finals. Answers are graded by hand with the official rubric. School Graded by people 59% Dropped
SuperCLUE-Math6 Models solve thousands of multi-step Chinese math word problems. Scores combine reasoning-step quality and accuracy into a comprehensive score and a reasoning level. School Graded by a model 83.3 System cards only
GMAT Quantitative Models answer the quantitative section of the official GMAT online exam. Scored as accuracy. University Checked automatically 96% System cards only
GSM-Plus Models solve GSM8K problems after perturbations such as changed numbers, added distractors and reversed questions. School Checked automatically 64.8% Dropped
Math23K Models solve Chinese elementary-school math word problems from a set of about 23,000. Scored by final-answer accuracy. School Checked automatically 88.5% System cards only
Math401 Models compute 401 arithmetic expressions covering addition, multiplication, exponents, logarithms and trigonometry. Scored by exact numeric accuracy. Everyday Checked automatically 70.1% System cards only
MathQA Models answer multiple-choice math word problems drawn from AQuA. Each problem is annotated with an operation program. Scored by accuracy. School Checked automatically 33.9% System cards only
SAT Math Models take the SAT math section. Scored on the 200 to 800 scale. School Checked automatically 700.0 System cards only
MAWPS Models solve one-unknown arithmetic and algebra word problems drawn from several earlier collections. Each problem has an equation and a numeric answer, and the final answer is scored by exact match. School Checked automatically Not in cards
SVAMP Models solve 1,000 grade-school arithmetic word problems. The problems are variations of problems from earlier datasets, made to catch solvers that ignore the question or the word order. The final numeric answer is scored by exact match. School Checked automatically Not in cards

Competition problems with answers

BenchmarkLevelScoredBest scoreStatus
AIME 2025 Models answer the 30 problems from the 2025 AIME I and II exams. Each answer is an integer from 0 to 999, scored by match. Competition Checked automatically 100% In use
OTIS Mock AIME 2024-2025 Models solve 45 problems from three OTIS Mock AIME exams written by students of the OTIS program. Integer answers are scored by exact match. Competition Checked automatically 100% Not in cards
MATH 12,500 competition math problems with step-by-step solutions. Final answer scored by exact match. Competition Checked automatically 98.2% Fading
AIME 2024 Models solve the 30 problems of the 2024 AIME I and II. Answers are integers from 0 to 999, scored by exact match. Competition Checked automatically 96.6% Fading
MATH-500 Models solve 500 competition math problems drawn from the MATH test set. Final answers are checked against the reference answers. Competition Checked automatically 99.2% Fading
HMMT 2025 Models answer the 30 problems from the February 2025 Harvard-MIT Mathematics Tournament. Final answers are checked against the official answers. Competition Checked automatically 100% Fading
AIME 2026 Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. Competition Checked automatically 100% In use
MATH Level 5 Models solve the 1,324 hardest MATH test problems, drawn from competitions such as AMC and AIME. Scored by final answer match. Competition Checked automatically 98.1% Dropped
PolyMath Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Competition Checked automatically 86.5% In use
IMO-AnswerBench 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. Competition Checked automatically 91% In use
HMMT Nov 2025 Models solve the 30 problems of the November 2025 Harvard-MIT Mathematics Tournament. Final answers are checked automatically against the answer key. Competition Checked automatically 99.2% Fading
MathArena (Apex, IMO, USAMO) Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Competition Checked automatically 81.3% In use
HMMT 2026 Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. Competition Checked automatically 98.5% In use
MathArena Apex Shortlist Models solve 47 hard final-answer problems that MathArena selected from 2025 competitions, where the best models scored about 50% at selection. Scored by accuracy over repeated attempts. Competition Checked automatically 98.4% In use
BRUMO 2025 Models answer the 30 final-answer problems of the 2025 Brown University Math Olympiad. MathArena scores the average accuracy over four attempts. Competition Checked automatically 100% Not in cards
SMT 2025 Models answer 53 final-answer problems from the 2025 Stanford Math Tournament. MathArena scores the average accuracy over four attempts. Competition Checked automatically 93.4% Not in cards
CMIMC 2025 Models answer 40 final-answer problems from the 2025 Carnegie Mellon Informatics and Mathematics Competition. MathArena scores the average accuracy over four attempts. Competition Checked automatically 95.6% Not in cards
AMO-Bench Models solve 50 original math problems at or above IMO difficulty that need only a final answer. Answers are graded automatically for accuracy. Competition Checked automatically 52.4% Fading
Beyond AIME Models answer 100 original competition-style math problems at or above the hardest AIME level. Each answer is a positive integer checked by exact match. Competition Checked automatically 87% Fading
CNMO 2024 Models answer problems from the 2024 Chinese National High School Mathematics Olympiad. DeepSeek reports Pass@1 averaged over 16 samples. Competition Checked automatically 88.5% Dropped
HiddenMath Models solve 179 competition-level math problems that experts wrote from scratch. Answers are graded automatically. Competition Checked automatically 80.6% Dropped
Omni-MATH Models solve 4,428 Olympiad-level competition math problems across 33 subdomains and 10 difficulty levels. A GPT-4 or Omni-Judge grader checks answers. Competition Checked automatically 60.5% Dropped
OlympiadBench Models solve bilingual olympiad-level math and physics problems in Chinese and English, many with images. Answers are checked against a key. Competition Checked automatically 18% Dropped
AMC Models solve problems from the AMC 8, 10, and 12 competitions. Gemini 1.5 used 250 problems from 2022 and 2023, scored by accuracy. Competition Checked automatically 69.4% Fading
LiveMathBench Models solve recent competition math problems, sampled many times. The G-Pass@k metric rewards both peak accuracy and stable answers across samples. Competition Checked automatically 61.2% Not in cards
HRM8K-KSM Models solve the 1,428 problems of the Korean School Math subset of HRM8K. Problems come from Korean olympiad and competition exams. Competition Checked automatically 91.1% Fading
Ko-AIME 2025 (Upstage) Models solve AIME 2025 competition problems in Korean. Upstage built the set in house. Competition Checked automatically 97.7% Fading
OMEGA Models solve generated competition-style math problems that test exploratory, compositional and transformative generalization beyond training patterns. Answers are checked automatically. Competition Checked automatically 53.4% Fading
Project Euler (MathArena) Models solve recent Project Euler problems that need both mathematical insight and code. MathArena adds problems as Project Euler releases them, starting from problem 943. Expert Checked automatically 89% Not in cards
AMC 10 2023 Models take the 2023 AMC 10 A and B exams. Scored out of 150 points. Competition Checked automatically 72.0 System cards only
AMC 12 2023 Models take the 2023 AMC 12 A and B exams. Scored out of 150 points. Competition Checked automatically 63.0 System cards only
AMC 8 2023 Models take the 25 question 2023 AMC 8 exam. Scored out of 150 points. Competition Checked automatically 84.0 System cards only
QuantBench Models answer 25 multiple choice versions of quant trading firm reasoning puzzles. Scored as cons@16 accuracy. Competition Checked automatically 60% System cards only
AIME 1983-2024 Models solve 949 past AIME problems from 1983 to 2024. Integer answers are checked exactly. Competition Checked automatically 89.4% System cards only
CSAT Math 2025 Models solve the mathematics sections of South Korea's 2025 College Scholastic Ability Test in Korean. The score averages the three elective subjects: calculus, statistics and geometry. School Checked automatically 94.5% Dropped
Functional MATH Models solve new instances of MATH problems that code generates for each snapshot. The gap from static MATH accuracy measures memorization. Competition Checked automatically 64.6% Dropped
Gaokao Math 2026 Models sit the five 2026 Chinese Gaokao mathematics papers (National I, National II, Beijing, Shanghai and Tianjin). Each paper is graded out of 150 points. School Checked automatically 134.9 In use
HiddenMath-Hard Models solve the harder part of an expert-written, never-published set of AIME and AMC style competition problems. Competition Checked automatically 80.5% System cards only
Hy-Math Models solve Tencent in-house math problems without tools. Task and scoring details are not published. Expert Checked automatically 60.9% In use
AMC 10 2022 Models take a 2022 AMC 10 exam. Scored out of 150 points. Competition Checked automatically 30.0 System cards only
AMC 10/12 (November 2024) Models take the four November 2024 AMC 10 and 12 contests, 78 questions in all. Scored in contest points out of 150, averaged over tests. Competition Checked automatically 91.8 System cards only
AMC 12 2022 Models take a 2022 AMC 12 exam. Scored out of 150 points. Competition Checked automatically 60.0 System cards only
AMC 12 2024 Models answer the 25 multiple-choice problems of the 2024 AMC 12 contests. Scored on the official 150-point scale. Competition Checked automatically 143.7 System cards only
Ko-AIME 2024 (Upstage) Models solve AIME 2024 competition problems translated into Korean. Upstage built the set in house. Competition Checked automatically 80.3% System cards only
AIME 2023 Models solve the 30 problems of the 2023 AIME I and II. Answers are integers from 0 to 999, scored by exact match. Competition Checked automatically Not in cards
HMMT February 2024 Models answer the problems from the February 2024 Harvard-MIT Mathematics Tournament. Final answers are checked against the official answers. Competition Checked automatically Not in cards
LiveMath Models solve 250 math problems written with six professional mathematicians. The sets are 150 regular high school problems, 50 at about AIME level and 50 multimodal problems with figures. Competition Checked automatically Not in cards
ProcessBench Models read step-by-step solutions to GSM8K, MATH, OlympiadBench and Omni-MATH problems and name the earliest wrong step, or say none is wrong. Scored by F1. Competition Checked automatically Not in cards

Written proofs

BenchmarkLevelScoredBest scoreStatus
IneqMath Models solve Olympiad-level inequality problems, recast as two checkable tasks: estimating a tight bound and predicting the relation between two expressions. IMO-level medalists wrote and reviewed the 200 test problems. Five model judges check the final answer and look for toy-case reasoning, logical gaps and numerical shortcuts, and a solution counts only if it passes all five. Competition Graded by a model 47% Not in cards
IMO-ProofBench Models write proofs for 60 olympiad problems, split into a basic set and an advanced set up to IMO-hard. Each proof is graded from 0 to 7 by experts or a Gemini autograder. Competition Graded by people 88.1% Fading
IMO 2025 Models write full proofs for the six problems of the 2025 International Mathematical Olympiad. Graders score proofs out of 42 points, mapped to medal cutoffs. Competition Graded by people 83.3% Fading
USAMO 2026 Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. Competition Graded by people 98.2% In use
USAMO 2025 Models write full proofs for the 6 problems of the 2025 USA Mathematical Olympiad. Expert human graders score each proof. Competition Graded by people 30.1% Dropped
Putnam 2025 Models solve the 12 problems of the 2025 William Lowell Putnam competition with full proofs. Scored out of 120 points. Competition Graded by people 120.0 Fading
Putnam-like Models write proofs for 96 university problems in 8 sets of 12, in the style of the easier and middle Putnam tiers. Each problem has a Lean statement and a point rubric for grading. University Graded by a model 94.5% Not in cards
CMO 2025 Models write full proofs for the six problems of the 2025 China Mathematical Olympiad. Graders score proofs out of 126 points. Competition Graded by people 114.0 Fading
AdvancedMathBench Models write proofs for 245 advanced problems at undergraduate and doctoral qualifying exam levels. A verifier trained on expert annotations grades each proof. Expert Graded by a model 58.1% In use
Putnam 2024 Models write full proofs for the 12 problems of the December 2024 William Lowell Putnam competition. Proofs are graded out of 120 points. Competition Graded by people 98.3% Dropped
CMO 2024 Models write full proofs for the six problems of the 2024 China Mathematical Olympiad. Experts grade the proofs for points. Competition Graded by people 73.8% System cards only
IMC 2025 Models write proofs for the 10 problems of the 2025 International Mathematics Competition for University Students. Human judges grade each proof. University Graded by people 94.5% Not in cards
IMO 2026 Models write full proofs for the six problems of the 2026 International Mathematical Olympiad. Proofs are scored out of 42 points. Competition Graded by people 100% System cards only
IMO Shortlist 2024 Models write proofs for the 31 problems of the 2024 IMO Shortlist. DeepSeek scores the proofs with its own verifier model. Competition Graded by a model 27% System cards only
Miklós Schweitzer 2025 Models write proofs for the 2025 Miklós Schweitzer competition, a hard Hungarian university-level contest. MathArena runs and grades it. Research frontier Graded by people 90% Not in cards
BrokenMath Models are asked to prove false versions of 2025 competition problems. An LLM judge counts how often a model writes a proof instead of rejecting the statement. Competition Graded by a model Not in cards
CNML-Level Problems (DeepSeek) Models prove 91 theorems in algebra, geometry, number theory, combinatorics and inequalities at Chinese National High School Mathematics League level. A verifier model scores the proofs. Competition Graded by a model System cards only
IMO-GradingBench Models grade 1,000 olympiad proof attempts, and their grades are compared with human expert grades. Tests automatic evaluation of long-form math answers. Competition Checked automatically Not in cards
ProofRank Models write proofs for hard competition problems. An LLM judge compares correct proofs pairwise on conciseness, computational ease, simplicity, diversity and adaptivity, giving Elo ratings. Competition Graded by a model Not in cards
USAMO 2024 Models write full proofs for problems from the 2024 USA Mathematical Olympiad, graded by human experts. Competition Graded by people Not in cards

Formal proofs

BenchmarkLevelScoredBest scoreStatus
ProofBench Models write Lean 4 proofs of graduate-level theorems, with Mathlib search and code tools for up to 40 turns. Scored by the share that compile. Expert Checked automatically 78% Not in cards
ProofBench v1.1 Models write Lean 4 proofs of graduate-level theorems from paired natural-language and formal statements. Scored by the share the Lean checker accepts. Expert Checked automatically 100% Not in cards
PutnamBench Models write formal Lean proofs of Putnam competition problems. A proof counts only when the verifier accepts it. Competition Checked automatically 87.5% Fading
miniF2F Models write formal proofs of olympiad and high school competition statements, such as AMC, AIME and IMO problems. A proof assistant checks each proof, and scores are pass@k on the 244 test problems. Competition Checked automatically 100% Fading
ArXivLean Models prove or disprove theorems from recent arXiv papers, stated in Lean 4. A strict comparator checks each submitted proof against the fixed reference statement. Research frontier Checked automatically 47.2% System cards only
CombiBench Provers solve 100 combinatorics problems formalized in Lean 4, from middle school to IMO and university level. Lean checks each proof. Competition Checked automatically 48% Fading
FLTEval Agents complete the formal Lean proofs and define the new mathematical concepts in real pull requests to the Fermat's Last Theorem (FLT) formalization project. Lean and SafeVerify check each submitted diff. Research frontier Checked automatically 43.2 Fading
miniCTX-v2 Provers prove Lean theorems from real projects using long repository context, with theorems taken after a November 28, 2024 cutoff. Expert Checked automatically 81.8% Dropped
FATE-H Provers write Lean proofs for 100 abstract and commutative algebra problems at honors course or graduate exam level. The Lean compiler checks each proof. Expert Checked automatically 87% Fading
FATE-X Provers write Lean proofs for 100 algebra problems at or beyond PhD qualifying exam level, some beyond what Mathlib covers. The Lean compiler checks each proof. Research frontier Checked automatically 34% Fading
Past IMO (Seed-Prover formal set) Provers write Lean proofs for 155 past IMO problems, mostly adapted from Compfiles and MiniF2F. Competition Checked automatically 78.1% Dropped
ProverBench (DeepSeek) Provers write Lean 4 proofs for 325 formalized problems: 15 from AIME 2024 and 2025 and 310 from textbooks across number theory, algebra, calculus and analysis. Lean checks each proof. University Checked automatically 59.1% Dropped
Fermat's Last Theorem in Lean (Anthropic) Agent teams formalize Fermat's Last Theorem in Lean. Scores measure the weighted proportion of 60 accepted intermediate theorems. Research frontier Checked automatically 35% System cards only
FormalMATH Provers write Lean 4 proofs for 5,560 formal statements from high-school olympiad problems to undergraduate theorems. The Lean checker scores each proof. Competition Checked automatically 28.3% System cards only
FormalMATH Lite Models prove statements in the FormalMATH Lite subset. The board reports proof success at the stated sampling budget. Competition Checked automatically 61.9% Not in cards
IMO-LeanProofBench Models write Lean 4 proofs for the 60 IMO-ProofBench problems, formalized by hand and checked by Lean experts. The Lean compiler verifies each proof. Competition Checked automatically 11.6% Not in cards
ProofNet Provers write formal proofs for 371 undergraduate theorems from analysis, algebra and topology textbooks, each with a formal statement and a natural-language proof. The proof checker grades them. University Checked automatically 37.1% System cards only
LeanDojo Benchmark Models prove theorems from Lean's mathlib library inside the Lean proof assistant, which checks every proof. The benchmark has 98,734 theorems and proofs with premise annotations. A novel-premises split tests whether a prover can use lemmas it never saw in training. University Checked automatically Not in cards
OpenFrontiers-Bench Agents try to rediscover proofs of research math problems, most resolved during 2026, stated in Lean 4. Two independent checkers grade the formal proofs, with closed-book, frozen-retrieval and live-web settings. Research frontier Checked automatically Not in cards

Research mathematics

BenchmarkLevelScoredBest scoreStatus
FrontierMath Tiers 1-3 v2 Models solve 295 unpublished math problems from undergraduate to advanced graduate level and submit answers that code checks automatically. Scored by share correct. Expert Checked automatically 95.1% In use
FrontierMath Unpublished research-level math problems with automatically verifiable answers. Tier 4 is the hardest tier. Expert Checked automatically 50% Fading
FrontierMath Tier 4 v2 Models solve 43 unpublished research-level math problems and submit answers that code checks automatically. Scored by share correct. Research frontier Checked automatically 100% In use
FrontierMath Tier 4 Models solve research-level math problems that take experts days, submitting answers through a Python function. Each answer is verified automatically as correct or incorrect. Research frontier Checked automatically 47.9% Fading
Riemann-bench 25 private research-mathematics problems from professors and IMO medalists. Agents with tools, 100 runs per problem. Research frontier Checked automatically 83.2% In use
Leipzig Benchmark Models answer 100 research-level math questions with known answers, written by 49 mathematicians at a Max Planck Institute workshop in April and May 2026. Research frontier Checked automatically 91% Not in cards
LiveMathematicianBench Models answer multiple choice questions about theorems from arXiv papers published after their training cutoff. Wrong options follow plausible but invalid proof sketches. Research frontier Checked automatically 45.7% System cards only
ArXivMath Final-answer research-level math problems extracted monthly from recent arXiv papers and filtered to be self-contained. Research frontier Checked automatically 92.8% In use
FrontierMath Erdős Models try to prove or disprove 68 open Erdős problems in Lean 4. A result counts only if the Lean checker accepts the full proof. Research frontier Checked automatically 4.41% Not in cards
HorizonMath Models attack 113 mostly unsolved problems in eight areas of mathematics and the mathematical sciences. A program checks each proposed closed form, construction or bound. Research frontier Checked automatically 11.5% In use
IMProofBench Agents with web search and SageMath write proofs for private PhD-level problems written by mathematicians. Experts grade the proofs, and final-answer subquestions are graded automatically. Research frontier Graded by people 49.3% Not in cards
BrokenArXiv Models are asked to prove statements from recent arXiv papers that were changed to be plausible but false. Full credit goes to models that say the statement is false. Research frontier Graded by a model 83.1% In use
First Proof Third Batch The editors run five AI systems once each on ten unpublished research problems with proofs of up to 12 pages. About 30 referees score correctness, exposition and attribution. The score is average correctness, from negative 1 to 1. Research frontier Graded by people 0.65 Not in cards
Hy-Euler pro Models with tools solve Tencent in-house STEM problems. Task and scoring details are not published. Expert Checked automatically 24.2% In use
First Proof AI systems attempt ten unpublished research problems that arose in the authors' own work. Mathematicians judge whether each proof is correct. Research frontier Graded by people Not in cards
First Proof Second Batch The editors run AI harnesses once each, with 24 hours, on ten unpublished research problems with human proofs of up to eight pages. Referees grade each answer blind. Research frontier Graded by people Not in cards
FrontierMath: Open Problems Models attack unsolved research math problems proposed by working mathematicians. Each problem has a program that checks a proposed solution. Epoch tracks which problems AI has solved. Research frontier Checked automatically Not in cards
Soohak Models solve 439 new research-level and graduate math problems written by mathematicians. A 340-item Challenge subset tests solving and a 99-item Refusal subset tests spotting ill-posed problems. Research frontier Checked automatically Not in cards
TCS-Bench Models write proofs of target results from STOC, FOCS and SODA papers, given the needed context. A verification agent, checked against expert judgments, grades each proof. Research frontier Graded by a model Not in cards