Epoch AI
We track 20 benchmarks that Epoch AI made or helped make.
2 in use 3 with public code 16 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| FrontierMath Unpublished research-level math problems with automatically verifiable answers. Tier 4 is the hardest tier. | Research mathematics | Nov 2024 | 52.4% | Fading |
| FrontierMath Tier 4 Models solve research-level math problems that take experts days, submitting answers through a Python function. Each answer is verified automatically as correct or incorrect. | Research mathematics | Jul 2025 | 39.6% | Fading |
| FrontierMath Tiers 1-3 v2 Models solve 295 unpublished math problems from undergraduate to advanced graduate level and submit answers that code checks automatically. Scored by share correct. | Research mathematics | Jun 2026 | 95.1% | In use |
| FrontierMath Tier 4 v2 Models solve 43 unpublished research-level math problems and submit answers that code checks automatically. Scored by share correct. | Research mathematics | Jun 2026 | 100% | In use |
| FrontierMath Erdős Models try to prove or disprove 68 open Erdős problems in Lean 4. A result counts only if the Lean checker accepts the full proof. | Research mathematics | Sep 2026 | 10.3% | System cards only |
| Epoch Automation Reports Agents in coding harnesses do real Epoch AI work tasks such as graphic design, data insights and research design. Epoch staff grade outputs against the rubric they use for employees. | Simulated companies | Oct 2026 | 81.7% | System cards only |
| Epoch AI no-CoT serial reasoning steps How many serial reasoning steps of fixed difficulty a model can do in a single output token, without a chain of thought, at 25% above chance. | Deception and alignment | Nov 2026 | 12.3 | System cards only |
| Epoch Capabilities Index An index that fits model capability and benchmark difficulty from scores on over 50 benchmarks in Epoch AI's hub, using item response theory. Scores read like Elo ratings. | Composite indices | Oct 2025 | 174.0 | System cards only |
| OTIS Mock AIME 2024-2025 Models solve 45 problems from three OTIS Mock AIME exams written by students of the OTIS program. Integer answers are scored by exact match. | Competition problems with answers | Dec 2024 | 100% | Not in cards |
| Chess Puzzles Models find the single best next move in 100 chess puzzles that Epoch generated with a chess engine. Scored by the share solved. | Puzzles and chess | Dec 2025 | 79% | Not in cards |
| Mystery Game Puzzles 100 mid-game positions from an undisclosed puzzle variant of a well-known game. Model picks the single best move, checked against an answer key. | Puzzles and chess | Jul 2026 | 95% | Not in cards |
| Cyber ECI Epoch estimates model capability from cybersecurity benchmark scores and places the result on the general index scale. | Composite indices | 171.1 | Not in cards | |
| Furniture Assembly Benchmark Model gets an IKEA manual and a photo of a partly built item, then names the faulty step or next step. Accuracy on 60 photos. | Spatial reasoning and counting | Sep 2026 | 90% | Not in cards |
| MirrorCode Agent rebuilds a whole CLI program from scratch using only the original binary as an oracle. Solved only if 100% of end-to-end tests pass. | Building whole projects | Jun 2026 | 97.8% | Not in cards |
| EBR-bench Agent plays the board game Earthborne Rangers 10-30 times in a row. Scored on objectives completed (of 21) to test learning from experience. | Puzzles and chess | Jul 2026 | 92.4% | Not in cards |
| SpireClimb Agents play Slay the Spire 1 and 2 at max difficulty through a computer-use harness on fixed seeds. Score is the mean win rate. | Video games | Oct 2026 | 41.3% | Not in cards |
| SpireClimb StS1 A20 Heart Agents play Slay the Spire 1 through its graphical interface at Ascension 20 and must defeat the Heart. Scored by win rate over fixed seeds. | Video games | Oct 2026 | 33.3% | Not in cards |
| SpireClimb StS2 A10 Agents play Slay the Spire 2 at Ascension 10 through screenshots and mouse and keyboard, 15 runs per character. Scoring is the win rate. | Video games | Oct 2026 | 49.3% | Not in cards |
| FrontierMath: Open Problems Models attack unsolved research math problems proposed by working mathematicians. Each problem has a program that checks a proposed solution. Epoch tracks which problems AI has solved. | Research mathematics | Jan 2026 | Not in cards | |
| InnovationEval Agents with thousands of GPU hours try to discover an ML technique that matches a recent human paper they have not seen. The score is the share of that paper's gain the agent reaches. | AI research tasks | Oct 2026 | Not in cards |