How are evals changing?
Agent tasks made up 6% of the new benchmarks in 2023 and 48% in 2026. Multiple-choice tests fell from 24% to 5%. We count every benchmark in our catalog by the year it came out.
Agent tasks made up 6% of the new benchmarks labs cite in 2023 and 47% in 2026. Multiple-choice tests fell from 34% to 3%. We count every benchmark in our catalog that a lab has named in a launch post by the year it came out.
How are new benchmarks built?
Benchmarks scored by another model went from 10% of the year's new benchmarks in 2023 to 28% in 2026. Those that keep some test items private went from 15% to 35%.
Agent tasks
The model acts over many steps in an environment, a browser or a terminal
Multiple choice
The model picks an answer from a list
Scored by a model
Another model grades the answers, often against a rubric
Private test items
Some or all of the test is kept out of public view, to stop leaks into training data
Source: The Benchmark Almanac catalog
Who makes new benchmarks?
Universities and institutes helped make 37% of the new benchmarks in 2020 and 18% in 2026. Eval orgs went from 0% to 11%, and data vendors from 0% to 4%.
AI labs
Companies and groups that train and release their own models
Universities and institutes
Universities, their labs, and public research institutes
Eval orgs
Groups whose main work is evaluating or ranking AI, such as Vals AI, Epoch AI and METR
Data vendors
Companies that sell training data, expert labeling or raters to labs, such as Scale AI and Mercor
Other companies
AI products and startups that release no models of their own, and companies outside AI
Government and nonprofits
Government institutes, nonprofits, exam boards and open communities
Source: The Benchmark Almanac catalog
How are new benchmarks built?
Benchmarks scored by another model went from 8% of the year's new benchmarks labs cite in 2023 to 36% in 2026. Those that keep some test items private went from 11% to 57%.
Agent tasks
The model acts over many steps in an environment, a browser or a terminal
Multiple choice
The model picks an answer from a list
Scored by a model
Another model grades the answers, often against a rubric
Private test items
Some or all of the test is kept out of public view, to stop leaks into training data
Source: The Benchmark Almanac catalog
Who makes new benchmarks?
Universities and institutes helped make 38% of the new benchmarks labs cite in 2020 and 13% in 2026. Eval orgs went from 0% to 19%, and data vendors from 0% to 8%.
AI labs
Companies and groups that train and release their own models
Universities and institutes
Universities, their labs, and public research institutes
Eval orgs
Groups whose main work is evaluating or ranking AI, such as Vals AI, Epoch AI and METR
Data vendors
Companies that sell training data, expert labeling or raters to labs, such as Scale AI and Mercor
Other companies
AI products and startups that release no models of their own, and companies outside AI
Government and nonprofits
Government institutes, nonprofits, exam boards and open communities
Source: The Benchmark Almanac catalog
What do new benchmarks test?
Coding tests rose from 6% of new benchmarks in 2023 to 15% in 2026, as knowledge tests fell from 38% to 12%.
Select an area to break it down.
Knowledge and language 14% expert or harder in 2026
Math and science 68% expert or harder in 2026
Coding 77% expert or harder in 2026
Agents and tools 35% expert or harder in 2026
Professional work 62% expert or harder in 2026
Vision, audio and robotics 12% expert or harder in 2026
Safety and security 72% expert or harder in 2026
The last half year runs through November 21, so its count is partial.
Source: The Benchmark Almanac catalog
How fast do benchmarks saturate?
Ten months after release, 13% of the benchmarks from 2023 had a score past 85%. Of those from 2026, 26% did. Each line follows the benchmarks that came out in one year, from release to the first score past 85%.
Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
How much of the catalog is past 85%?
Today 26% of the benchmarks with scores have a best score past 85%, and 32% of the ones labs featured in launch posts. A year ago those shares were 21% and 33%.
Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
Inspired by Ethan Mollick’s SatBench. Pick a day on the Time machine to see the leaders then.
Which areas saturate first?
Math and science benchmarks saturate first: 42% of them have a score past 85%. Coding trails at 20%. Each chart is the share of an area's benchmarks with a score past 85%, at the end of each half year.
Knowledge and language
Math and science
Coding
Agents and tools
Professional work
Vision, audio and robotics
Safety and security
Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
How fast do benchmarks saturate?
Nine months after release, 9% of the benchmarks from 2023 had a score past 85%. Of those from 2026, 23% did. Each line follows the benchmarks labs cite that came out in one year, from release to the first score past 85%.
Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
How much of the catalog is past 85%?
Today 32% of the benchmarks labs have featured in launch posts have a best score past 85%, against 33% a year ago.
Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
Inspired by Ethan Mollick’s SatBench. Pick a day on the Time machine to see the leaders then.
Which areas saturate first?
Math and science benchmarks saturate first: 55% of them have a score past 85%. Professional work trails at 14%. Each chart is the share of an area's benchmarks labs cite with a score past 85%, at the end of each half year.
Knowledge and language
Math and science
Coding
Agents and tools
Professional work
Vision, audio and robotics
Safety and security
Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
How this is counted
The catalog holds 6,423 benchmarks with a release month. We find recent benchmarks more completely than old ones, so the number per year rises faster than the field did. The shares are less affected, and each point's tooltip shows how many benchmarks it rests on. The current year isn't over, so its point is hollow.
"Only benchmarks labs cite" keeps the 1,104 benchmarks that at least one lab named in a launch post or the documents published with it. It applies to every chart but "What do new benchmarks test?", which counts the whole catalog.
Format, scoring and test access come from each benchmark's entry, the same fields the benchmarks table filters on. A benchmark with no value for a field is left out of that chart.
Makers come from each benchmark's credit. AI labs are the labs in our lab list that release models. Universities and institutes are named as such. Eval orgs, data vendors, other companies, government and nonprofits are sorted by hand for the makers with the most benchmarks. A benchmark can count under two kinds. Independent researchers made too few to chart. 39% of the new benchmarks from 2026 have only makers we haven't sorted yet.
A benchmark saturates here when any score we know of passes 85%, from a leaderboard, Epoch AI or a lab's own report. A model's score counts from the day the model came out, so a benchmark that older models already passed counts as saturated on day one. Benchmarks reach different ages, so each line is a survival estimate: every month, of the benchmarks old enough and not yet past 85%, the share that passed. A line stops when fewer than 15 benchmarks are left to follow. The comparison above uses month 10, the latest month every line reaches.
More insights
-
What labs report
In the first half of 2024, 34% of the benchmarks labs named had come out in the year before the release. In the second half of 2026 so far, 63% had.
-
How fast is the frontier moving?
The best open model today, Kimi K3.1 Pro, scores what the best closed model scored seven months ago. Since 2024 the median gap has been six months.
-
How do new models get tested?
Of the benchmarks labs named in launch posts since 2024, an official leaderboard or Epoch AI has scored the same model on 14%.