Benchmark Almanac

Query the almanac

Every score, benchmark, model and lab document we track sits in one SQLite database. Query it here in your browser, or download it.

202,563 scores 7,002 benchmarks 1,488 models 892 lab documents Data as of November 21st, 2026
Ctrl Enter

Your first query loads the database (21.7 MB).

25 rows

appbenchmarksnames
Microsoft Excel24Turing Agentic Environments, ATLAS-Finance, BankerToolBench, BankerToolBench v1.1, DesignArena Spreadsheet, ExcelFixBench, Finance Modeling, Finance Modeling Pro, Finch (FinWorkBench), GDP.xlsx, Hebbia Financial Services Benchmark, Hy-FinmodelBench v2, Internal Finance Modeling (Moonshot), Investment Banking Modeling Tasks (OpenAI), MBABench, MBABench 2.0, MEWC, MEWC v2, SpreadsheetBench, SpreadsheetBench Verified, Excel Modeling Benchmark, Excel Modeling Benchmark v1.1, Vals Finance Index, Vals Index v2
Blender8BlenderBench (Refresh), BlenderWorld, BVB, CineRecon, DesignArena Blender, Frontier3DBench, QwenBlenderBench, VibePrevisBench
OpenClaw7ClawProBench, FORTE, Kimi Claw 24/7 Bench, MM-ClawBench, QwenClawBench, SuperCLUE-OpenClaw, ZClawBench
Godot5DesignArena Godot Game Dev, GameCraft-Bench, GameDevBench, GameForgeBench, SWE-Game
Microsoft PowerPoint4BankerToolBench, BankerToolBench v1.1, PPTCoEditBench, PPTFixBench
Microsoft Word4BankerToolBench, BankerToolBench v1.1, MissionReport, WordEditBench
Unreal Engine4Code4Scene, Frontier3DBench, GameEngineBench, VoxelCodeBench
CadQuery3BenchCAD 2.0, BenchCAD 2.1, BenchCAD 2.1-beta
KiCad3Cua-Bench EDA (KiCad v1), PCBBench (pcbbench.org), PCBWorld-Bench
Lean3CombiBench, FrontierMath Erdős, LeanDojo Benchmark
Roblox Studio3OpenGameEval, OpenGameEval Expanded Eval Set, OpenGameEval Debug Evals
Salesforce3CRMArena, CRMArena-Pro, SCUBA
WorkBuddy3MdCoEditBench, WorkBuddy User Blind Evaluation, WorkIndex
dbt3ADE-bench, data-eng-bench, Spider2-V
Feishu2Feishu Office v1, SuperCLUE-LarkClaw
FreeCAD2Parametric CAD Bench, Parametric CAD Bench V1
Grafana2o11y-bench, ORCA-bench
Kubernetes2ITBench, SREGym-Lite
Linear2Dojo Mini Bench, MEMTRACK
LinkedIn2Browser Use LinkedIn, Dojo Mini Bench
Odoo2ApprenticeBench, ERP-Bench
OpenFOAM2AutoCFD Incompressible Flow Benchmark, FoamBench V2
SageMath2ArXivMath, IMProofBench
ServiceNow2WorkArena, WorkArena++
Slack2Company Agent Bench, MEMTRACK

Data: Benchmark Almanac (benchmarkalmanac.com), from the original source of each score. How to credit it

How to credit this data

  • Credit Benchmark Almanac and link to benchmarkalmanac.com.
  • Credit the source of each score you use. The boards table names each leaderboard and its terms.
  • Some sources allow only non-commercial use, such as the SWE-bench boards (CC BY-NC 4.0) and MathArena’s datasets (CC BY-NC-SA 4.0). Filter on the noncommercial column before any commercial use.
Cite the almanac

Zhou, B. (2026). Benchmark Almanac. Retrieved November 21, 2026, from https://www.benchmarkalmanac.com/

BibTeX
@misc{zhou2026benchmarkalmanac,
  author = {Zhou, Brian},
  title = {Benchmark Almanac},
  year = {2026},
  url = {https://www.benchmarkalmanac.com/},
  note = {Accessed November 21, 2026}
}

Download the database

Open it with sqlite3, Datasette, DuckDB or any other SQLite tool. We rebuild it each time the site updates.

gunzip benchmark-almanac.db.gz
sqlite3 benchmark-almanac.db "select * from about"

Our code is under the MIT License, which doesn’t cover the data. Each data source keeps its own terms, listed in boards.terms and abstracts.license. Epoch AI’s scores are CC BY 4.0.

What’s in it

12 tables and 2 views. The scores table has one row per run, so a model can appear several times on one benchmark. Board scores on models we track are dated by the model’s release, not by when the board ran them. Lab numbers are dated by their document.

about 10 rows What this file is, when we built it, and how to credit it. One row per key.
key text name, url, built, data_as_of, credit, how_to_credit, citation, bibtex, license or dates
value text The text for that key.
labs 110 rows The labs and companies that make models.
id text Lab id, as in the site’s addresses.
name text Lab name.
country text Country code.
url text Website.
aliases text Other names, as a JSON array.
models 1,488 rows One row per model snapshot.
id text Model id.
name text Model name.
version text Snapshot label when a lab shipped several versions under one name.
lab text Lab id (labs.id).
family text Model family.
released text Release date, YYYY-MM-DD.
weights text open or closed.
tier text flagship, mid, small or open-weights.
card_url text The lab’s own card for the model.
epoch_name text The model’s name in Epoch AI’s data.
aliases text Other names, as a JSON array.
benchmarks 7,002 rows Every benchmark we catalog. Versions of one test share a family.
id text Benchmark id.
name text Benchmark name.
aliases text Other names, as a JSON array.
family text Shared by every version of one test.
version text Version label within the family.
area text One of eight areas: knowledge, reasoning, code, agents, work, multimodal, safety or other.
category text Category id.
category_name text Category name.
subcategory text Finer category id.
level text Who could solve the tasks: everyday, school, university, competition, expert or frontier.
scoring text How a score is decided: verified, model-judged, expert-graded, preference, head-to-head or outcome.
format text What the model produces.
org text Who made it.
released text Release month, YYYY-MM.
added text The day it joined the almanac, for records added since November 2026.
unit text The unit of its scores: %, elo, score, index or usd.
better text higher or lower.
access text public, private or both.
confidence text How sure we are of the record: verified, likely or unverified.
description text What it tests, in our words.
note text Caveats and findings.
successor text The benchmark that replaced it.
repo text Code repository.
github text owner/name of a GitHub repository (repos.repo).
dataset text Dataset link.
leaderboard text Leaderboard link.
paper text Paper link.
site text Website.
edi real Epoch Difficulty Index, on the ECI scale.
epoch_name text The benchmark’s name in Epoch AI’s data.
refreshes text Dated question refreshes that kept one record, as JSON.
changes text Dated changes that alter what a score means, as JSON.
facets 101,983 rows Each benchmark’s facets, one row per value (data/facets.json): what it measures, its risk and behavior, subject, language, input, output, interaction, environment, item source, refresh and who runs it, plus the occupations and human time a benchmark states. Allowed values are in data/facets-codebook.json.
benchmark text Benchmark id (benchmarks.id).
facet text construct, risk, propensity, subject, language, input, output, interaction, environment, source, refresh, runner, occupations or humanTime.
value text One value. Environments may name an app or game after a colon, as application:Blender. Subjects are OECD field codes such as 1.6, and languages ISO 639 codes.
boards 4,766 rows The official leaderboards we read, one per benchmark, with the terms that apply to each.
benchmark text Benchmark id.
url text The page people read.
feed text The file or endpoint we read.
host text Who runs the board, when it isn’t the benchmark’s makers.
headline text How the importer made the headline score, when the source gives no overall score.
paper integer 1 when the scores come from a fixed table in the benchmark’s paper, which gains no newer models.
terms text The terms that apply to reading and reusing it.
license text A Creative Commons license the terms name, when there is one.
noncommercial integer 1 when the terms allow only non-commercial use.
retrieved text The day we last read it.
rows integer Rows it returned.
unmatched integer Board names we could not match to a model.
cost_per text What a cost covers: task, or run for the whole test set.
chart text The board’s own chart, as JSON.
parts text The parts of the benchmark the board scores on their own, as JSON: metric key to name, group and unit. Each row keeps its part scores in scores.metrics under these keys.
last_failed text The day reading it last failed, if it did.
scores 202,563 rows Every score from every source, one row per run. A model can have several rows on one benchmark.
id integer Row number.
benchmark text Benchmark id.
model text Model id, or empty for an unlisted entry.
entry_name text The board’s own name for an unlisted entry (a fine-tune, agent or product we don’t track).
entry_kind text model, or unlisted. Unlisted entries never count toward leaders on the site.
source text leaderboard, epoch (Epoch AI’s own run), collected (a score Epoch AI collected elsewhere) or card (a lab’s own document).
via text For a collected score: board, lab, paper or unknown.
from_source text For a collected score: the source Epoch AI names.
independent integer 1 for a leaderboard run, an Epoch AI run, or a board Epoch AI copied.
claim integer 1 for a lab’s own number or a claim Epoch AI collected.
site_hidden integer 1 for an Epoch AI copy of a board we read directly. The site leaves these out.
setting text Effort level, tools or another way of running the whole benchmark.
harness text Agent, scaffold or harness, when the source names one.
value real The score, in the unit column. Percentages run from 0 to 100.
unit text The score’s unit.
date text For a board row on a model, the model’s release date. For an unlisted entry, the date the board gives. For a lab number, the document’s date.
cost real US dollars per task as the board prices it, or per run where boards.cost_per says run. Compare within one board only.
tokens real Output tokens per task, when the board reports them.
metrics text Other numbers the board reports for the run, as JSON.
card text For a lab number, the document id (cards.id).
license text The license of this score where we know it: CC BY 4.0 for Epoch AI, or the board’s Creative Commons license.
noncommercial integer 1 when the source allows only non-commercial use.
cards 892 rows Lab documents: launch posts, system cards, model cards and technical reports.
id text Document id.
kind text launch (a launch post) or system-card (a longer document).
doc text For a longer document: system card, model card, technical report or addendum.
lab text Lab id.
model text Model id.
date text Publication date.
title text Title.
url text Link.
tier text The model’s tier.
release text For a longer document, the launch post it goes with.
notes text Notes on how we read it.
card_results 17,122 rows Every result in every lab document, as the lab wrote it.
card text Document id (cards.id).
benchmark text Benchmark id.
score real The number, or empty when the document shows no number.
unit text The unit as written.
as_written text The row label or text the number came from.
section text For a longer document: capabilities, safety, bio-chem, cyber and so on.
eci 257 rows Epoch AI’s Capabilities Index (CC BY 4.0), one score per model.
model text Model id.
eci real The index.
low real Lower end of the interval.
high real Upper end of the interval.
abstracts 1,733 rows The abstract of each benchmark’s own paper, in its authors’ words, under the license each row names.
benchmark text Benchmark id.
title text Paper title.
url text Paper link.
source text arxiv, acl, crossref or openreview.
ref text How to name the paper, such as arXiv:2406.19314.
license text CC0 1.0 or CC BY 4.0.
fetched text The day we read it.
text text The abstract.
repos 1,399 rows Facts about each benchmark’s GitHub repository.
repo text owner/name.
stars integer Stars.
license text The repository’s license.
pushed text Last push.
archived integer 1 when archived.
model_scores view, 97,049 rows Scores on tracked models with benchmark, model and lab names, leaving out the copies the site hides.
select s.benchmark, b.name as benchmark_name, b.area, s.model, m.name as model_name, m.lab, l.name as lab_name,
  m.released, s.source, s.independent, s.claim, s.value, s.unit, b.better, s.setting, s.harness, s.date, s.cost, s.tokens, s.card
from scores s
join benchmarks b on b.id = s.benchmark
join models m on m.id = s.model
left join labs l on l.id = m.lab
where s.entry_kind = 'model' and s.site_hidden = 0
best_independent view, 59,880 rows Each model’s best independent score on each benchmark, the way the site ranks a board: best setting and harness.
select benchmark, benchmark_name, area, model, model_name, lab, lab_name, value, unit, better, source, setting, harness, date
from (
  select ms.*, row_number() over (
    partition by ms.benchmark, ms.model
    order by case when ms.better = 'lower' then ms.value else -ms.value end
  ) as rn
  from model_scores ms
  where ms.independent = 1
)
where rn = 1

Of the 202,563 scores, 99,720 are on models we track. The rest are unlisted entries that boards show under their own names.