Query the almanac
Every score, benchmark, model and lab document we track sits in one SQLite database. Query it here in your browser, or download it.
| app | benchmarks | names |
|---|---|---|
| Microsoft Excel | 24 | Turing Agentic Environments, ATLAS-Finance, BankerToolBench, BankerToolBench v1.1, DesignArena Spreadsheet, ExcelFixBench, Finance Modeling, Finance Modeling Pro, Finch (FinWorkBench), GDP.xlsx, Hebbia Financial Services Benchmark, Hy-FinmodelBench v2, Internal Finance Modeling (Moonshot), Investment Banking Modeling Tasks (OpenAI), MBABench, MBABench 2.0, MEWC, MEWC v2, SpreadsheetBench, SpreadsheetBench Verified, Excel Modeling Benchmark, Excel Modeling Benchmark v1.1, Vals Finance Index, Vals Index v2 |
| Blender | 8 | BlenderBench (Refresh), BlenderWorld, BVB, CineRecon, DesignArena Blender, Frontier3DBench, QwenBlenderBench, VibePrevisBench |
| OpenClaw | 7 | ClawProBench, FORTE, Kimi Claw 24/7 Bench, MM-ClawBench, QwenClawBench, SuperCLUE-OpenClaw, ZClawBench |
| Godot | 5 | DesignArena Godot Game Dev, GameCraft-Bench, GameDevBench, GameForgeBench, SWE-Game |
| Microsoft PowerPoint | 4 | BankerToolBench, BankerToolBench v1.1, PPTCoEditBench, PPTFixBench |
| Microsoft Word | 4 | BankerToolBench, BankerToolBench v1.1, MissionReport, WordEditBench |
| Unreal Engine | 4 | Code4Scene, Frontier3DBench, GameEngineBench, VoxelCodeBench |
| CadQuery | 3 | BenchCAD 2.0, BenchCAD 2.1, BenchCAD 2.1-beta |
| KiCad | 3 | Cua-Bench EDA (KiCad v1), PCBBench (pcbbench.org), PCBWorld-Bench |
| Lean | 3 | CombiBench, FrontierMath Erdős, LeanDojo Benchmark |
| Roblox Studio | 3 | OpenGameEval, OpenGameEval Expanded Eval Set, OpenGameEval Debug Evals |
| Salesforce | 3 | CRMArena, CRMArena-Pro, SCUBA |
| WorkBuddy | 3 | MdCoEditBench, WorkBuddy User Blind Evaluation, WorkIndex |
| dbt | 3 | ADE-bench, data-eng-bench, Spider2-V |
| Feishu | 2 | Feishu Office v1, SuperCLUE-LarkClaw |
| FreeCAD | 2 | Parametric CAD Bench, Parametric CAD Bench V1 |
| Grafana | 2 | o11y-bench, ORCA-bench |
| Kubernetes | 2 | ITBench, SREGym-Lite |
| Linear | 2 | Dojo Mini Bench, MEMTRACK |
| 2 | Browser Use LinkedIn, Dojo Mini Bench | |
| Odoo | 2 | ApprenticeBench, ERP-Bench |
| OpenFOAM | 2 | AutoCFD Incompressible Flow Benchmark, FoamBench V2 |
| SageMath | 2 | ArXivMath, IMProofBench |
| ServiceNow | 2 | WorkArena, WorkArena++ |
| Slack | 2 | Company Agent Bench, MEMTRACK |
Data: Benchmark Almanac (benchmarkalmanac.com), from the original source of each score. How to credit it
How to credit this data
- Credit Benchmark Almanac and link to benchmarkalmanac.com.
- Credit the source of each score you use. The boards table names each leaderboard and its terms.
- Some sources allow only non-commercial use, such as the SWE-bench boards (CC BY-NC 4.0) and MathArena’s datasets (CC BY-NC-SA 4.0). Filter on the noncommercial column before any commercial use.
Zhou, B. (2026). Benchmark Almanac. Retrieved November 21, 2026, from https://www.benchmarkalmanac.com/
BibTeX
@misc{zhou2026benchmarkalmanac,
author = {Zhou, Brian},
title = {Benchmark Almanac},
year = {2026},
url = {https://www.benchmarkalmanac.com/},
note = {Accessed November 21, 2026}
} Download the database
- benchmark-almanac.db.gz 21.7 MB, gzipped
- benchmark-almanac.db 110.6 MB
Open it with sqlite3, Datasette, DuckDB or any other SQLite tool. We rebuild it each time the site updates.
gunzip benchmark-almanac.db.gz sqlite3 benchmark-almanac.db "select * from about"
Our code is under the MIT License, which doesn’t cover the data. Each data source keeps its own terms, listed in boards.terms and abstracts.license. Epoch AI’s scores are CC BY 4.0.
What’s in it
12 tables and 2 views. The scores table has one row per run, so a model can appear several times on one benchmark. Board scores on models we track are dated by the model’s release, not by when the board ran them. Lab numbers are dated by their document.
about 10 rows What this file is, when we built it, and how to credit it. One row per key.
| key | text | name, url, built, data_as_of, credit, how_to_credit, citation, bibtex, license or dates |
| value | text | The text for that key. |
labs 110 rows The labs and companies that make models.
| id | text | Lab id, as in the site’s addresses. |
| name | text | Lab name. |
| country | text | Country code. |
| url | text | Website. |
| aliases | text | Other names, as a JSON array. |
models 1,488 rows One row per model snapshot.
| id | text | Model id. |
| name | text | Model name. |
| version | text | Snapshot label when a lab shipped several versions under one name. |
| lab | text | Lab id (labs.id). |
| family | text | Model family. |
| released | text | Release date, YYYY-MM-DD. |
| weights | text | open or closed. |
| tier | text | flagship, mid, small or open-weights. |
| card_url | text | The lab’s own card for the model. |
| epoch_name | text | The model’s name in Epoch AI’s data. |
| aliases | text | Other names, as a JSON array. |
benchmarks 7,002 rows Every benchmark we catalog. Versions of one test share a family.
| id | text | Benchmark id. |
| name | text | Benchmark name. |
| aliases | text | Other names, as a JSON array. |
| family | text | Shared by every version of one test. |
| version | text | Version label within the family. |
| area | text | One of eight areas: knowledge, reasoning, code, agents, work, multimodal, safety or other. |
| category | text | Category id. |
| category_name | text | Category name. |
| subcategory | text | Finer category id. |
| level | text | Who could solve the tasks: everyday, school, university, competition, expert or frontier. |
| scoring | text | How a score is decided: verified, model-judged, expert-graded, preference, head-to-head or outcome. |
| format | text | What the model produces. |
| org | text | Who made it. |
| released | text | Release month, YYYY-MM. |
| added | text | The day it joined the almanac, for records added since November 2026. |
| unit | text | The unit of its scores: %, elo, score, index or usd. |
| better | text | higher or lower. |
| access | text | public, private or both. |
| confidence | text | How sure we are of the record: verified, likely or unverified. |
| description | text | What it tests, in our words. |
| note | text | Caveats and findings. |
| successor | text | The benchmark that replaced it. |
| repo | text | Code repository. |
| github | text | owner/name of a GitHub repository (repos.repo). |
| dataset | text | Dataset link. |
| leaderboard | text | Leaderboard link. |
| paper | text | Paper link. |
| site | text | Website. |
| edi | real | Epoch Difficulty Index, on the ECI scale. |
| epoch_name | text | The benchmark’s name in Epoch AI’s data. |
| refreshes | text | Dated question refreshes that kept one record, as JSON. |
| changes | text | Dated changes that alter what a score means, as JSON. |
facets 101,983 rows Each benchmark’s facets, one row per value (data/facets.json): what it measures, its risk and behavior, subject, language, input, output, interaction, environment, item source, refresh and who runs it, plus the occupations and human time a benchmark states. Allowed values are in data/facets-codebook.json.
| benchmark | text | Benchmark id (benchmarks.id). |
| facet | text | construct, risk, propensity, subject, language, input, output, interaction, environment, source, refresh, runner, occupations or humanTime. |
| value | text | One value. Environments may name an app or game after a colon, as application:Blender. Subjects are OECD field codes such as 1.6, and languages ISO 639 codes. |
boards 4,766 rows The official leaderboards we read, one per benchmark, with the terms that apply to each.
| benchmark | text | Benchmark id. |
| url | text | The page people read. |
| feed | text | The file or endpoint we read. |
| host | text | Who runs the board, when it isn’t the benchmark’s makers. |
| headline | text | How the importer made the headline score, when the source gives no overall score. |
| paper | integer | 1 when the scores come from a fixed table in the benchmark’s paper, which gains no newer models. |
| terms | text | The terms that apply to reading and reusing it. |
| license | text | A Creative Commons license the terms name, when there is one. |
| noncommercial | integer | 1 when the terms allow only non-commercial use. |
| retrieved | text | The day we last read it. |
| rows | integer | Rows it returned. |
| unmatched | integer | Board names we could not match to a model. |
| cost_per | text | What a cost covers: task, or run for the whole test set. |
| chart | text | The board’s own chart, as JSON. |
| parts | text | The parts of the benchmark the board scores on their own, as JSON: metric key to name, group and unit. Each row keeps its part scores in scores.metrics under these keys. |
| last_failed | text | The day reading it last failed, if it did. |
scores 202,563 rows Every score from every source, one row per run. A model can have several rows on one benchmark.
| id | integer | Row number. |
| benchmark | text | Benchmark id. |
| model | text | Model id, or empty for an unlisted entry. |
| entry_name | text | The board’s own name for an unlisted entry (a fine-tune, agent or product we don’t track). |
| entry_kind | text | model, or unlisted. Unlisted entries never count toward leaders on the site. |
| source | text | leaderboard, epoch (Epoch AI’s own run), collected (a score Epoch AI collected elsewhere) or card (a lab’s own document). |
| via | text | For a collected score: board, lab, paper or unknown. |
| from_source | text | For a collected score: the source Epoch AI names. |
| independent | integer | 1 for a leaderboard run, an Epoch AI run, or a board Epoch AI copied. |
| claim | integer | 1 for a lab’s own number or a claim Epoch AI collected. |
| site_hidden | integer | 1 for an Epoch AI copy of a board we read directly. The site leaves these out. |
| setting | text | Effort level, tools or another way of running the whole benchmark. |
| harness | text | Agent, scaffold or harness, when the source names one. |
| value | real | The score, in the unit column. Percentages run from 0 to 100. |
| unit | text | The score’s unit. |
| date | text | For a board row on a model, the model’s release date. For an unlisted entry, the date the board gives. For a lab number, the document’s date. |
| cost | real | US dollars per task as the board prices it, or per run where boards.cost_per says run. Compare within one board only. |
| tokens | real | Output tokens per task, when the board reports them. |
| metrics | text | Other numbers the board reports for the run, as JSON. |
| card | text | For a lab number, the document id (cards.id). |
| license | text | The license of this score where we know it: CC BY 4.0 for Epoch AI, or the board’s Creative Commons license. |
| noncommercial | integer | 1 when the source allows only non-commercial use. |
cards 892 rows Lab documents: launch posts, system cards, model cards and technical reports.
| id | text | Document id. |
| kind | text | launch (a launch post) or system-card (a longer document). |
| doc | text | For a longer document: system card, model card, technical report or addendum. |
| lab | text | Lab id. |
| model | text | Model id. |
| date | text | Publication date. |
| title | text | Title. |
| url | text | Link. |
| tier | text | The model’s tier. |
| release | text | For a longer document, the launch post it goes with. |
| notes | text | Notes on how we read it. |
card_results 17,122 rows Every result in every lab document, as the lab wrote it.
| card | text | Document id (cards.id). |
| benchmark | text | Benchmark id. |
| score | real | The number, or empty when the document shows no number. |
| unit | text | The unit as written. |
| as_written | text | The row label or text the number came from. |
| section | text | For a longer document: capabilities, safety, bio-chem, cyber and so on. |
eci 257 rows Epoch AI’s Capabilities Index (CC BY 4.0), one score per model.
| model | text | Model id. |
| eci | real | The index. |
| low | real | Lower end of the interval. |
| high | real | Upper end of the interval. |
abstracts 1,733 rows The abstract of each benchmark’s own paper, in its authors’ words, under the license each row names.
| benchmark | text | Benchmark id. |
| title | text | Paper title. |
| url | text | Paper link. |
| source | text | arxiv, acl, crossref or openreview. |
| ref | text | How to name the paper, such as arXiv:2406.19314. |
| license | text | CC0 1.0 or CC BY 4.0. |
| fetched | text | The day we read it. |
| text | text | The abstract. |
repos 1,399 rows Facts about each benchmark’s GitHub repository.
| repo | text | owner/name. |
| stars | integer | Stars. |
| license | text | The repository’s license. |
| pushed | text | Last push. |
| archived | integer | 1 when archived. |
model_scores view, 97,049 rows Scores on tracked models with benchmark, model and lab names, leaving out the copies the site hides.
select s.benchmark, b.name as benchmark_name, b.area, s.model, m.name as model_name, m.lab, l.name as lab_name, m.released, s.source, s.independent, s.claim, s.value, s.unit, b.better, s.setting, s.harness, s.date, s.cost, s.tokens, s.card from scores s join benchmarks b on b.id = s.benchmark join models m on m.id = s.model left join labs l on l.id = m.lab where s.entry_kind = 'model' and s.site_hidden = 0
best_independent view, 59,880 rows Each model’s best independent score on each benchmark, the way the site ranks a board: best setting and harness.
select benchmark, benchmark_name, area, model, model_name, lab, lab_name, value, unit, better, source, setting, harness, date
from (
select ms.*, row_number() over (
partition by ms.benchmark, ms.model
order by case when ms.better = 'lower' then ms.value else -ms.value end
) as rn
from model_scores ms
where ms.independent = 1
)
where rn = 1 Of the 202,563 scores, 99,720 are on models we track. The rest are unlisted entries that boards show under their own names.